Difference between revisions of "FPC JVM/Building"

From Free Pascal wiki
Jump to navigationJump to search
m (Mismatched FPC versions can cause build problems)
(One intermediate revision by the same user not shown)
Line 6: Line 6:
  
 
==Prerequisites==
 
==Prerequisites==
A complete and working installation of FPC 2.6.0, the [ftp://ftp.freepascal.org/pub/fpc/contrib/jvm/fpcjvmutilities.zip FPC JVM utilities], and an installation of the JDK 1.5 or later (with the binaries/jar files of all of them in the path).
+
A complete and working installation of FPC 2.6.0 or later (matching the version of the JVM compiler you want to build), the [ftp://ftp.freepascal.org/pub/fpc/contrib/jvm/fpcjvmutilities.zip FPC JVM utilities], and an installation of the JDK 1.5 or later (with the binaries/jar files of all of them in the path).
  
 
==Obtaining the FPC sources==
 
==Obtaining the FPC sources==
Line 20: Line 20:
 
'''Note''': all of the commands below require that the latest native FPC release for your platform is in the PATH.
 
'''Note''': all of the commands below require that the latest native FPC release for your platform is in the PATH.
  
First you have to build a version of the ''fpcmake'' program that knows about the JVM target (use ''move'' instead of ''mv'' under Windows):
+
Execute the following command in the top level directory of the checked out sources, i.e. in the ''fpc'' directory (replace ''FullPathToInstallLocation'' with the directory where to install the result):
  cd fpc/utils/fpcm
+
  make CROSSOPT="-O2 -g" CPU_TARGET=jvm OS_TARGET=java INSTALL_PREFIX=''FullPathToInstallLocation'' all crossinstall
  mv Makefile Makefile.org
 
  fpcmake -Tall Makefile.fpc.fpcmake
 
  make clean all
 
  mv Makefile.org Makefile
 
 
 
Afterwards, move the generated ''fpcmake(.exe)'' somewhere outside the fpc sources directory.
 
 
 
Execute the following command in the top level directory of the checked out sources, i.e. in the ''fpc'' directory (replace ''FullPathToInstallLocation'' with the directory where to install the result, and ''FullPathToFpcmakeCompiledAbove'' with the directory where you put the ''fpcmake(.exe)'' you compiled above):
 
  make FPCMAKE=''FullPathToFpcmakeCompiledAbove/fpcmake(.exe)'' CROSSOPT="-O2 -g" CPU_TARGET=jvm OS_TARGET=java INSTALL_PREFIX=''FullPathToInstallLocation'' all crossinstall
 
  
 
If you wish to also create a run time library that can be used for Android development, use the following command (replace ''FullPathToInstallLocation'' with the directory where to install the result):
 
If you wish to also create a run time library that can be used for Android development, use the following command (replace ''FullPathToInstallLocation'' with the directory where to install the result):
  make FPCMAKE=''FullPathToFpcmakeCompiledAbove/fpcmake(.exe)'' CROSSOPT="-O2 -g" CPU_TARGET=jvm OS_TARGET=android INSTALL_PREFIX=''FullPathToInstallLocation all crossinstall''
+
  make CROSSOPT="-O2 -g" CPU_TARGET=jvm OS_TARGET=android INSTALL_PREFIX=''FullPathToInstallLocation all crossinstall''
  
 
The generated compiler is called ''ppcjvm''/''ppcjvm.exe''.
 
The generated compiler is called ''ppcjvm''/''ppcjvm.exe''.

Revision as of 13:49, 22 May 2020

Building the JVM compiler

The instructions below are for Unix-like platforms, but by changing the forward slashes to backslashes (and adding .exe after the compiler binary name when copying it) most if not all commands will work on Windows too.

Every sequence of commands below starts from the assumption that you are in the top-level FPC source directory.

Prerequisites

A complete and working installation of FPC 2.6.0 or later (matching the version of the JVM compiler you want to build), the FPC JVM utilities, and an installation of the JDK 1.5 or later (with the binaries/jar files of all of them in the path).

Obtaining the FPC sources

The FPC JVM backend is now part of svn trunk. To check it out using svn:

 svn co http://svn.freepascal.org/svn/fpc/trunk fpc

If the latest official release of FPC, installed on your system, does not match the version in the svn trunk, don't use it to build the JVM. Instead check out the relevant tagged release from the repository!

Building the compiler and RTL

Note: all of the commands below require that the latest native FPC release for your platform is in the PATH.

Execute the following command in the top level directory of the checked out sources, i.e. in the fpc directory (replace FullPathToInstallLocation with the directory where to install the result):

make CROSSOPT="-O2 -g" CPU_TARGET=jvm OS_TARGET=java INSTALL_PREFIX=FullPathToInstallLocation all crossinstall

If you wish to also create a run time library that can be used for Android development, use the following command (replace FullPathToInstallLocation with the directory where to install the result):

make CROSSOPT="-O2 -g" CPU_TARGET=jvm OS_TARGET=android INSTALL_PREFIX=FullPathToInstallLocation all crossinstall

The generated compiler is called ppcjvm/ppcjvm.exe.

In both cases, a ppcjvm (or ppcjvm.exe) binary will be created and installed. The compiler generated by both build processes is the same, so you only need one of them. When compiling programs using this compiler, make sure to specify the correct -Fu parameter to point to the Java/Android units.

Building the fpcjres utility

This step is optional, and is only required for the raw {$r xxx} support. Run the following command in the top level directory of the checked out sources:

make all

The compiled fpcjres utility will be located in utils/fpcres.

Additional tools

Building javapp

The tool use to generate Pascal import headers from Java classes is called javapp. It is written in Java, because it is based on the javap utility from the JDK (version 6).

The source code for javapp can be found under utils/javapp in the FPC source tree. To build it, create a new Eclipse project and import the sources. Then add jgrapht.jar from http://www.cs.sunysb.edu/~algorith/implement/jgrapht/distrib/ to the project (used for topological sorting in order to determine the dependencies between the classes). The main class file is Main.java.

Creating the Java RTL headers

To create the rtl/java/java_sysh.inc and rtl/java/java_sys.inc files, run javapp as follows:

 java -jar javapp.jar -protected java.lang.System java.lang.Object java.io.Serializable java.io.IOException java.io.IIOException java.lang.Readable java.lang.Float java.lang.Double java.lang.Byte java.lang.Short java.lang.Integer java.lang.Long java.lang.Boolean java.lang.String java.lang.StringBuilder java.lang.StringBuffer java.lang.CharSequence java.lang.AbstractStringBuilder java.lang.Appendable java.lang.Number java.lang.IndexOutOfBoundsException java.lang.RuntimeException java.lang.IllegalArgumentException java.lang.IllegalStateException java.lang.UnsupportedOperationException java.lang.Exception java.lang.Throwable java.lang.NoSuchMethodException java.lang.LinkageError java.lang.Error java.lang.Comparable java.lang.Character java.text.Collator java.lang.Cloneable java.lang.Runtime java.lang.ThreadLocal java.math.BigInteger java.util.Comparator java.lang.reflect.Array java.lang.Class java.lang.reflect.GenericDeclaration java.lang.reflect.Type java.lang.reflect.Method java.lang.reflect.AccessibleObject java.lang.reflect.Member java.lang.reflect.AnnotatedElement java.lang.reflect.InvocationTargetException java.lang.reflect.Field java.util.Arrays java.util.Map java.util.HashMap java.util.Set java.util.Collection java.util.AbstractMap java.util.Calendar java.lang.Iterable java.lang.Enum  java.lang.Math java.util.AbstractCollection java.util.AbstractSet java.util.EnumSet java.util.BitSet java.util.Iterator java.nio.Buffer java.nio.ByteBuffer java.nio.CharBuffer java.nio.charset. -i -o java_sys

To create the rtl/java/jdk15.pas and rtl/java/jdk15.inc files, use

 java -jar javapp.jar -x java.lang.System -x java.lang.Object -x java.io.Serializable -x java.io.IOException -x java.io.IIOException -x java.lang.Readable -x java.lang.Float -x java.lang.Double -x java.lang.Float -x java.lang.Double -x java.lang.Byte -x java.lang.Short -x java.lang.Integer -x java.lang.Long -x java.lang.Boolean -x java.lang.String -x java.lang.StringBuilder -x java.lang.StringBuffer -x java.lang.CharSequence -x java.lang.AbstractStringBuilder -x java.lang.Appendable -x java.lang.Number -x java.lang.IndexOutOfBoundsException -x java.lang.RuntimeException -x java.lang.IllegalArgumentException -x java.lang.IllegalStateException -x java.lang.UnsupportedOperationException -x java.lang.Exception -x java.lang.Throwable -x java.lang.LinkageError -x java.lang.Error -x java.lang.Comparable -x java.lang.Character  -x java.text.Collator -x java.lang.Cloneable -x java.lang.Runtime -x java.lang.ThreadLocal -x java.math.BigInteger -x java.util.Comparator -x java.lang.reflect.Array -x java.lang.Class -x java.lang.reflect.GenericDeclaration -x java.lang.reflect.Type -x java.lang.reflect.Method -x java.lang.reflect.AccessibleObject -x java.lang.reflect.Member -x java.lang.reflect.AnnotatedElement -x java.lang.reflect.Field -x java.lang.reflect.InvocationTargetException -x java.util.Arrays -x java.util.Map -x java.util.HashMap -x java.util.Set -x java.util.Collection -x java.util.AbstractMap -x java.util.Calendar -x java.lang.Iterable -x java.lang.Enum -x java.lang.Math -x java.util.AbstractCollection -x java.util.AbstractSet -x java.util.EnumSet -x java.util.BitSet -x java.util.Iterator -x java.nio.Buffer -x java.nio.ByteBuffer -x java.nio.CharBuffer -x java.nio.charset. -a sun. -a com.sun. -a apple. -a java.awt.Dialog -protected java. javax. org. -o jdk15

Creating the Android RTL headers

To create the rtl/android/jvm/java_sysh_android.inc and rtl/android/jvm/java_sys_android.inc files, use (same command line as for creating the equivalent Java RTL includes, except for the additional bootclasspath and extdirs parameters)

 java -jar javapp.jar -bootclasspath /full/path/to/android-sdk/platforms/android-14/android.jar -extdirs . -protected java.lang.System java.lang.Object java.io.Serializable java.io.IOException java.io.IIOException java.lang.Readable java.lang.Float java.lang.Double java.lang.Byte java.lang.Short java.lang.Integer java.lang.Long java.lang.Boolean java.lang.String java.lang.StringBuilder java.lang.StringBuffer java.lang.CharSequence java.lang.AbstractStringBuilder java.lang.Appendable java.lang.Number java.lang.IndexOutOfBoundsException java.lang.RuntimeException java.lang.IllegalArgumentException java.lang.IllegalStateException java.lang.UnsupportedOperationException java.lang.Exception java.lang.Throwable java.lang.NoSuchMethodException java.lang.LinkageError java.lang.Error java.lang.Comparable java.lang.Character java.text.Collator java.lang.Cloneable java.lang.Runtime java.lang.ThreadLocal java.math.BigInteger java.util.Comparator java.lang.reflect.Array java.lang.Class java.lang.reflect.GenericDeclaration java.lang.reflect.Type java.lang.reflect.Method java.lang.reflect.AccessibleObject java.lang.reflect.Member java.lang.reflect.AnnotatedElement java.lang.reflect.InvocationTargetException java.lang.reflect.Field java.util.Arrays java.util.Map java.util.HashMap java.util.Set java.util.Collection java.util.AbstractMap java.util.Calendar java.lang.Iterable java.lang.Enum  java.lang.Math java.util.AbstractCollection java.util.AbstractSet java.util.EnumSet java.util.BitSet java.util.Iterator java.nio.Buffer java.nio.ByteBuffer java.nio.CharBuffer java.nio.charset. -i -o java_sys

To create the rtl/android/jvm/androidr14.pas and rtl/android/jvm/androidr14.inc files, use

 java -jar javapp.jar -bootclasspath /full/path/to/android-sdk/platforms/android-14/android.jar -extdirs . -protected -x java.lang.System -x java.lang.Object -x java.io.Serializable -x java.io.IOException -x java.io.IIOException -x java.lang.Readable -x java.lang.Float -x java.lang.Double -x java.lang.Float -x java.lang.Double -x java.lang.Byte -x java.lang.Short -x java.lang.Integer -x java.lang.Long -x java.lang.Boolean -x java.lang.String -x java.lang.StringBuilder -x java.lang.StringBuffer -x java.lang.CharSequence -x java.lang.AbstractStringBuilder -x java.lang.Appendable -x java.lang.Number -x java.lang.IndexOutOfBoundsException -x java.lang.RuntimeException -x java.lang.IllegalArgumentException -x java.lang.IllegalStateException -x java.lang.UnsupportedOperationException -x java.lang.Exception -x java.lang.Throwable -x java.lang.LinkageError -x java.lang.Error -x java.lang.Comparable -x java.lang.Character  -x java.text.Collator -x java.lang.Cloneable -x java.lang.Runtime -x java.lang.ThreadLocal -x java.math.BigInteger -x java.util.Comparator -x java.lang.reflect.Array -x java.lang.Class -x java.lang.reflect.GenericDeclaration -x java.lang.reflect.Type -x java.lang.reflect.Method -x java.lang.reflect.AccessibleObject -x java.lang.reflect.Member -x java.lang.reflect.AnnotatedElement -x java.lang.reflect.Field -x java.lang.reflect.InvocationTargetException -x java.util.Arrays -x java.util.Map -x java.util.HashMap -x java.util.Set -x java.util.Collection -x java.util.AbstractMap -x java.util.Calendar -x java.lang.Iterable -x java.lang.Enum -x java.lang.Math -x java.util.AbstractCollection -x java.util.AbstractSet -x java.util.EnumSet -x java.util.BitSet -x java.util.Iterator -x java.nio.Buffer -x java.nio.ByteBuffer -x java.nio.CharBuffer -x java.nio.charset. -a sun. -a com.sun. -a apple. -a java.awt. java. javax. org. android. junit. -o androidr14

You will have to make several manual changes to the result before it compiles though:

  • there is a circular dependency involving the AVViewGroup.LayoutParams nested class, which results in javapp being unable to order all classes. You will have to manually extract that class and redeclare it as AVViewGroup_LayoutParams = class external 'android.view' name 'ViewGroup$LayoutParams' (JLObject), and then replace all references to the original class to this new class
  • there is a class with a constant field called CREATE, which results in javapp generating wrong names for its constructors (<init>_). Rename the constant to CREATE_ and the constructors to Create
  • The declaration for AAActionBar.InnerTab is missing, add it as InnerTab = class abstract external 'android.app' name 'Tab' end; inside AAActionBar

Creating interfaces for other Java classes

Example to create a unit called bcel containing all classes under org.apache.bcel that can be found in bcel-6.0-SNAPSHOT.jar:

 java -jar full/path/to/javapp.jar -classpath full/path/to/bcel-6.0-SNAPSHOT.jar -protected -o bcel org.apache.bcel.

The long parameter lists above are due to the split of the classes between the system unit and the jdk15 unit. The extra -classpath parameter is required when dumping classes that are not part of the standard JDK, and has to point to the jar file(s) and/or directory/ies containing the class hierarchy you want to dump. If they are in the current directory, do not forget to explicitly pass "-classpath ."

Important:

  • If an identifier argument to javapp ends in a ".", it is interpreted as a package name, otherwise as a class name (not the '.' at the end of org.apache.bcel. in the example above).
  • In case circular dependencies of inner classes are involved, you will have to sacrifice one of the classes involved in the cycle by passing it using the -a option (anonymous). In that case, this class will be translated as a formal class (like java.awt.Dialog above, which is involved in a circular dependency with an inner class of java.awt.Window).
  • The -i option causes javapp to generate include files, otherwise it generates a standalone unit
  • Pascal does not support the concept of Java-style inner classes, but only that of static nested classes. As a result, Java inner classes are not added to the translated headers.

The javapp source code is unfortunately not very clean. The fact that the javap source code was not very clean to start with did not help, but the situation has not exactly improved...

Building javapp from command-line

It might be that installation and usage Eclipse might not be desirable. You can compile javapp using JDK tools only. JDK is prerequisite, thus you should have it installed anyway.

In order to compile you must call javac and compile all .java files in "fpc\utils\javapp\src\fpc\tools\javapp" directory. As mentioned earlier, sources depend on jgrapht-jdk1.5.jar, thus class path must be specified during compilation.

Once sources are compiled all .class files must be gathered together into a .jar file. Manifest file must declare the entry point class, as well as dependency on jgrapht is specified.

Below is the Windows batch (.bat) script file that performs the compilation. The script is intenteded to be ran from fpc\utils\javapp folder. Directory referencing JDK and jgrapht location should be adjusted acording to your end system configuration.

:: setting up environmental variables.
:: if you'jave JDK bin folder in your "PATH" environmental variables
:: you can remove the reference to jdkbindir at all
:: but make sure that all slash-separtors are also removed
set jdkbindir=C:\Program Files\Java\jdk1.8.0_91\bin

:: this is the directory where jgrapht .jar file is located
:: make sure you remove the slash at the end 
set jgraphtdir=C:\jgrapht-0.7.2

:: This generates a small manifest file template
:: It's needed so, "jar" utility will create a dependency for jgrapht-jdk1.5 library
@echo Class-Path: jgrapht-jdk1.5.jar > manifest.txt

::: Compiling sources using java
:: -d .\ forces "javac" to put all the class files hierarhcy  into the current directory
:: once "javac" finished you should have following file structure created "./fpc/tools/javapp/"
:: the structure is important for .jar file
"%jdkbindir%\javac.exe" ^
  .\src\fpc\tools\javapp\*.java ^
  -d .\ ^
  -cp %jgraphtdir%\jgrapht-jdk1.5.jar

:: Creating .jar file for the application
::  Including manifest file (manifest.txt)
::  Specifying entry point  (fpc.tools.javapp.Main)
::  "fpc" is the folder where .class files are located
"%jdkbindir%\jar" cfme javapp.jar manifest.txt fpc.tools.javapp.Main fpc

copy /Y "%jgraphtdir%\jgrapht-jdk1.5.jar" jgrapht-jdk1.5.jar

:: Clean up time!
:: We don't this file any more
:: /Q is quite mode
@del /Q manifest.txt
:: Remove the directory structure containing .classes. We don't need them, since they now should be part
:: of the .jar file. Removing it. Quietly 
@rmdir /S /Q fpc

As the result you'll have two files javapp.jar and jgrapht-jdk1.5.jar. If you need to move javapp.jar to another directory, make sure you copy jgrapht-jdk1.5.jar as well.

External tools

Building Jasmin

Jasmin is the Java byte code assembler used by the compiler to translate the generated assembler files (*.j) into Java class files (*.class).

Building BCEL

BCEL is a Java byte code verification framework that can be used to look for errors in Java class files. See FPC_JVM/Debugging for usage information.

  • Check out the svn sources using the following command
 svn co http://svn.eu.apache.org/repos/asf/commons/proper/bcel/trunk bcel
  • Before building, apply the patch at ftp://ftp.freepascal.org/pub/fpc/contrib/jvm/bcel-r1157681-returntype.patch to prevent incorrect verification errors about return type mismatches.
  • To build BCEL, you first have to install Apache Maven
  • Once Maven is installed and its mvn binary is in your PATH, run mvn package in the bcel source directory. The first time you do so, Maven will download and configure a massive amount of plugins, which takes quite some time. After building, the new version of BCEL will be available as target/bcel-6.0-SNAPSHOT.jar