ZSeries/Part 2

From Free Pascal wiki
Revision as of 06:45, 15 January 2012 by Rfc1394 (talk | contribs) (more details)
Jump to navigationJump to search
Go back to zSeriesGo Back to Part 1Go Forward to Part 3

To Begin

This compiler is huge. It's hundreds of source files, and is going to be an enormous task. Where do you start? Well, you start with the main program of the command-line compiler, and you look at it. That file is pp.pas.

Note that line numbers indicated in any source file are from the version 2.6.0 compiler sources and as such, as lines are added, other line numbers where things were found and changed will increase. So line numbers will be referenced in a file from top to bottom so the references should match. Also, so as not to brand this as "windows centric" since the hope is to build a cross-compiler for I370 that could run on either Windows or Linux, when file names are specified, directory separators will use /.

Note that from this point on, all editing occurs in our "sandbox" directory separate from the original compiler.

How units and include files are listed

Since this series of articles is intended to be a tutorial on how to add a new architecture to the compiler, files that call other files or units are listed at the top of the section dealing with that file. Where a unit is called by another, that caller is listed. Same for files that are included by other files. Any units the unit or file uses, are also listed. As new units which are referenced are found, they are added to the list of all the units that need to be checked and inspected, and potentially edited, as well as new units that have to be created.

The format of the first line is generally like this (items not used will not appear on the line):

Called from: [file names that call this file as a unit, printed in bold] (or) main program, included by [files that call this file using include], includes [files that this file calls as include files], requires external procedures [non-Pascal procedures this unit uses], calls units [units that this unit or file uses] or "no called units noted" if it does not use any units

If the file is not normally in the Compiler directory that is noted, or if it is in a subdirectory. As items are noted in the source code of that file or unit, they will be mentioned.

pp.pas

main program, includes fpcdefs.inc, calls units cmem, profile, catch, globals, compiler

pp.pas is the main program of the compiler. We're going to edit this file to create an IBM-370 cross-compiler. First, we'll decide what is the switch for this, and we'll use I370. So we'll add that to the source comments to indicate that, by inserting the middle line in the comment block (about line 35):

 VIS                 generate a compiler for the VIS
 I370                generate a compiler for the IBM 370/390/zSeries mainframes
 DEBUG               version with debug code is generated       

This program includes fpcdefs.inc so we'll check that later. We have to add the indication to only select one target compiler, so we'll select for I370 (about line 142, where we'll add everything starting at the sixth line where we check to see if i370 is defined):

{$ifdef support_mmx}
  {$ifndef i386}
    {$fatal I386 switch must be on for MMX support}
  {$endif i386}
{$endif support_mmx}
{$ifdef i370}
  {$ifdef CPUDEFINED}
    {$fatal ONLY one of the switches for the CPU type must be defined}
  {$endif CPUDEFINED}
  {$define CPUDEFINED}
{$endif i370}

The rest of the main program seems to be okay, but we will have to go through and look at all the units that this program uses, which, depending on which options have been set, are or can be: cmem, profile, catch, globals, and compiler. The rest of this file seems okay, so we'll save it.

fpcdefs.inc

Called as include file from pp.pas, called as include file from units catch, compiler, verbose (and virtually all other units), no called units noted

fpcdefs.inc provides various definitions regarding what processor we're compiling for to most units and many other files. We need to define the processor, so we'll borrow the generic one, and add or remove items as we need them. So we'll start with the block beginning with the line {$ifdef generic_cpu} through the line {$endif generic_cpu}. From later work, I discover I'll have to mark the target machine as big endian, so I'll include that. (Note that the definitions will include things as I discover them, so this may include things I haven't explained.)

So around line 151, between the lines

{$endif mips}

{$IFDEF MACOS}      

We'll update this to the following:

{$endif mips}

{$ifdef i370}
  {$define cpu32bit}
  {$define cpu32bitaddr}
  {$define cpu32bitalu}
  {$define cpuflags}
  {$define cpuextended}
  {$define ENDIAN_BIG}
{$endif i370}

{$IFDEF MACOS}      

Otherwise, now, fpcdefs.inc looks okay.

Unit cmem

Called from: pp.pas, file located in directory rtl/inc outside of the compiler directory, requires external procedures malloc, free, realloc, calloc, no called units noted

This is used as a bridge to the C memory management library and its functions malloc, free, realloc, calloc. In some architectures it uses a specific library. I will probably borrow these from an existing C library or simulate them. Or this module may be rewritten. For now, there is nothing I need to do here.

Unit profile

Called from: pp.pas, file located in directory rtl/go32v2 outside of the compiler directory, no units noted

This is only used for profiling under the Go32 system, so this module is not relevant to our cross-compiler.

Unit catch

Called from: pp.pas, includes fpcdefs.inc, no called units noted

catch.pas deals with handling control-c and segfaults. We might have to throw some flags to enable this or to handle it since the compiler doesn't run locally on the mainframe, it runs on the user's PC. Right now, we're just trying to get to the point of being able to create code for the zSeries. So in the absence of anything requiring we deal with this, we'll leave it alone for now.

Unit globals

Called from: pp.pas, includes fpcdefs.inc, calls unit comphook

globals.pas, starting at about line 372, has some definitions for specific target processors. So, again, I'll borrow from {$ifdef GENERIC_CPU} and change as necessary. At line 424, we have

 {$endif mips}

so we'll add the 370 after this, and put in the following:

 {$endif mips}
 {$ifdef i370}
       cputype : cpu_i370;
       optimizecputype : cpu_i370;
       fputype : fpu_i370;
 {$endif i370}

Note, these values will be defined elsewhere. Note that this unit references a few other units depending on the OS and machine, but the only one relevant to us will be comphook so we'll put that on the "stack" of units that have to be inspected and possibly edited.

Unit compiler

Called from: pp.pas, includes fpcdefs.inc, calls units fksysutils, sysutils, math, verbose, comphook, systems, cutils, cfileutl, cclasses, globals, options, fmodule, parser, symtable, assemble, link, dbgbase, import, export, tokens, pass_1, wpobase, wpo, cpupara, cpupi, cgcpu, cpunode, cputarg, i_i370, globtype

Starting at about line 24, this unit lists the units it uses. Some units depend on other flags, but the units it does always require are: verbose, comphook, systems, cutils, cfileutl, cclasses, globals, options, fmodule, parser, symtable, assemble, link, dbgbase, import, export, tokens, pass_1, wpobase, wpo, cpupara, cpupi, cgcpu, cpunode, and cputarg. These are also added to the "stack" for checking on references.

At this point I do not know whether or not I need to set the flag USE_FAKE_SYSUTILS, if I do, I only need to add one unit, fksysutils, but if I do need the "real" sysutils unit, then it will automatically include the units sysutils, math. So I'll presume all three have to be looked at. They also go on the stack for checking. Doing a directory scan, apparently every architecture does use the sysutils unit, and none use fksysutils, so I'll presume that to be the case.

This unit also defines the machine that is the target for this compiler, so we have to add a unit which will define our target architecture. There are units for every machine, so we have to insert one. At the code around line 116:

{$ifdef nativent}
  ,i_nativent
{$endif nativent}
  ,globtype;    

we change to:

{$ifdef nativent}
  ,i_nativent
{$endif nativent}
{$ifdef i370}
  ,i_i370
{$endif i370}
  ,globtype;    

This means we've now added a new source file which will eventually have to be created, i_i370.pas for the new unit i_i370, which almost certainly will be in the I370 subdirectory. Also unit globtype which is at the end of the list of units used. The rest looks okay so for the moment we're done with this unit.

Unit fksysutils

This is only used if the particular architecture sets the flag that says it does not use the sysutils and math units. Given that all of them do use them, I will presume for now I do not need to create this file.

Unit sysutils

Called from: compiler.pas; verbose.pas; rtl/objpas/math.pas (outside Compiler directory) located in rtl/i370 (outside Compiler directory), includes fpcdefs.inc, no called units noted

This file is a local file for each architecture, so it will be in the rtl subdirectory in its i370 subdirectory. I'll take a look at a couple of architectures to see what's expected to be included. It is used for platform dependent calls, is used by units math, verbose, and if there are no platform dependent calls, it will simply be a stub unit. For the moment, I'll just create a stub file and come back later if I need to.

{

    This file is part of the Free Pascal run time library.
    Copyright (c) 2012 by Viridian Development Corporation

    Sysutils unit for IBM 370/390/zSystem

    See the file COPYING.FPC, included in this distribution,
    for details about the copyright.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

 **********************************************************************}
unit sysutils;

interface
implementation

end.

Unit math

Called from: compiler.pas, located in rtl/objpas outside the compiler directory, includes fpcdefs.inc, calls unit sysutils

This might need to be changed later. The IBM 370 series uses a different floating point system and its numeric limits are not the same as IEEE. For the moment I'll leave this alone and come back later. Note that when it is compiled, a copy of the PPU needs to be in rtl/i370 (outside of the Compiler directory).

Unit verbose

Called from: compiler.pas, includes msgtxt.inc, msgidx.inc, uses units sysutils, fksysutl, cutils, globtype, finput, cmsgs

verbose.pas handles generating long ("verbose") error messages. It uses unit sysutils unless USE_FAKE_SYSUTILS is defined, then it uses unit fksysutl instead. If EXTERN_MSG is not defined, it includes msgtxt.inc. Always includes msgidx.inc. Since this is in the compiler, not the user's code I can ignore it for now.

msgtxt.inc

Called as include file from verbose.pas, no called units noted

Causes the entire 253 different error messages to be included in the verbose unit as a block of strings. Since it's in the compiler, not the user's program, it's nothing I have to worry about right now.

msgidx.inc

Called as include file from verbose.pas, no called units noted

This is a list of constants that translate the names of error messages into reference numbers. It is intended to be included into the verbose unit. Again, I can ignore this for now.

Unit comphook

Called from: compiler.pas, includes fpcdefs.inc, uses units sysutils, fksysutl, globtype, finput;

comphook.pas only uses fksysutl if sysutils isn't present. Is a compiler hook to external programs, right now I don't need to do anything with it.

Unit systems

Called from: compiler.pas; cfileutl.pas, includes fpcdefs.inc; systems.inc, calls unit cutils

Relates to information about the target systems supported. Lines 303-305 need to be altered to add the 370 (TsystemCpu will be adjusted to compensate):

      cpu2str : array[TSystemCpu] of string[10] =
           (,'i386','m68k','alpha','powerpc','sparc','vm','ia64','x86_64',
            'mips','arm', 'powerpc64', 'avr', 'mipsel','i370');

At around line 851 we have to indicate the target, replacing the one line

{$endif mips}

to

{$endif mips}
{$ifdef i370}
  default_target(system_i370_mainframe);
{$endif i370}  

systems.inc

Called as include file from: systems.pas, no called units noted

This adds the cpu types available. About line 50, change

            cpu_mipsel                    { 13 }

to

            cpu_mipsel,                   { 13 }
            cpu_i370                      { 14 }    

About line 67, change

           ,asmmode_avr_gas

to

           ,asmmode_avr_gas
           ,asmmode_i370

About line 151, change

            system_powerpc_wii         { 70 }

to

            system_powerpc_wii,        { 70 }
            system_i370_mainframe      { 71 }

About line 185, change

            ,as_i386_nlmcoff

to

            ,as_i386_nlmcoff
            ,as_i370    

Otherwise seems okay for now.

Unit cutils

Called from: compiler.pas; verbose.pas; systems.pas; cfileutl.pas, includes fpcdefs.inc, no called units noted

This provides a number of utility functions. It also requires the target be defined as big endian, although that may only be important if the compiler is running on the target rather than on the cross-compiling host.

Unit cfileutl

Called from: compiler.pas, includes fpcdefs.inc, uses units SysUtils; fksysutl; GlobType; CUtils; CClasses; Systems

(To be added later)

Unit cclasses

Called from: compiler.pas; cfileutl.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit globals

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit options

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit fmodule

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit parser

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit symtable

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit assemble

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit link

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit dbgbase

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit import

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit export

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit tokens

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit pass_1

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit wpobase

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit wpo

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cpupara

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cpupi

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cgcpu

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cpunode

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cputarg

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit globtype

Called from: compiler.pas; verbose.pas; comphook.pas; cfileutl.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit finput

Called from: verbose.pas; comphook.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cmsgs

Called from: verbose.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit i_i370

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(This will be created later once I know what has to go into it)

t_i370.pas

(The cross-compiler instructions say I'll need this. It will be in the systems subdirectory. Will figure out later.) It appears to be used for running the support programs such as the assembler and the linker, if so, it's going to be a stub.

Other files

From the description for porting the compiler, a few other files have to be edited.

  • utils/fpcm/fpcmmain.pp - add a new entry at the end of TCpu and TOs, near line 85:

     TCpu=(
       c_i386,c_m68k,c_powerpc,c_sparc,c_x86_64,c_arm,c_powerpc64,c_avr,c_armeb,
       c_armel,c_mips,c_mipsel,c_mips64,c_mips64el,c_i370
     );
     TOS=(
       o_linux,o_go32v2,o_win32,o_os2,o_freebsd,o_beos,o_haiku,o_netbsd,
       o_amiga,o_atari, o_solaris, o_qnx, o_netware, o_openbsd,o_wdosx,
       o_palmos,o_macos,o_darwin,o_emx,o_watcom,o_morphos,o_netwlibc,
       o_win64,o_wince,o_gba,o_nds,o_embedded,o_symbian,o_nativent,o_iphonesim,
       o_wii,o_i370
     );

About line 81, add the new item to these entries:

     CpuStr : array[TCpu] of string=(
       'i386','m68k','powerpc','sparc','x86_64','arm','powerpc64','avr','armeb',
       'armel', 'mips', 'mipsel', 'mips64', 'mips64el','i370'
     );
     CpuSuffix : array[TCpu] of string=(
       '_i386','_m68k','_powerpc','_sparc','_x86_64','_arm','_powerpc64','avr',
       '_armeb', '_armel', '_mips', '_mipsel', '_mips64', '_mips64el','_i370'
     );
     ppcSuffix : array[TCpu] of string=(
       '386','m68k','ppc','sparc','x86_64','arm','ppc64','avr','armeb', 'armel',
       'mips', 'mipsel', 'mips64', 'mips64el','i370'
     );
     OSStr : array[TOS] of string=(
       'linux','go32v2','win32','os2','freebsd','beos','haiku','netbsd',
       'amiga','atari','solaris', 'qnx', 'netware','openbsd','wdosx',
       'palmos','macos','darwin','emx','watcom','morphos','netwlibc',
       'win64','wince','gba','nds','embedded','symbian','nativent',
       'iphonesim', 'wii','i370'
     );
     OSSuffix : array[TOS] of string=(
       '_linux','_go32v2','_win32','_os2','_freebsd','_beos','_haiku','_netbsd',
       '_amiga','_atari','_solaris', '_qnx', '_netware','_openbsd','_wdosx',
       '_palmos','_macos','_darwin','_emx','_watcom','_morphos','_netwlibc',
       '_win64','_wince','_gba','_nds','_embedded','_symbian','_nativent',
       '_iphonesim','_wii','_i370'
     );

And then this huge table is changed to the following:

     { This table is kept OS,Cpu because it is easier to maintain (PFV) }
     OSCpuPossible : array[TOS,TCpu] of boolean = (
       { os          i386    m68k  ppc    sparc  x86_64 arm    ppc64  avr    armeb  armel  mips   mipsel mips64 misp64el, i370}
       { linux }   ( true,  true,  true,  true,  true,  true,  true,  false, true,  false, false, true,  false,  false,  true),
       { go32v2 }  ( true,  false, false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { win32 }   ( true,  false, false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { os2 }     ( true,  false, false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { freebsd } ( true,  true,  false, false, true,  false, false, false, false, false, false, false, false,  false,  false),
       { beos }    ( true,  false, false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { haiku }   ( true,  false, false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { netbsd }  ( true,  true,  true,  true,  false, false, false, false, false, false, false, false, false,  false,  false),
       { amiga }   ( false, true,  true,  false, false, false, false, false, false, false, false, false, false,  false,  false),
       { atari }   ( false, true,  false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { solaris } ( true,  false, false, true,  true,  false, false, false, false, false, false, false, false,  false,  false),
       { qnx }     ( true,  false, false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { netware } ( true,  false, false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { openbsd } ( true,  true,  false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { wdosx }   ( true,  false, false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { palmos }  ( false, true,  false, false, false, true,  false, false, false, false, false, false, false,  false,  false),
       { macos }   ( false, false, true,  false, false, false, false, false, false, false, false, false, false,  false,  false),
       { darwin }  ( true,  false, true,  false, true,  true,  true,  false, false, false, false, false, false,  false,  false),
       { emx }     ( true,  false, false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { watcom }  ( true,  false, false, false ,false, false, false, false, false, false, false, false, false,  false,  false),
       { morphos } ( false, false, true,  false ,false, false, false, false, false, false, false, false, false,  false,  false),
       { netwlibc }( true,  false, false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { win64   } ( false, false, false, false, true,  false, false, false, false, false, false, false, false,  false,  false),
       { wince    }( true,  false, false, false, false, true,  false, false, false, false, false, false, false,  false,  false),
       { gba    }  ( false, false, false, false, false, true,  false, false, false, false, false, false, false,  false,  false),
       { nds    }  ( false, false, false, false, false, true,  false, false, false, false, false, false, false,  false,  false),
       { embedded }( true,  true,  true,  true,  true,  true,  true,  true,  true , false, false, false, false,  false,  false),
       { symbian } ( true,  false, false, false, false, true,  false, false, false, false, false, false, false,  false,  false),
       { nativent }( true,  false, false, false, false, false, false, false, false, false, false, false, false,  false,  false),
       { iphonesim }( true,  false, false, false, false, false, false, false, false, false, false, false, false, false,  false),
       { wii }     ( false, false, true,  false, false, false, false, false, false, false, false, false, false,  false,  false),
       { i370 }    ( false, false, false, false, false, false, false, false, false, false, false, false, false,  false,  true)
     );


Go back to zSeriesGo Back to Part 1Go Forward to Part 3

An editor has declared this article to be a stub, meaning that it needs more information. Can you help out and add some? If you have some useful information, you can help the Free Pascal Wiki by clicking on the edit box on the left and expanding this page.