Difference between revisions of "ZSeries/Part 1"

From Free Pascal wiki
Jump to navigationJump to search
(moved explanations to part 2)
Line 40: Line 40:
  
 
:''[[ZSeries|Go back to zSeries]]'' — [[ZSeries/Part 2|Onward to Part 2]]
 
:''[[ZSeries|Go back to zSeries]]'' — [[ZSeries/Part 2|Onward to Part 2]]
==pp.pas==
 
:''Main Program, includes '''fpcdefs.inc''', calls units '''cmem, profile, catch, globals, compiler'''''
 
'''pp.pas''' is the main program of the compiler.  We're going to edit this file to create an IBM-370 cross-compiler.  First, we'll decide what is the switch for this, and we'll use I370.  So we'll add that to the source comments to indicate that, by inserting the middle line in the comment block (about line 35):
 
  VIS                generate a compiler for the VIS
 
  I370                generate a compiler for the IBM 370/390/zSeries mainframes
 
  DEBUG              version with debug code is generated     
 
This program includes '''fpcdefs.inc''' so we'll check that later.  We have to add the indication to only select one target compiler, so we'll select for I370 (about line 142, where we'll add everything starting at the sixth line where we check to see if i370 is defined):
 
{$ifdef support_mmx}
 
  {$ifndef i386}
 
    {$fatal I386 switch must be on for MMX support}
 
  {$endif i386}
 
{$endif support_mmx}
 
{$ifdef i370}
 
  {$ifdef CPUDEFINED}
 
    {$fatal ONLY one of the switches for the CPU type must be defined}
 
  {$endif CPUDEFINED}
 
  {$define CPUDEFINED}
 
{$endif i370}
 
The rest of the main program seems to be okay, but we will have to go through and look at all the units that this program uses, which, depending on which options have been set, are or can be: '''<code>cmem, profile, catch, globals,</code> and <code>compiler</code>'''.  The rest of this file seems okay, so we'll save it.
 
 
==fpcdefs.inc==
 
:''Called as include file from '''pp.pas''', called as include file from units '''catch, compiler''' (and virtually all other units), no called units noted''
 
'''fpcdefs.inc''' provides various definitions regarding what processor we're compiling for to most units and many other files.  We need to define the processor, so we'll borrow the generic one, and add or remove items as we need them.  So we'll start with the block beginning with the line <code>{$ifdef generic_cpu}</code> through the line <code>{$endif generic_cpu}</code>.  From later work, I discover I'll have to mark the machine as big endian, so I'll include that.  (Note that the definitions will include things as I discover them, so this may include things I haven't explained.)
 
 
So around line 151, between the lines
 
{$endif mips}
 
 
{$IFDEF MACOS}     
 
We'll update this to the following:
 
{$endif mips}
 
 
{$ifdef i370}
 
  {$define cpu32bit}
 
  {$define cpu32bitaddr}
 
  {$define cpu32bitalu}
 
  {$define cpuflags}
 
  {$define cpuextended}
 
  {$define ENDIAN_BIG}
 
{$endif i370}
 
 
{$IFDEF MACOS}     
 
Otherwise, now, '''fpcdefs.inc''' looks okay.
 
 
==Unit cmem==
 
:''Called from: '''pp.pas''', file located in directory '''rtl/inc''' outside of the ''compiler'' directory, no called units noted, requires external procedures ''' malloc, free, realloc, calloc  ''
 
This is used to bridge to the C memory management library and its functions <code>malloc, free, realloc, calloc</code>. I will probably borrow these from an existing C library or simulate them.  Or this module may be rewritten.  For now, there is nothing I need to do here.
 
 
==Unit profile==
 
:''Called from: '''pp.pas''', file located in directory '''rtl/go32v2''' outside of the ''compiler'' directory, no units noted
 
This is only used for profiling under the Go32 system, so this module is not relevant to our cross-compiler.
 
 
==Unit catch==
 
:''Called from: '''pp.pas''', includes '''fpcdefs.inc''',  no called units noted''
 
'''catch.pas''' deals with handling control-c and segfaults, neither of which we'll really have to worry about since the compiler doesn't run locally on the mainframe, it runs on the user's PC.  So in the absence of anything requiring we deal with this, we'll leave it alone for now.
 
 
==Unit globals==
 
:''Called from: '''pp.pas''', includes '''fpcdefs.inc''', calls unit '''comphook'''''
 
'''globals.pas''', starting at about line 372, has some definitions for specific target processors.  So, again, I'll borrow from <code>{$ifdef GENERIC_CPU}</code> and change as necessary.  At line 424, we have
 
  {$endif mips}
 
so we'll add the 370 after this, and put in the following:
 
  {$endif mips}
 
  {$ifdef i370}
 
        cputype : cpu_i370;
 
        optimizecputype : cpu_i370;
 
        fputype : fpu_i370;
 
  {$endif i370}
 
Note, these values will be defined elsewhere. Note that this unit references a few other units depending on the OS and machine, but the only one relevant to us will be '''<code>comphook</code>''' so we'll put that on the "stack" of units that have to be inspected and possibly edited.
 
 
==Unit compiler==
 
:''Called from: '''pp.pas''', includes '''fpcdefs.inc''', calls units '''fksysutils, sysutils, math, verbose, comphook, systems, cutils, cfileutl, cclasses, globals, options, fmodule, parser, symtable, assemble, link, dbgbase, import, export, tokens, pass_1, wpobase, wpo, cpupara, cpupi, cgcpu, cpunode, cputarg, i_i370, globtype'''''
 
 
Starting at about line 24, this unit lists the units it uses.  Some units depend on other flags, but the units it does always require are: '''<code>verbose, comphook, systems, cutils, cfileutl, cclasses, globals, options, fmodule, parser, symtable, assemble, link, dbgbase, import, export, tokens, pass_1, wpobase, wpo, cpupara, cpupi, cgcpu, cpunode,</code>''' and '''<code>cputarg</code>'''.  These are also added to the "stack" for checking on references.
 
 
At this point I do not know whether or not I need to set the flag '''<code>USE_FAKE_SYSUTILS</code>''', if I do, I only need to add one unit, '''<code>fksysutils</code>''', but if I do need the "real" sysutils unit, then it will automatically include the units '''<code>sysutils, math</code>'''.  So I'll presume all three have to be looked at.  They also go on the stack for checking.  Doing a directory scan, apparently ''every'' architecture does use the <code>sysutils</code> unit, and ''none'' use '''<code>fksysutils</code>''', so I'll presume that to be the case.
 
 
This unit also defines the machine that is the target for this compiler, so we have to add a unit which will define our target architecture.  There are units for every machine, so we have to insert one.  At the code around line 116:
 
{$ifdef nativent}
 
  ,i_nativent
 
{$endif nativent}
 
  ,globtype;   
 
we change to:
 
{$ifdef nativent}
 
  ,i_nativent
 
{$endif nativent}
 
{$ifdef i370}
 
  ,i_i370
 
{$endif i370}
 
  ,globtype;   
 
This means we've now added a new source file which will eventually have to be created, '''i_i370.pas''' for the new unit '''<code>i_i370</code>''', which almost certainly will be in the I370 subdirectory.  Also unit '''<code>globtype</code>''' which is at the end of the list of units used.  The rest looks okay so for the moment we're done with this unit.
 
 
==Unit comphook==
 
:''Called from: '''globals.pas''', includes '''fpcdefs.inc''', no called units noted''
 
For the moment, there does not appear to be anything I need to do here.
 
==Unit fksysutils==
 
This is only used if the particular architecture sets the flag that says it does not use the '''<code>sysutils</code>''' and '''<code>math</code>''' units.  Given that all of them do use them, I will presume for now I do not need to create this file.
 
==Unit sysutils==
 
:''Called from: '''compiler.pas''', '''rtl/objpas/math.pas''' (outside Compiler directory), located in '''rtl/i370''' outside the ''compiler'' directory, includes '''fpcdefs.inc''', no called units noted''
 
This file is a local file for each architecture, so it will be in the rtl subdirectory in its i370 subdirectory.  I'll take a look at a couple of architectures to see what's expected to be included.  It is used for platform dependent calls, is used by math, and if there are none, it will simply be a stub unit.  For the moment, I'll just create a stub file and come back later if I need to.
 
{
 
 
    This file is part of the Free Pascal run time library.
 
    Copyright (c) 2012 by Viridian Development Corporation
 
 
    Sysutils unit for IBM 370/390/zSystem
 
 
    See the file COPYING.FPC, included in this distribution,
 
    for details about the copyright.
 
 
    This program is distributed in the hope that it will be useful,
 
    but WITHOUT ANY WARRANTY; without even the implied warranty of
 
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 
 
  **********************************************************************}
 
unit sysutils;
 
 
interface
 
implementation
 
 
end. 
 
 
==Unit math==
 
:''Called from: '''compiler.pas''', located in '''rtl/objpas''' outside the ''compiler'' directory, includes '''fpcdefs.inc''', calls unit '''sysutils'''''
 
This might need to be changed later.  The IBM 370 series uses a different floating point level and its numeric limits are not the same as IEEE.  For the moment I'll leave this.  Note that when it is compiled, a copy of the PPU needs to be in '''rtl/i370''' (outside of the '''Compiler''' directory).
 
==Unit verbose==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit comphook==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit systems==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit cutils==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit cfileutl==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit cclasses==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit globals==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit options==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit fmodule==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit parser==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit symtable==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit assemble==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit link==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit dbgbase==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit import==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit export==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit tokens==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit pass_1==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit wpobase==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit wpo==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit cpupara==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit cpupi==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit cgcpu==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit cpunode==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit cputarg==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
==Unit globtype==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(To be added later)
 
 
==Unit i_i370==
 
:''Called from: '''compiler.pas''', includes '''fpcdefs.inc''', no called units noted''
 
(This will be created later once I know what has to go into it)
 
{{Stub}}
 

Revision as of 22:05, 14 January 2012

Introduction

Go back to zSeriesOnward to Part 2

My name is Paul Robinson, I am the chief programmer at Viridian Development Corporation, which has decided to develop a cross-compiler version of Free Pascal for the IBM 370/390/zSeries mainframe computer. I decided it would be a learning experience, it would allow me to better understand how the Free Pascal compiler works, and because I didn't particularly want to work with a compiler written in C (such as the GCC Pascal Compiler), I wanted to work with one written in Pascal.

There is an existing more-or-less open source compiler for the 370 architecture which I have a copy of the sources and run-time library, it was modified from an earlier incrementally updated Pascal Compiler called P6 or P7 (depending on which release it was), I think it was P6 when it was on the Decsystem/20 mainframe back in the late 1970s and early 1980s (I have the source to that one too), and may have been P7 by the time it was upgraded for the latest release for Control Data Corporation computers. (When Nicklaus Wirth was creating the Pascal language at ETH in Zurich, that's what they had, so Pascal originally started with Control Data computers. In fact, because of the functionality provided by the existing libraries, Wirth wrote the first Pascal compiler using Fortran. I am not kidding.)

This version of P6/P7 was developed by the Australian Atomic Energy Commission. Only problem with it is it's over 25 years old and only supports Standard Pascal. No objects, no strings, and doesn't even come up to the level of capability of Turbo Pascal 3 for DOS. It's old, and I wanted to work with something more recent. I can, however, borrow from it to figure out how some Pascal keywords are translated into 370 code.

I also should have a copy of the Stanford Pascal Compiler sources (also over 20 years old), and I believe that the CBT tapes archive (a huge set of over 200 mainframe source code magnetic tapes originally collected by Connecticut Bank and Trust) may contain either a Pascal Compiler or some other compiler source I can use. It also contains lots of IBM 360/370 assembly language sources which will be useful. All these resources and others should help in doing this implementation.

Before Getting Started

The normal distribution includes most of the sources including the run-time library but does not include the sources to the compiler itself because most people do not need it and it's about another 40 meg. You need to obtain the zip/tar archive file for the compiler from the download location you're using for the rest of Free Pascal (probably Sourceforge or a mirror) and extract from that archive the compiler directory, and include it with the 2.6.0 source release.

First thing is to start by creating a new directory and copying all files and subdirectories from the Compiler subdirectory (and all files in its subdirectories) to a new directory, in order not to contaminate the pristine sources of the current compiler. This compiler will be a cross-compiler, it will run on a PC and will generate an assembly language file for the 370 Architecture with the use of the standard High-Level Assembler syntax. That file will be uploaded to the target mainframe (real or simulated) and run through the high-level assembler there.

You would also want to copy the rtl directory (which is normally outside of the Compiler directory) because you may need to modify some files there. That will also require an i370 subdirectory for its run-time library, which might be different depending on which mainframe OS is targeted. We'll worry about that later.

There will also be created a new I370 subdirectory within the Compiler directory for all the local files related to that architecture.

Note that the pages here will just basically walk through what was done, if a correction is more than a few lines, the user will be directed to the replacement source file. Once the work is completed a zip file containing all of the new or changed source files will be available.

Issues

There are a number of issues when doing this. The 370 has a number of quirks different from the Wintel architecture or Mac hardware

  • It's big-endian
  • It uses non-IEEE floating point so it may have different limit values
  • While it has more registers (15), about 5 of these are generally not usable due to conventions or hardware requirements
  • The maximum amount of memory you can directly address at one time is much smaller (you can only address about 4K at a time, either as code or data). If you're working with two pieces of data, either may be up to 4K in size that you can work with directly.
  • Depending on whether you target a 370, a 390 or a zSystem you may have access to a 32-bit address space or a 64-bit address space and a much larger area than 4K. Since the target I'm going to be using is a 370, I have to restrict the code to a 4K "page" and 32-bit addressing.
  • There are several different operating systems that could be targeted, either the MUSIC/SP emulator I'm using (not going to be very popular), a program running under VM, a program running on the TSO timesharing system, a program running as a batch job on OS/VS1, a program potentially running as a screen application on the CICS terminal monitor (very similar to how Windows programs work, with a few gotchas), or a program running on Linux/370. This will be dealt with by using generic I/O instructions and having an appropriate run-time library for the particular system.
  • The IBM 370/390/zSystem uses the EBCDIC character set, PCs use ASCII (or Unicode). Unicode support may be available but I won't depend upon it. Where the program has to generate constants, only standard, known characters will be used because they translate fine from ASCII to EBCDIC. If there is anything important where the value matters, hexadecimal constants will be used.

Target Choice

I don't really have access to a real zSystem or 390, I have a 370 simulator running on my PC that has an operating system (MUSIC/SP) and an assembler, so the cross-compiler will be restricted to 370 instructions and 32-bit addressing, with a potential for upgrading this if circumstances permit.

To Begin

This compiler is huge. It's hundreds of source files, and is going to be an enormous task. Where do you start? Well, you start with the main program of the command-line compiler, and you look at it. That file is pp.pas.

Note that line numbers indicated in any source file are from the version 2.60 compiler sources and as such, as lines are added, other line numbers where things were found and changed will increase. So line numbers will be referenced in a file from top to bottom so the references should match. Also, so as not to brand this as "windows centric" since the hope is to build a cross-compiler for I370 that could run on either Windows or Linux, when file names are specified, directory separators will use /.

Note that from this point on, all editing occurs in our "sandbox" directory separate from the original compiler.

Go back to zSeriesOnward to Part 2