Difference between revisions of "ZSeries/Part 1"

From Free Pascal wiki
Jump to navigationJump to search
Line 1: Line 1:
 
=Introduction=
 
=Introduction=
:''[[ZSeries|Go back to zSeries]]''  
+
:''[[ZSeries|Go back to zSeries]]'' — [[ZSeries/Part 2|Onward to Part 2]]
 
My name is Paul Robinson, I am the chief programmer at Viridian Development Corporation, which has decided to develop a cross-compiler version of Free Pascal for the IBM 370/390/zSeries mainframe computer.  I decided it would be a learning experience, it would allow me to better understand how the Free Pascal compiler works, and because I didn't particularly want to work with a compiler written in C (such as the GCC Pascal Compiler), I wanted to work with one written in Pascal.   
 
My name is Paul Robinson, I am the chief programmer at Viridian Development Corporation, which has decided to develop a cross-compiler version of Free Pascal for the IBM 370/390/zSeries mainframe computer.  I decided it would be a learning experience, it would allow me to better understand how the Free Pascal compiler works, and because I didn't particularly want to work with a compiler written in C (such as the GCC Pascal Compiler), I wanted to work with one written in Pascal.   
  
Line 39: Line 39:
 
Note that from this point on, all editing occurs in our "sandbox" directory separate from the original compiler.
 
Note that from this point on, all editing occurs in our "sandbox" directory separate from the original compiler.
  
[[Zseries|Go back to Zseries]] — [[Zseries/Part 2|Go on to Part 2]]
+
:''[[ZSeries|Go back to zSeries]]'' — [[ZSeries/Part 2|Onward to Part 2]]
 
==pp.pas==
 
==pp.pas==
 
:''Main Program, includes '''fpcdefs.inc''', calls units '''cmem, profile, catch, globals, compiler'''''
 
:''Main Program, includes '''fpcdefs.inc''', calls units '''cmem, profile, catch, globals, compiler'''''

Revision as of 21:42, 14 January 2012

Introduction

Go back to zSeriesOnward to Part 2

My name is Paul Robinson, I am the chief programmer at Viridian Development Corporation, which has decided to develop a cross-compiler version of Free Pascal for the IBM 370/390/zSeries mainframe computer. I decided it would be a learning experience, it would allow me to better understand how the Free Pascal compiler works, and because I didn't particularly want to work with a compiler written in C (such as the GCC Pascal Compiler), I wanted to work with one written in Pascal.

There is an existing more-or-less open source compiler for the 370 architecture which I have a copy of the sources and run-time library, it was modified from an earlier incrementally updated Pascal Compiler called P6 or P7 (depending on which release it was), I think it was P6 when it was on the Decsystem/20 mainframe back in the late 1970s and early 1980s (I have the source to that one too), and may have been P7 by the time it was upgraded for the latest release for Control Data Corporation computers. (When Nicklaus Wirth was creating the Pascal language at ETH in Zurich, that's what they had, so Pascal originally started with Control Data computers. In fact, because of the functionality provided by the existing libraries, Wirth wrote the first Pascal compiler using Fortran. I am not kidding.)

This version of P6/P7 was developed by the Australian Atomic Energy Commission. Only problem with it is it's over 25 years old and only supports Standard Pascal. No objects, no strings, and doesn't even come up to the level of capability of Turbo Pascal 3 for DOS. It's old, and I wanted to work with something more recent. I can, however, borrow from it to figure out how some Pascal keywords are translated into 370 code.

I also should have a copy of the Stanford Pascal Compiler sources (also over 20 years old), and I believe that the CBT tapes archive (a huge set of over 200 mainframe source code magnetic tapes originally collected by Connecticut Bank and Trust) may contain either a Pascal Compiler or some other compiler source I can use. It also contains lots of IBM 360/370 assembly language sources which will be useful. All these resources and others should help in doing this implementation.

Before Getting Started

The normal distribution includes most of the sources including the run-time library but does not include the sources to the compiler itself because most people do not need it and it's about another 40 meg. You need to obtain the zip/tar archive file for the compiler from the download location you're using for the rest of Free Pascal (probably Sourceforge or a mirror) and extract from that archive the compiler directory, and include it with the 2.6.0 source release.

First thing is to start by creating a new directory and copying all files and subdirectories from the Compiler subdirectory (and all files in its subdirectories) to a new directory, in order not to contaminate the pristine sources of the current compiler. This compiler will be a cross-compiler, it will run on a PC and will generate an assembly language file for the 370 Architecture with the use of the standard High-Level Assembler syntax. That file will be uploaded to the target mainframe (real or simulated) and run through the high-level assembler there.

You would also want to copy the rtl directory (which is normally outside of the Compiler directory) because you may need to modify some files there. That will also require an i370 subdirectory for its run-time library, which might be different depending on which mainframe OS is targeted. We'll worry about that later.

There will also be created a new I370 subdirectory within the Compiler directory for all the local files related to that architecture.

Note that the pages here will just basically walk through what was done, if a correction is more than a few lines, the user will be directed to the replacement source file. Once the work is completed a zip file containing all of the new or changed source files will be available.

Issues

There are a number of issues when doing this. The 370 has a number of quirks different from the Wintel architecture or Mac hardware

  • It's big-endian
  • It uses non-IEEE floating point so it may have different limit values
  • While it has more registers (15), about 5 of these are generally not usable due to conventions or hardware requirements
  • The maximum amount of memory you can directly address at one time is much smaller (you can only address about 4K at a time, either as code or data). If you're working with two pieces of data, either may be up to 4K in size that you can work with directly.
  • Depending on whether you target a 370, a 390 or a zSystem you may have access to a 32-bit address space or a 64-bit address space and a much larger area than 4K. Since the target I'm going to be using is a 370, I have to restrict the code to a 4K "page" and 32-bit addressing.
  • There are several different operating systems that could be targeted, either the MUSIC/SP emulator I'm using (not going to be very popular), a program running under VM, a program running on the TSO timesharing system, a program running as a batch job on OS/VS1, a program potentially running as a screen application on the CICS terminal monitor (very similar to how Windows programs work, with a few gotchas), or a program running on Linux/370. This will be dealt with by using generic I/O instructions and having an appropriate run-time library for the particular system.
  • The IBM 370/390/zSystem uses the EBCDIC character set, PCs use ASCII (or Unicode). Unicode support may be available but I won't depend upon it. Where the program has to generate constants, only standard, known characters will be used because they translate fine from ASCII to EBCDIC. If there is anything important where the value matters, hexadecimal constants will be used.

Target Choice

I don't really have access to a real zSystem or 390, I have a 370 simulator running on my PC that has an operating system (MUSIC/SP) and an assembler, so the cross-compiler will be restricted to 370 instructions and 32-bit addressing, with a potential for upgrading this if circumstances permit.

To Begin

This compiler is huge. It's hundreds of source files, and is going to be an enormous task. Where do you start? Well, you start with the main program of the command-line compiler, and you look at it. That file is pp.pas.

Note that line numbers indicated in any source file are from the version 2.60 compiler sources and as such, as lines are added, other line numbers where things were found and changed will increase. So line numbers will be referenced in a file from top to bottom so the references should match. Also, so as not to brand this as "windows centric" since the hope is to build a cross-compiler for I370 that could run on either Windows or Linux, when file names are specified, directory separators will use /.

Note that from this point on, all editing occurs in our "sandbox" directory separate from the original compiler.

Go back to zSeriesOnward to Part 2

pp.pas

Main Program, includes fpcdefs.inc, calls units cmem, profile, catch, globals, compiler

pp.pas is the main program of the compiler. We're going to edit this file to create an IBM-370 cross-compiler. First, we'll decide what is the switch for this, and we'll use I370. So we'll add that to the source comments to indicate that, by inserting the middle line in the comment block (about line 35):

 VIS                 generate a compiler for the VIS
 I370                generate a compiler for the IBM 370/390/zSeries mainframes
 DEBUG               version with debug code is generated       

This program includes fpcdefs.inc so we'll check that later. We have to add the indication to only select one target compiler, so we'll select for I370 (about line 142, where we'll add everything starting at the sixth line where we check to see if i370 is defined):

{$ifdef support_mmx}
  {$ifndef i386}
    {$fatal I386 switch must be on for MMX support}
  {$endif i386}
{$endif support_mmx}
{$ifdef i370}
  {$ifdef CPUDEFINED}
    {$fatal ONLY one of the switches for the CPU type must be defined}
  {$endif CPUDEFINED}
  {$define CPUDEFINED}
{$endif i370}

The rest of the main program seems to be okay, but we will have to go through and look at all the units that this program uses, which, depending on which options have been set, are or can be: cmem, profile, catch, globals, and compiler. The rest of this file seems okay, so we'll save it.

fpcdefs.inc

Called as include file from pp.pas, called as include file from units catch, compiler (and virtually all other units), no called units noted

fpcdefs.inc provides various definitions regarding what processor we're compiling for to most units and many other files. We need to define the processor, so we'll borrow the generic one, and add or remove items as we need them. So we'll start with the block beginning with the line {$ifdef generic_cpu} through the line {$endif generic_cpu}. From later work, I discover I'll have to mark the machine as big endian, so I'll include that. (Note that the definitions will include things as I discover them, so this may include things I haven't explained.)

So around line 151, between the lines

{$endif mips}

{$IFDEF MACOS}      

We'll update this to the following:

{$endif mips}

{$ifdef i370}
  {$define cpu32bit}
  {$define cpu32bitaddr}
  {$define cpu32bitalu}
  {$define cpuflags}
  {$define cpuextended}
  {$define ENDIAN_BIG}
{$endif i370}

{$IFDEF MACOS}      

Otherwise, now, fpcdefs.inc looks okay.

Unit cmem

Called from: pp.pas, file located in directory rtl/inc outside of the compiler directory, no called units noted, requires external procedures malloc, free, realloc, calloc

This is used to bridge to the C memory management library and its functions malloc, free, realloc, calloc. I will probably borrow these from an existing C library or simulate them. Or this module may be rewritten. For now, there is nothing I need to do here.

Unit profile

Called from: pp.pas, file located in directory rtl/go32v2 outside of the compiler directory, no units noted

This is only used for profiling under the Go32 system, so this module is not relevant to our cross-compiler.

Unit catch

Called from: pp.pas, includes fpcdefs.inc, no called units noted

catch.pas deals with handling control-c and segfaults, neither of which we'll really have to worry about since the compiler doesn't run locally on the mainframe, it runs on the user's PC. So in the absence of anything requiring we deal with this, we'll leave it alone for now.

Unit globals

Called from: pp.pas, includes fpcdefs.inc, calls unit comphook

globals.pas, starting at about line 372, has some definitions for specific target processors. So, again, I'll borrow from {$ifdef GENERIC_CPU} and change as necessary. At line 424, we have

 {$endif mips}

so we'll add the 370 after this, and put in the following:

 {$endif mips}
 {$ifdef i370}
       cputype : cpu_i370;
       optimizecputype : cpu_i370;
       fputype : fpu_i370;
 {$endif i370}

Note, these values will be defined elsewhere. Note that this unit references a few other units depending on the OS and machine, but the only one relevant to us will be comphook so we'll put that on the "stack" of units that have to be inspected and possibly edited.

Unit compiler

Called from: pp.pas, includes fpcdefs.inc, calls units fksysutils, sysutils, math, verbose, comphook, systems, cutils, cfileutl, cclasses, globals, options, fmodule, parser, symtable, assemble, link, dbgbase, import, export, tokens, pass_1, wpobase, wpo, cpupara, cpupi, cgcpu, cpunode, cputarg, i_i370, globtype

Starting at about line 24, this unit lists the units it uses. Some units depend on other flags, but the units it does always require are: verbose, comphook, systems, cutils, cfileutl, cclasses, globals, options, fmodule, parser, symtable, assemble, link, dbgbase, import, export, tokens, pass_1, wpobase, wpo, cpupara, cpupi, cgcpu, cpunode, and cputarg. These are also added to the "stack" for checking on references.

At this point I do not know whether or not I need to set the flag USE_FAKE_SYSUTILS, if I do, I only need to add one unit, fksysutils, but if I do need the "real" sysutils unit, then it will automatically include the units sysutils, math. So I'll presume all three have to be looked at. They also go on the stack for checking. Doing a directory scan, apparently every architecture does use the sysutils unit, and none use fksysutils, so I'll presume that to be the case.

This unit also defines the machine that is the target for this compiler, so we have to add a unit which will define our target architecture. There are units for every machine, so we have to insert one. At the code around line 116:

{$ifdef nativent}
  ,i_nativent
{$endif nativent}
  ,globtype;    

we change to:

{$ifdef nativent}
  ,i_nativent
{$endif nativent}
{$ifdef i370}
  ,i_i370
{$endif i370}
  ,globtype;    

This means we've now added a new source file which will eventually have to be created, i_i370.pas for the new unit i_i370, which almost certainly will be in the I370 subdirectory. Also unit globtype which is at the end of the list of units used. The rest looks okay so for the moment we're done with this unit.

Unit comphook

Called from: globals.pas, includes fpcdefs.inc, no called units noted

For the moment, there does not appear to be anything I need to do here.

Unit fksysutils

This is only used if the particular architecture sets the flag that says it does not use the sysutils and math units. Given that all of them do use them, I will presume for now I do not need to create this file.

Unit sysutils

Called from: compiler.pas, rtl/objpas/math.pas (outside Compiler directory), located in rtl/i370 outside the compiler directory, includes fpcdefs.inc, no called units noted

This file is a local file for each architecture, so it will be in the rtl subdirectory in its i370 subdirectory. I'll take a look at a couple of architectures to see what's expected to be included. It is used for platform dependent calls, is used by math, and if there are none, it will simply be a stub unit. For the moment, I'll just create a stub file and come back later if I need to.

{

    This file is part of the Free Pascal run time library.
    Copyright (c) 2012 by Viridian Development Corporation

    Sysutils unit for IBM 370/390/zSystem

    See the file COPYING.FPC, included in this distribution,
    for details about the copyright.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

 **********************************************************************}
unit sysutils;

interface
implementation

end.  

Unit math

Called from: compiler.pas, located in rtl/objpas outside the compiler directory, includes fpcdefs.inc, calls unit sysutils

This might need to be changed later. The IBM 370 series uses a different floating point level and its numeric limits are not the same as IEEE. For the moment I'll leave this. Note that when it is compiled, a copy of the PPU needs to be in rtl/i370 (outside of the Compiler directory).

Unit verbose

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit comphook

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit systems

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cutils

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cfileutl

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cclasses

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit globals

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit options

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit fmodule

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit parser

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit symtable

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit assemble

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit link

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit dbgbase

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit import

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit export

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit tokens

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit pass_1

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit wpobase

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit wpo

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cpupara

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cpupi

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cgcpu

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cpunode

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit cputarg

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit globtype

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(To be added later)

Unit i_i370

Called from: compiler.pas, includes fpcdefs.inc, no called units noted

(This will be created later once I know what has to go into it)

An editor has declared this article to be a stub, meaning that it needs more information. Can you help out and add some? If you have some useful information, you can help the Free Pascal Wiki by clicking on the edit box on the left and expanding this page.