FPMake revamp

FPMake is a build-system for Pascal code specifically. It can use a make-over, though.

This document is a place to gather basic design-principles and other ideas for the new version. It is all work-in-progress and any input is very welcome.

Basic design principles

Keep it simple

One of the things that makes fpmake so complex, is that it tries to do everything that it's predecessors also did. And tries to accommodate all kind of different package-designs as possible. The new version will not do this any more. Other package-managers (yarn, npm) and their underlying build-systems show that enforcing one design is easier, and in the long term also more feasible.

This means:

• No functionality to adapt the location of .ppu, .o, .frs, binaries, help-files, tests or any other files. (The UnitInstallDir, UnitConfigFilesInstallDir etc)
• No need anymore for macro's ($(target) and such)
• Options to do all kind of fancy stuff, like copying files around, calling external applications, zipping files are removed or limited. So they are not tempted to create their own packages-layouts.
• etc

First define the packages, process them later

The packages are defined and configured first. In a later stage they are being processed (build, installed, etc)

This leads to a clear design, in which a package-developer knows what is possible and what not. It also makes it easier to create external packages in a repository. Due to the declarative process, it is easier to collect a packages properties inside a declaration-file. (On which the central repository can base it's information)

Clear separation of responsibilities

In fpmkunit their was clearly an attempt to separate different responsibilities. But the original ideas got lost and it was not clear anymore which class should do what.

In the new setup the separation should be crystal clear again, and is as much as possible enforced bij using (corba) interfaces for interfacing.

Separation between build, target and host

The original fpmake only dealt with the target (cpu, abi and operating system) and only implicitly the build' system. Now fpmake differentiates between the build-system (on which the system is running), the host system (on which the created compiler is running) and the target system, for which the compiler will create code.

Extensibility

Extensibility goes into two ways:

1. Extending the ability to define packages. For example: define packages based on the Lazarus .lpi-format.
2. Extend the actual installer. In `fpmkunit` the support of threading and other features that are not available are enabled using defines. The idea is that the new system uses another approach. To make it possible to select other 'runners'. Which each can offer different features, and support different platforms.

Cross-platform

It must be possible to run the basic console version on all platforms on which the compiler can be started. Usage of the classes unit is avoided. (But if that really has a purpose... I'm not sure)

Basic design

This is not a full documentation of fpmake. Only the basic building-blocks are described here.

Structures

Functionality is split into a few units. They are divided into three categories:

Package units

These units are only used to define packages. Package maintainers can choose on their own which package-units they use. They are only forced to implement the interfaces from the basic units. This does mean that runners can not use or rely on anything from these units. The world of packages and runners are completely separated. Only connected through the basic units.

Runner units

These units are used to create runners. They contain the actual logic to build, install, clean and more, based on the package-definitions. Package-definitions may not use any of these units, due to the simple fact that a runner could also use other implementations. The only communication is - again - trough the classes in the basic units.

Basic units

Units with the definitions that glue the world of the package-definition and runners together. These are the only units that can be used in package-definitions and runners.

Basic units

Can be used throughout all parts of fpmake. Note that a lot of these are interfaces. The actual implementation might differ. Depending on a particular package, or actual runner. (See #Extensibility)

fpmku

Basic definitions and functionality which are used throughout all parts of fpmake.

IStaticSettings

Holds all the settings which are independent of the individual packages. For each run of fpmake they are constant and the same for all packages. Such as the used compiler, host-, 'target- and build-architectures.

IDynamicSettings

Holds all the settings which can differ between packages. Like the unit-search paths which depends on a package dependencies. Of compiler-options.

IPackage

The base interface for a package

IPackageHelper

A helper class with lots of routines that makes live easier. Like logging, error-handling and such.

Basic package units

Units which are used by package-definitions. Note that every package determines on it's own how the package is actually defined. As long as it implements the IPackage interface.

fpmkpck

Base classes to define a package. A package may use this unit to define a package. But other implementations are also possible.

TBasicPackage = class(IPackage)

Basic class that implements the IPackage interface and can be used as a base class to base package-classes upon. The idea is that this class could also be used by other implementations. (An lpi-based one?) Keep this class as clean as possible. This means: add all kind of helper-methods to make stuff easier to TPackage

TPackage = class(TBasicPackage)

Basic package-class, used for all the packages included by fpc.

TUnitTarget = class(IGoal)

Basic class to represent an unit.

Runner units

fpmks

Base classes to handle settings. It does not depend on the classes unit or anything else, so it can be used in a really bare runner.

TBasicStaticSettings = class(IStaticSettings)

TBasicDynamicSettings = class(IDynamicSettings)

fpmksf

Extends the functionality of the classes in fpmks, but depends on fcl-process. So is not available on all platforms.

TFullStaticSettings = class(TBasicStaticSettings)

Adds some extra functionality to TBasicStaticSettings. It calls the compiler to obtain some defaults. (build- and host-architecture, compiler version)

fpmkr

Basic runner-implementation. Might be used to bas other runners on. Depends only on the rtl to make it as cross-platform as possible.

TFpmkCustomRunner

Base class that implements a runner.

fpmkcrun

Console runner. Depends only on the rtl to make it as cross-platform as possible.

TFpmkBasicConsoleRunner

Implementation of a console-runner.

fpmkcrf

Console runner with additional features. Supports less platforms. (For example: uses fcl-process)

TFpmkFullConsoleRunner

Implementation of a console-runner with additional features. (Like parsing of the compiler-output, threading, such things)

Stages

Each runner processes the packages in these well-defined stages.

1. Register all packages
   Every package is registered.
2. Collect (static) settings
   Collect all static settings and initialize the dynamic settings with some defaults. After this stage all static settings are immutable.
   1. Defaults
      For example: Set the default compiler-name.
   2. Config-files
      
   3. Environment
      
   4. Command-line
      
   5. Finalize
      For example: set the architecture of the build-system, if not set already.
3. Create packages
   Create all packages. The static settings can be used to alter the properties of a package. (Even to skip it)
4. Build package tree
   Builds a dependency tree, and places all packages in an ordered list that defines in which order they are processed. (In principle, a multi-threaded runner could adapt the order a bit, for example when a dependency is not ready yet.) This is also the place where external-dependencies are being evaluated.
5. Collect package settings
   In the order defined in the previous step, the dynamic settings are obtained for every package:
   1. Each package get it's own dynamic-settings, cloned from the already collected dynamic settings.
   2. The package can initialize it's own dynamic settings
   3. All dependencies can adapt the packages dynamic settings. Deepest dependencies first. This is also where the search-paths are set.
   4. The package adapts it's dynamic settings
6. Actual action (compile, install, clean etc. For all packages, in the defined order (maybe also possible to do things the other way around, for example in case of a clean?)