packages

FPC/Lazarus unfortunately are a bit ambiguous with respect to the term packages. In general it is a collection of units, but the precise way they are related varies. The most common definitions:

1. A set of units in FPC that are treated together. Both in the fpcmake as the newer fppkg packagemanager contexts.
2. A set of units in Lazarus (with designtime parts that require registration into the DLL). See Lazarus Packages
3. A set of units compiled into a DLL with some extra features that make the package a logical part of the main program, instead of an "external" part. Also called Delphi package. I call this Library package from now on.

This article is meant as a brainstorm session about the latter, one of the missing pieces of the Delphi compability. Most created from browsing an hour on the web, looking at articles about fixes and modding the packages system

Note that in time Lazarus packages might be implemented as Delphi packages.

Delphi packages

Maybe Delphi packages is a bad name btw. Library package would be better? I'll at least use it for this article till something better comes up.

A Library-package is essentially a DLL with some extra's.

• One of those extra's are two procedures, a pascal level initializer ("Initialize") and a module finalizer ("Finalize"). I'm not sure if they are library procedures that traverse compiler generated tables, or if they are wholly compiler generated.
• Some form of identification and dependancy. I'm not sure how much these requirements are on top of DLL, and what are part of DLL in general. I'm also not sure how fine grained this info is.
• If your system uses packages,
  • the RTL and other units the packages use, must also be in a package.
  • A unit can exist only once in one package (see Swart reference below), and probably only once in EXE+packages too .
  • On load of a package this condition is checked, but possibly only when packages are dynamically loaded by the program. (the rest being handled by normal dll dependancies) This combined with the previous condition avoids problems like multiple definitions of the VMTs and RTL state in general. Note also that this means a list of units per package must be accessable to the main program on load (?) It is not entire clear what "module" means in library package context. Probably only librarypackage, but maybe a little additional metadata or also e.g. unit/debug data.
    • So note that apps that load packages dynamically, must have all relevant modules loaded. The following scenario would be interesting. Package C depends on B and A, and B also depends on A. Now the exe is statically linked to pkg A (the RTL dll), and then dynloads first B and then C. Does this work?
  • Since they are DLLs, due to the Windows DLL symbol resolving, packages can probably survive some minor patching (allowing implementations to be fixed), but in general it must be pretty much the same packages as the program was compiled with.
• Packages can use higher level functions (like ansistrings and classes) over packages borders. This means they use the same memory manager. It is not know if this automatically is changed to COM compatible memmanager sharemem (e.g. in the rtl BPL creation) or if this is the normal suballocator.
• Like DLLs, packages can be statically linked (so that its presence is required for startup), and programs can load additional packages using loadpackage (unloadpackage?). Since a package can only depends on existing packages, and not on the exe (RTL also in package remember!), this means that dynloaded packages can be crafted AFTER .exe generation, allowing for plugin systems.
• packages are loaded by a procedure called "safeloadlibrary" which is a loadlibrary wrapper that saves the FPU statusword and disables some windows errorwindows on failure to load. (?!?)
• relocation, for this a little thought experiment using the following realistic Delphi scenario: Assume packages A and B both import packages RTL, C, D, but are compiled separately and used in one .EXE program. This means that the compiler can't guarantee all BPL's are already on a unique baseaddress and addresspace (since A and B could be compiled without the other present). -> they are relocatable, though probably DLL loading will resolve this transparantly mostly.

References

• Swart on pkgs

What needs to be done

The following points are very roughly the steps to be taken.

1. RTL into a DLL
2. mainprogram uses RTL, and can access all symbols it could without. (properties, vars, typed constants, functions, RTTI)
3. Add another package to the mix. (and multiple in general)
4. Try to dynload package.
5. anything else
   1. (more delphi compat, but will probably happen evolutionary)
   2. Meta data to better automated manage packages. (iow, being able to read the dependancies of a package, what units are in it, timestamps etc)

This might require PPU changes to list what package an unit is in, _OR_, better I think a kind of library PPU that lists the units in a package as well as some timestamping CRCs etc. (.dcp in Delphi?)

Lazarus and Library packages

Well, first there needs to be a base package system, but then Delphi packages could be loaded too.