Difference between revisions of "Taking advantage of multi core/multi threading machines"

From Free Pascal wiki
Jump to navigationJump to search
Line 1: Line 1:
 +
Please discuss at the discussion page.
 +
 
== Introduction ==
 
== Introduction ==
 
During the past years, the focus of the development of FPC was on making FPC multi platform. This goal has been successfully reached, FPC is available on all important platforms so we need new challenges.
 
During the past years, the focus of the development of FPC was on making FPC multi platform. This goal has been successfully reached, FPC is available on all important platforms so we need new challenges.
Line 29: Line 31:
  
 
Further, e.g. for the build system, information how long a building a certain package is estimated to take might be interesting as well because this would allow a better planning how dependend packages should be build. (Marcov: good point. detailed Profiling info of the compilation process, with better granularity than the overall runtime would certainly help :-)
 
Further, e.g. for the build system, information how long a building a certain package is estimated to take might be interesting as well because this would allow a better planning how dependend packages should be build. (Marcov: good point. detailed Profiling info of the compilation process, with better granularity than the overall runtime would certainly help :-)
 
=== make -j 2===
 
 
Parallel compilation using make -j 2 currently fails miserably because if two packages use a third unit, two compilers can start compiling that unit at once. Because of this package dependencies alone are not enough; a make tool would need to scan the source code of two packages to check if they are really independend.
 
 
Marco: make -j 2 is too expensive due to constant restarting of the compiler anyway. It is faster to let FPC compile a few units single threaded in one run, than to let it really compile parallel using -j 2 (iow with multiple processes). This also because due to caching of ppu's, disc traffic for the overall process is less.
 

Revision as of 23:18, 5 February 2006

Please discuss at the discussion page.

Introduction

During the past years, the focus of the development of FPC was on making FPC multi platform. This goal has been successfully reached, FPC is available on all important platforms so we need new challenges.

If one looks at the roadmaps of AMD and Intel, you'll see that future processors won't get have performance gains by higher clockrates as it was the last decades. Instead, the CPUs will be multi core. However, multi core processors require new apporaches to take advantage of these cores. So I consider this as one of the main task for the next years: making FPC ready for multi core systems.

This must done on several levels:

    • language
    • libaries
    • compiler
    • build system

Making the language multi core ready

This requires a lot of dicussion I fear: which new language elements can be added to make building multi threaded applications easier.

e.g.

  • monitor classes ?

Libraries

This task is almost finished, most libraries are multi threading ready. Some things like the heap manager need tuning but that's it.

Multi threading in the compiler

This is a long term task, making the compiler multithreaded, i.e. start different threads to compile several units at once.

Build system

This should be rather easy with proper package dependencies: the building of different packages being independed can be done simultaneously. (who determines this? Does this mean the compiler should know about packages and their dependancies? Package groups?)

All this requires also a common resource management, i.e. how much threads/processes should be spawned to take an optimum advantage of the multiple cores. It makes no sense to start the building of 10 packages at once on a dual core system. This causes only cache thrashing etc and slows down things. (Marco: 2 cpu's : 3-4 threads 4 CPU's : 6-8 threads)

Further, e.g. for the build system, information how long a building a certain package is estimated to take might be interesting as well because this would allow a better planning how dependend packages should be build. (Marcov: good point. detailed Profiling info of the compilation process, with better granularity than the overall runtime would certainly help :-)