نديم ابن ﺤﻣﻮﺪﺓ الخمير - Nadim Khemir > PerlBuildSystem > doc/pnw2004/a_pbs_slide.pod

Download:
PerlBuildSystem-0.44.tar.gz

Annotate this POD

CPAN RT

New  4
Open  2
View/Report Bugs
Source  

PBS ^

PBS (Perl Build System) pronounced "peps".

Introduction ^

History

Authors

Project Start

Gmake's power ^

Who is this man? ^


Gmake's problems ^

Is make broken?

TMOWTDI ^

Whether it's because of shortcomings in "gmake", or simply because people want to try something new, lots of other build systems exists

Links:

http://www.cbel.com/build_management/

http://www.linuxlinks.com/Software/Programming/Development/Tools/Make_Tools/

Rule based systems ^

  CC = gcc
  
  CFLAGS = -Wall                          \
          -DG_DISABLE_DEPRECATED          \
          -DGDK_DISABLE_DEPRECATED        \
          -DGDK_PIXBUF_DISABLE_DEPRECATED \
          -DGTK_DISABLE_DEPRECATED
  
  ttt_test: tictactoe.o ttt_test.o
         $(CC) ttt_test.o tictactoe.o -o ttt_test `pkg-config --libs gtk+-2.0`
  
  ttt_test.o: ttt_test.c tictactoe.h
          $(CC) -c ttt_test.c -o ttt_test.o $(CFLAGS) `pkg-config gtk+-2.0 --cflags`
  
  tictactoe.o: tictactoe.c tictactoe.h
          $(CC) -c tictactoe.c -o tictactoe.o $(CFLAGS) `pkg-config gtk+-2.0 --cflags`
  
  clean:
          rm -f *.o ttt_test

"Smart" systems ^

  Import qw( CONS BIN );
  
  InstallAs $CONS "$BIN/byacc_Fs.exe", 'byacc.exe' ;
  
  $CONS->Program
                        (
                        'byacc.exe'
                        ,qw
                                (
                                closure.c
                                error.c
                                lalr.c
                                lr0.c
                                main.c
                                mkpar.c
                                output.c
                                reader.c
                                skeleton.c
                                symtab.c
                                verbose.c
                                warshall.c
                                )
                        ) ;

http://www.dsmit.com/cons/stable/cons.html

Script language implementation anyone? ^

More and more systems use scripted languages, Ruby, Tcl, Python and Perl.

Many of the programs listed above use Perl.

Army of darkness. ^

Trapped in time

Surrounded by evil

Low on gas

Introduction to pbs. pbs is PBS is Perl is PBS. ^

Goals

Result

Architecture ^

Each pass is implemented in a separate module, giving you the ability to depend and check a system without building it or building it 'manually'.

  [nadim@khemir warp]$ pbs -tt -tno
  
  ** Depending [PBS/0] **
  Depending './b1'  with sub pbs 'W1:./warp1.pl'
     No user defined [W1] Build(), using DefaultBuild() with [BuiltIn, User] rules and [BuiltIn, User] configs.
     ** Depending [W1/1] **
  
  ** Checking **
  Tree for __PBS_warp_tree:
  `- ./all [H1]
     |- ./a1 [H2]
     |- ./a2 [H3]
     |- ./a3 [H4]
     |- ./b0 [H5]
     |- ./b1 [H6]
     |  |- ./c1 [H7]
     |  |  |- ./b0 [H8 -> H5]
     |  |  |- ./d1 [H9]
     |  |  |- ./f1.nad [H10]
     |  |  |- ./f2.nad [H11]
     |  |  |- ./f3.nad [H12]
     |  |  |- ./f4.nad [H13]
     |  |  `- ./f5.nad [H14]
     |  `- ./c2 [H15]
     |     `- ./d1 [H16 -> H9]
     |- ./b2 [H17]
     |- ./b3 [H18]
     `- ./source [H19]
  
  ** Building **
  Number of nodes in the dependency tree: 20.
  2 [0V] nodes scheduled for build.
  #------------------------------------------------------------------------------
  Node './b1' [/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/warp/out/b1] :
  touch /home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/warp/out/b1
  #------------------------------------------------------------------------------
  Node './all' [/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/warp/out/all] :
  touch /home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/warp/out/all
  Build Done.

Switch extravaganza ^

  [nadim@khemir pnw2004]$ pbs -h | wc -l
  PerlBuildSystem:
  
          PBS [-p Pbsfile[.pl]] [[-switch]...] target [target ...]
  
  switches:
      152

The switches can be categorized as follow:

More environment control ^

Pbsfile ^

Pbsfiles are package 'less' perl scripts. Pbsfile can have any name you operating system supports.

If no Pbsfile is given on the command line, pbs will try:

If no Pbsfile.pl is found, pbs exits with an error message.

Evaluation order

Fiddling with namespaces

You write Pbfiles as package less scripts but PBS fiddles with you file.

Original:

  AddRule [VIRTUAL], 'all', ['all' => 'file.o:1.0', 'file2.o:1.0'], "echo hi" ;

Modified:

  #>>>>> start of file '/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004/examples/package_fiddle.pl'
  
  #line 0 '/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004/examples/package_fiddle.pl'
  package PBS::Runs::PBS_1 ;
  use strict ;
  use warnings ;
  use PBS::Constants ;
  use PBS::Shell ;
  use PBS::Output ;
  use PBS::Rules ;
  use PBS::Triggers ;
  use PBS::PostBuild ;
  use PBS::PBSConfig ;
  use PBS::Config ;
  use PBS::Check ;
  use PBS::PBS ;
  use PBS::Digest;
  PBS::Digest::AddFileDependencies('PBSFILE:/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004/examples/package_fiddle.pl') ;
  
  
  #line 1 '/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004/examples/package_fiddle.pl'
  AddRule [VIRTUAL], 'all', ['all' => 'file.o:1.0', 'file2.o:1.0'], "echo hi" ;
  
  
  
  # load OK
  1 ;
  
  #<<<<< end of file '/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004/examples/package_fiddle.pl'

PBS is Rule based ^

  AddRule [VIRTUAL], 'all', ['all' => 'ds_master.pdf'], BuildOk("Done.");
  
  AddRule 'tex2pdf', ['*.pdf' => '*.tex'],
      '/usr/in/tex2pdf %FILE_TO_BUILD %DEPENDENCY_FILES' ;
  
  AddRule 'master', ['ds_master.tex' => @tex], \&BuildMaster;

Rule Components:

PBS is, obviously, not compatibility with gmake. And that's good!

Dependers ^

Cumulative dependers

If multiple rules match a node/file, the sum of the dependencies returned by matching dependers will become the node/file dependencies.

  AddRule 'o_c', ['*.o' => '*.c'] ;
  AddRule 'o_s', ['*.o' => '*.s'] ;

are used on file compress.o, the dependers would generate the following dependencies: compress.c and compress.s.

gmake is, IMO, too magical in its way of handling your rules. I don't mean it is wrong but that it simply doesn't fit the pbs way of generating dependencies.

 Node './nailara/os/rtos/rtos.objects' [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/rtos.objects] :
   Inserted at /devel/nailara/os/rtos/Pbsfile.pl [rtos]:__ROOT:PBS::Runs::rtos_1:BuiltIn:PBS_INTERNAL_/usr/local/lib/perl5/site_perl/5.8.0/PBS/PBS.pm:253.
   dep:
   ./nailara/os/rtos/rtos.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/rtos.o]
   ./nailara/os/rtos/irq.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/irq.o]
   ./nailara/os/rtos/critical.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/critical.o]
   ./nailara/os/rtos/reset.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/reset.o]
   ./nailara/os/rtos/device.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/device.o]
   ./nailara/os/rtos/context.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/context.o]
   ./nailara/os/rtos/exceptions.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/exceptions.o]
   ./nailara/os/rtos/irq_asm.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/irq_asm.o]
   ./nailara/os/rtos/os_init_argus2.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/os_init_argus2.o]
   ./nailara/os/rtos/exceptions_asm.o [/devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/exceptions_asm.o]
   
  rebuild because of:
        __SELF (Doesn't exist)
 
 matching rule: #6[B] 'Object file list:PBS::Runs::rtos_1:User:/devel/nailara/Pbs/Rules/BuildSystem.pm:52'
    => none
    
 matching rule: #7 'Object_files_0 @ '/devel/nailara/os/rtos/Pbsfile.pl':17 :PBS::Runs::rtos_1:User:/devel/nailara/Pbs/Rules/BuildSystem.pm:139'
    => ./nailara/os/rtos/rtos.o ./nailara/os/rtos/irq.o ./nailara/os/rtos/critical.o ./nailara/os/rtos/reset.o ./nailara/os/rtos/device.o ./nailara/os/rtos/context.o ./nailara/os/rtos/exceptions.o ./nailara/os/rtos/irq_asm.o
    
 matching rule: #8 'Object_files_1 @ '/devel/nailara/os/rtos/Pbsfile.pl':29 :PBS::Runs::rtos_1:User:/devel/nailara/Pbs/Rules/BuildSystem.pm:139'
    => ./nailara/os/rtos/os_init_argus2.o
    
 matching rule: #9 'Object_files_2 @ '/devel/nailara/os/rtos/Pbsfile.pl':70 :PBS::Runs::rtos_1:User:/devel/nailara/Pbs/Rules/BuildSystem.pm:139'
    => ./nailara/os/rtos/exceptions_asm.o
 
 Using builder: #6 'Object file list:PBS::Runs::rtos_1:User:/devel/nailara/Pbs/Rules/BuildSystem.pm:52'

Depender Anatomy ^

A depender is a perl sub.

  sub Depender
  {
  my ($dependent_to_check, $config, $tree, $inserted_nodes, $dependencies, $builder_override, $argument_override) = @_ ;
  
  my $build_directory    = $tree->{__PBS_CONFIG}{BUILD_DIRECTORY} ;
  my $source_directories = $tree->{__PBS_CONFIG}{SOURCE_DIRECTORIES} ;
  
  my @my_dependencies ;
  
  if(defined $dependencies && @$dependencies && $dependencies->[0] == 1 && @$dependencies > 1)
          {
          # previous depender defined dependencies
          unshift @my_dependencies, $dependencies->[1 .. -1] ;
          }
  
  unshift @my_dependencies, 1 ;
  
  return(\@my_dependencies, $builder_override, $argument_override) ;
  }

Simplified Depender ^

A depender is a perl sub but we can build the sub in different ways.

AddRule to the rescue

  AddRule 'a.o_rule name',
        ['a.o' => \&SpecialDepender], ...
  
  AddRule 'a.o_rule name',
        ['a.o' => 'a.c', '/path/file', \&SpecialDepender], ...
        
  AddRule 'a.o_rule name',
        ['a.o' => 'a.c', '/path/file'], ...
 
  AddRule 'r4', ['a.h' => '[path]/[basename].[ext]'] ; # this is cyclic of course.

globbing

  AddRule 'r1', ['*/*.c' => '*.h'] ;
  
  AddRule 'r2', ['*/*.c' => 'somefile'] ;
  
  AddRule 'r3', ['*/*.c' => '/full_path/somefile'] ;

Generating your own depender subs

Enough! Give me perl back! ^

If the dependent regexp is a regexp object or a sub, PBS (AddRule) considers the rule to be a pure perl rule.

  AddRule 'rule_1', [qr<\./all$> => '$path/muu/all', '$path/f1', '$path/a.o', '$path/muu/xxxxx.o'] ;
  
  AddRule 'rule_2', [qr<\.o$> => '$path/$basename.c'] ;
  
  AddRule 'rule_3', 
        [       # creator
                [sub{return(@_[4 .. 6])}] =>            
                
                 #regexp
                 AndMatch(qr<\.c$>, NoMatch(qr/xx/)) => 
                                 
                                #dependencies
                                
                                  '$path/$basename.h' 
                                  
                                , [ # post depender
                                        sub
                                                {
                                                return([1, "hi_there2"], @_[5 .. 6])
                                                }
                                  ]
                                  
                                , sub #depender
                                        {
                                        return([1, "hi_there1"], @_[5 .. 6])
                                        }
        ] ;
  [nadim@khemir pnw2004]$ pbs -p ../Pbsfiles/pure_perl_rule/Pbsfile.pl -tta -tt -tno all
    ...  
  ** Checking **
  Tree for __PBS_root_1_pbs_.._Pbsfiles_pure_perl_rule_Pbsfile.pl:
  `- ./all [H1]
     |- ./a.o [H2]
     |  `- ./a.c [H3]
     |     `- hi_there2 [H4]
     |- ./f1 [H5]
     |- ./muu/all [H6]
     `- ./muu/xxxxx.o [H7]
        `- ./muu/xxxxx.c [H8]

Builders ^

Single shell command:

  AddRule 'rule_name',  ['X' => 'x1', 'x2'], "touch %FILE_TO_BUILD" ;

Perl sub:

  AddRule [VIRTUAL], 'rule_name', ['test' => 'all'], sub{ 1, "test OK\n"} ;

Alternatively define your builders in a perl module.

  use YourModule ;
  #or
  PbsUse 'YourPbsModule' ;
  
  AddRule 'rule_name',  ['X' => 'x1', 'x2'], \&YourBuilder ;

Multiple commands:

  AddRule [VIRTUAL], 'test', ['test' => 'all'],
        [
        "touch %FILE_TO_BUILD", # hmm!
        sub{ 1, "test OK\n"} ;
        ]

Variable interpolation

  AddRule 'c_objects', [ '*.o' => '*.c' ],
        '%CC %CFLAGS -c -o %FILE_TO_BUILD %DEPENDENCY_LIST' ;

BuildOK

  AddRule [VIRTUAL], '1', [ 'test' => 'b'], BuildOk('Done', 1) ;

Argument to builder ^

The last argument passed to AddRules is passed as an argument to the builder.

  AddRule 'o_c', ['*.o' => '*.c'], \&BuildAnObject ;

It is up to the builder to interpret the argument (a scalar that can point to other perl types if you want it to). The argument to builder allows to do something special on a specific node. For example, you'd like to compile a specific C file with a -O2 switch to the compiler. You could use the following rules:

  # General rule to build object file
  AddRule 'o_c', ['*.o' => '*.c'], \&BuildAnObject ;
  
  #specific rule 
  AddRule 'special_o_file', ['special_file.o' => 'special_file.c'], undef, '-O2';

All the .o files will be generated by BuildAnObject builder. When building 'special_file.o', BuildAnObject will be passed the argument '-O2'.

What arguments are passed to the builders?

The last defined argument for a node/file is passed to the builder. PBS will warn you if multiple arguments are selected.

Re-using rules: PbsUse ^

PBS having no built-in rules, it would be cumbersome to have to redefine the rules you use in all the Pbsfiles. PBS support in include mechanism that looks like perl's 'use'l. PbsUse takes the name of a file which contains rules or configuration variables definitions.

 File '.../Rules/C.pm':
  AddRule 'exe', [exe => undef], \&BuildAnExe ;
  AddRule 'O_to_C', ...
  AddRule 'C_to_H', ...
  ...

You can then include it in you Pbsfile.

  PbsUse('Rules/C') ; 

PbsUse will automatically append '.pm' at the end of the file name. If the file can't be found in the same directory as the Pbsfile, the environment variable PBS_LIB_PATH will be used to point to the directories where the files are to be searched.

Meta rules ^

 AddRule 'c_objects', [ '*/*.o' => '*.c' ], ...
 AddRule 'cpp_objects', [ '*/*.o' => '*.cpp' ], ...
 AddRule 's_objects', [ '*/*.o' => '*.s' ], ...
 
 AddRuleTo 'BuiltIn', [META_RULE], 'o_cs_meta',
        [\&FirstAndOnlyOneOnDisk, ['cpp_objects', 'c_objects', 's_objects'], 'c_objects'] ;

When you define the above 'o_meta' rule, PBS removes the slave rules from it's rule list (in the current package only). FirstAndOnlyOneOnDisk will be called with a reference to the slaves rules as arguments. This allows you to define your own 'magic'. FirstAndOnlyOneOnDisk source code can be found in the distribution.

Built-in Rules void ^

NO BUILT-IN RULES!

Unlike other build systems, PBS doesn't define any Built-in rule. It is very easy to define and use libraries of rules. Those become your Built-in rules.

Why?

What about my C files? ^

Rules

file://home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/PBSLib/Rules/C.pm

Automatic dependencies

 Depending './nailara/os/rtos/rtos.objects'  with sub pbs 'rtos:/devel/nailara/os/rtos/Pbsfile.pl'
 
 Generating '/devel/nailara/os/rtos/critical.c' dependency file
 './nailara/os/rtos/critical.c' Includes:
 |- /devel/nailara/hal/argus2/argus2.h [H1]
 |  `- /devel/nailara/hal/pagetable.h [H2]
 |     `- /devel/nailara/hal/argus2/pagetable/pagetable.h [H3]
 |- /devel/nailara/hal/argus2/cp15.h [H4]
 |- /devel/nailara/hal/argus2/cpu.h [H5]
 |  |- /devel/nailara/hal/argus2/argus2.h [H6 -> H1]
 |  `- /devel/nailara/hal/argus2/cp15.h [H7 -> H4]
 |- /devel/nailara/hal/argus2/pagetable/pagetable.h [H8 -> H3]
 |- /devel/nailara/hal/cpu.h [H9]
 |  `- /devel/nailara/hal/argus2/cpu.h [H10 -> H5]
 |- /devel/nailara/hal/pagetable.h [H11 -> H2]
 |- /devel/nailara/os/rtos/critical.h [H12]
 |- /devel/nailara/os/rtos/prios.h [H13]
 `- /devel/nailara/os/rtos/rtos.h [H14]
    `- /devel/nailara/os/rtos/prios.h [H15 -> H13]

digest file

file://devel/nailara/projects/argus2_test/nadim_out/nailara/os/rtos/critical.c.depend

Dynamic rules definition ^

Configuration ^

Adding configuration to your Pbsfile:

  AddConfig 'a' => 1 ;
  AddConfig 'a' => 2 ;
  
  AddConfig 'b:locked' => 1 ;
  AddConfig 'b' => 2 ;

Gives this when run:

  [nadim@khemir pnw2004]$ pbs -p ../Pbsfiles/config/lock.pl -tta all
  No source directory! Using '/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004'.
  No Build directory! Using '/home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/pnw2004'.
  Overriding config 'PBS::Runs::PBS_1::CURRENT::User::a' it is now:
  |- ORIGIN [A1]
  |  |- 0 [S2] = PBS::Runs::PBS_1:'../Pbsfiles/config/lock.pl':14 => 1
  |  `- 1 [S3] = PBS::Runs::PBS_1:'../Pbsfiles/config/lock.pl':15 => 2
  `- VALUE [S4] = 2
  Configuration variable 'b' defined at PBS::Runs::PBS_1:'../Pbsfiles/config/lock.pl':18, wants to override locked variable:
  PBS::Runs::PBS_1::CURRENT::User::b:
  |- LOCKED [S1] = 1
  |- ORIGIN [A2]
  |  `- 0 [S3] = PBS::Runs::PBS_1:'../Pbsfiles/config/lock.pl':17 => 1
  `- VALUE [S4] = 1
  Died at /usr/local/lib/perl5/site_perl/5.8.0/PBS/Config.pm line 409.
          ...propagated at /usr/local/lib/perl5/site_perl/5.8.0/PBS/PBS.pm line 483.

Run Example:

  time pbs -p ../Pbsfiles/config/force.pl -dc all

Nodes ^

Node type

Note attributes

  AddRule 'object1',['object.o' => 'object.c:4.2.0'] ;

You can give attributes to the nodes, the attribute is declare by following the node name with a colon and a textual attribute (see the example bellow). When PBS finds such an attribute, it calls a user sub registrated via RegisterUserCheckSub. The user sub receives the following arguments:

The sub should return a file full name (path/name) this is most often the first argument it receives, or die with an error message. The node attribute could be used to, for example, verify the version of a node.

Attributes checker

  RegisterUserCheckSub
  (
  sub
        {
        my ($full_name, $user_attribute) = @_ ;
        #print "$full_name => $user_attribute\n" ;
        return($_[0]) ; # must return a file name
        }
  ) ;

Cyclic dependencies ^

  AddRule 'all', [ all => '1', '2', 'cyclic'] ;
  
  AddRule 'cyclic',  [cyclic => 'cyclic2'] ;
  AddRule 'cyclic2', [cyclic2 => 'cyclic3'] ;
  AddRule 'cyclic3', [cyclic3 => 'cyclic'] ;

Example 2:

  AddRule 'test1', ['all' => 'HERE', 'A'] ;
  AddRule 'test2', ['HERE' => 'x.z'] ;
  AddRule 'test3', ['A' => 'x.z'] ;
  AddRule 'test4', ['this.lib' => 'HERE', 'lib.z'] ;
  AddRule 'test5', ['all' => 'lib.h', 'HERE'] ;
  AddRule 'test6', ['x.z' => 'all'] ;
  AddRule 'test7', ['lib.h' => 'HERE'] ;

Run example:

  time pbs -p ../Pbsfiles/test1/cyclic.pl all and with -o and -dd switch

Repositories ^

finding out what PBS found out

 example in /devel/nailara/projects/argus2_test
 try -dsd -dsi -daa

Separating source from output ^

Using --build_directory, you can have PBS place the generated files in a directory different from the current directory. This allows you to separate your source files from the generated files.

Default directories

If no build or source directory is specified, PBS will use the current directory. If you specify source directories, PBS will search exclusively in the specified directories. The current directory is not searched. If you want the current directory and other directories to be searched, you must specify the current directory too.

When no default build and/or source directory is given, pbs will display an information line.

 No source directory! Using '/devel/nailara/projects/argus2_test'.
 No Build directory! Using '/devel/nailara/projects/argus2_test'.
 No user defined [PBS] Build(), using DefaultBuild() with [BuiltIn, User] rules and [BuiltIn, User] configs.
 ** Depending [PBS/0] **
   Depending './nailara/nailara.objects'  with sub pbs 'NAILARA:/devel/nailara/Pbsfile.pl'

Hierarchical builds ^

Rule definition for sub dependencies

Within a pair of matching curly braquets'{}', list:

  AddRule 'sub_depend',
        {
           NODE_REGEX => 'x.lib'
         , PBSFILE  => './P2.pl'
         , PACKAGE => 'LIB'
         , BUILD_DIRECTORY => '/bd_P2'
         , SOURCE_DIRECTORIES=> ['/sd_P2_2', '/sd_P2_1']
         } ;

Subpbsfiles are Pbsfiles.

Rules

No rule Inheritance. Period!

show example with -dur in any nailara project

Configuration

When PBS starts a sub Pbsfile, PBS pushes (merges) the the parent configuration in the child Pbs. This is done automatically by PBS just before calling the child Build() sub or the default Build() sub.

Magnus Ladulås ^

Magnus Ladulås, hertig, kung av Sverige 1275-1290, född 1240, död 18 december 1290 på Visingsö.

http://sv.wikipedia.org/wiki/Magnus_Ladulås

Configuration variable attributes

 [nadim@khemir PerlBuildSystem-0.25]$pbs -p Pbsfiles/config/parent.pl -no_build -tt -tno -dc -dpos parent

Overriding ^

 /devel/nailara
 |-- Pbs
 |-- communication
 |-- ...
 |-- hal
 |   |-- argus
 |   |-- argus2
 |   |-- arguslt
 |   |-- drivers
 |   `-- pcb
 |       |-- arguslt
 |       |-- olympus
 |       |-- olympus2
 |       |   |-- P1
 |       |   `-- bootinfo-headers
 |       |       `-- testprograms
 |       `-- olympuslt
 |-- ...
 |-- projects
 |   |-- gdb_example
 |   |   |-- nadim_out
 |   |   |   `-- nailara
 |   |   |       |-- hal
 |   |   |       |   |-- argus
 |   |   |       |   |   `-- pagetable
 |   |   |       |   |-- drivers
 |   |   |       |   |   |-- nand
 |   |   |       |   |   `-- uart
 |   |   |       |   `-- pcb
 |   |   |       |       `-- olympus
 |   |   |       `-- os
 |   |   |           |-- debug
 |   |   |           |-- fs
 |   |   |           |   |-- nandfs
 |   |   |           |   `-- vfs
 |   |   |           |-- mem
 |   |   |           |   `-- poolheap
 |   |   |           |-- rtos
 |   |   |           `-- utils
 |   |   `-- nailara
 |   |       |-- hal
 |   |       |   `-- pcb
 |   |       |       |-- nadim
 |   |       |       `-- xxolympus
 |   |       `-- os
 |   |           `-- mem
 |   |               `-- poolheap
 |   `-- usbmass_demo
 `-- utils

(pbs -bi '*Bitmap*')

(./gdb_example/nailara/hal/pcb/xxPbsfile.pl)

Digest ^

PBS automatically generates digests when a node is build. Time Stamps are ignored(*).

The digest contains the following elements:

1 The md5 of the Pbsfile that generated it
2 The md5 of the files that have been included through PbsUse
3 Any element you have specified through the Add...Dependencies functions
4 The md5 of the node's dependencies
5 The md5 of the generated node

If all the elements needed to build the node are found in the file's digest, PBS uses it, otherwise the node is rebuild.

PBS will expects a digest for all nodes/files. We have to tell PBS how to make the difference between a generated file and a source file.

ExcludeFromDigestGeneration() allows you to exempt a certain type of files from the digest.

  ExcludeFromDigestGeneration('c_files' => qr/\.c$/) ;

The first argument is a description string, the second one a reference to a regexp object. Node names matching the regexp will be exempted.

Some source files are automatically generated (ex by flex, yacc, your own generators, ...), ForceDigestGeneration() can selectively re-impose a digest on a certain file that would have been exempted by ExcludeFromDigestGeneration.

  ForceDigestGeneration( 'a.c is generated' => qr/a\.c$/) ;

Example of PBS generated digest:

  # This file is automaticaly generated by PBS (Perl Build System).
  
  # File: /devel/nailara/projects/argus2_test/nadim_out/main.o.pbs_md5
  # Date: Tue Mar  9 20:34:25 2004 
  # User: nadim @ khemir.net
  # PBS_LIB_PATH: /home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/PBSLib/:/devel/nailara/Pbs/:/devel/nailara
  # Pbsfile: ./Pbsfile.pl
  
  $digest = {
             './main.o' => 'dc74f3b18b88e845027a683a68c88cbb',
             '__NODE_VARIABLE:CDEFINES' => ' -DARCH_OLYMPUS2=1 -DARGUS2=1 -DCPU_ARGUS2=1 -DEXCEPTION_BASE=0x06000000 -DKERNEL=1 -DKHEAP_32BIT_POOL=1 -DKHEAP_SIZE=0x20000 -DPROC_MAX_PROCESSES=16 -DPROC_QUANTUM=1000000 -DROBASE=0x06000000 -DROBASE_PHYS=0x06000000 -DRWBASE=0x06100000 -DRWBASE_PHYS=0x06100000 -DRWLIMIT=0x08000000',
             '__PBS_LIB_PATH/Builders/HostedBuilder.pm' => '080bac06fe0fc6338ccd64f6c02a75cb',
             '__PBS_LIB_PATH/projects/argus2_test/Config.pm' => '28dc45bf0943c1a97745de98ba82b8b6',
             '__PBS_LIB_PATH/Configs/Projects/Nailara.pm' => 'c85daa008b4d5eb2abbbff59abf5d6ca',
             '__PBS_LIB_PATH/Configs/Compilers/gcc.pm' => '680336b56f1a9614a270114e3a03a6c9',
             '__PBS_LIB_PATH/Dependers/Locator.pm' => '1695797f4dea0a29cd755606daf7af2b',
             '__PBS_LIB_PATH/ShellConfig.pm' => '6cc04b44ec134dca90b6eddd13716550',
             '__PBS_LIB_PATH/Rules/Nailara.pm' => 'ee2642a484fd63dcd657cdd9c2bc13e8',
             './main.c' => '912c52d714d2edf8aa0d43601390ae7b',
             '__PBS_LIB_PATH/Rules/C.pm' => '9a3e1eeef74aa887e723a1e6135ba876',
             '__PBS_LIB_PATH/Configs/Compilers/compiler.pm' => 'ab5c5fba23b3b45f30bae03ac206e72f',
             '__PBS_LIB_PATH/Rules/C_depender.pm' => 'deadefe169fa3ecb947d6211d796c5a2',
             '__PBS_LIB_PATH/Configs/ConfigureProject.pm' => '6f692d3717eccef03452b3ac1882e2ce',
             '__PBSFILE' => '425fbf4ed9c8e7f3034b6e0acde50e71',
             '__PBS_LIB_PATH/MetaRules/FirstAndOnlyOneOnDisk.pm' => 'edc87f9805eec8a7dd61f33a70e770d0',
             '__PBS_LIB_PATH/Rules/BuildSystem.pm' => '353e53811d6c757bae8f654b95fbae11'
              };

Tuning digests ^

AddFileDependencies() : PBS will compute an md5 for each file in the list you pass as argument and add it to the digest.

AddEnvironementDependencies(): PBS will add each environment variable you name in the list passed as argument. If the environment variable is not set, PBS will add the variable to the digest and give it the empty string value.

AddSwitchDependencies() : : PBS will add the variables and their values to the digest. Only Defined (-D) and User Defines (-u) can be added.

  AddSwitchDependencies('-D*') ; # depend on all command line defines 
  AddSwitchDependencies('-u*') ; # depend on all user variables from the command line
  AddSwitchDependencies('-u something', '-D debug', -D clean) ; # add only the given variables to the digest

AddVariableDependency() : This allows you to insert a variable name and it's value into the digest. For example, this could be used if you are cross compiling for an embedded platform from different OSes. The cross compilers would have different md5 on the OSes, so you can't add the cross compiler through AddFileDependencies().

  my $compiler_version = GetCompilerNameAndVersion(...) ;
  AddVariableDependency('compiler_version' => $compiler_version) ;

Fine granularity control

  AddNodeFileDependencies(qr/^.\/z0$/, 'pbs.html') ;
  AddNodeFileDependencies(qr/c/, 'pbs.pod') ;
  AddNodeVariableDependencies(qr/c/, 'a' => 1, 'b' => '2') ;

What's not in the digest?

Cross Compilation ^

PBS can log into a remote computer through SSH or Telnet and execute a specific command.

  use PBS::Shell_SSH ;
  PbsUse('Builders/HostedBuilder') ;
  
  my $shell =  new PBS::Shell_SSH
                (
                  HOST_NAME        => 'localhost'
                , USER_NAME        => 'nadim'
                #~ , PROTOCOL         => 1 # default is SSH2
                #~ , REUSE_CONNECTION => 1
                ) ;
  
  AddConfig  C_COMPILER_HOST => $shell ;
                
  PbsUse('Rules/C') ;
  PbsUse('Configs/gcc') ;
  
  AddRule [VIRTUAL], 'all', ['*/all' => qw(source.o source2.o source3.o)], BuildOk() ;

In Rules/C:

 $c_compiler_host = GetConfig('C_COMPILER_HOST') ;
 
 AddRuleTo 'BuiltIn', 'c_objects', [ '*/*.o' => '*.c' ]
  , HostedBuilder
    (
     'c_objects'
    , $c_compiler_host
    , "%CC %CFLAGS %CDEFINES %CFLAGS_INCLUDE -I%PBS_REPOSITORIES -o %FILE_TO_BUILD -c %DEPENDENCY_LIST" 
    ) ;

Parallel build ^

IThreads

PBS didn't work nice with IThreads.

The net result was a systemt was much (order of magnitudes) faster with a single thread.

Using fork while waiting for low level threads

Low level threads with synchronisation responsibility placed on the application would be PBS favorit solution. In the mean time an experimental implementation using fork has been started.

Triggers ^

Trigger is a sytem to allow the current build to hook into other dependency trees. This can be used to (for example) synchronize libraries. the details are located in the library Pbsfile.

file://home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/trigger/Pbsfile.pl

file://home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/Pbsfiles/trigger/trigger.pl

Other possible uses ^

Boilerplate wizardry ^

  [nadim@khemir PerlBuildSystem-0.25]$ tree PBSLib/
  PBSLib/
  |-- Builders
  |-- Configs
  |-- Dependers
  |-- MetaRules
  |-- Rules
  |-- UserBuild
  `-- Wizards
      |-- Breakpoint.pl
      |-- BuilderSub.pl
      |-- SimpleDependerSub.pl
      |-- StandardPbsfile.pl
      |-- Subpbs.pl
      |-- menu.pl
      `-- template.pl

file://devel/nailara/Pbs/Wizards/nailara_project.pl

file:///devel/nailara/Pbs/Wizards/nailara.pl

 pbs -w nailara

Documenting your build system ^

Let there be POD!

Sensible output ^

Coloring (PBS and User)

Colors for these functions can be defined through the command line or PBS_FLAGS. The "depend" step is indented. Output from user is also indented.

Verbosity control

  [nadim@khemir PerlBuildSystem-0.25]$ pbs -hs v
  v|verbosity=s: Used in user defined modules.

-- verbose is not used by PBS. It is intended for user defined modules.

I recommend to use the following settings:

0 Completely silent (except for errors)
1 Display what is to be done
2 Display serious warnings
3 Display less serious warnings
4 Display display more information about what is to be done
5 Display detailed information
6 Not defined (yet)
7 Debug information level 1
8 Debug information level 2
9 All debug information
'string' => user defined verbosity level (ex 'my_module_9')

Multiple verbosity switches can be given, they are store in {__PBS_CONFIG}{ VERBOSITY}.

Coloring output from gcc

C-Tech build system uses colorgcc. http://www.mindspring.com/~jamoyers/software/colorgcc/

Taking control (if you dare) ^

User Build()

PBS is a three pass sytem.

Using default Build():

 pbs -p Pbsfiles/user_build/user_build.pl -no_user_build -tt -tno -fb -no_digest x.lib

Using your own Build():

 pbs -p Pbsfiles/user_build/user_build.pl -tt -tno -no_digest -dbsno x.lib

The dreaded ARM compiler

Enter 'miners'

 Rules ...
 Rules ...
  
 #-------------------------------------------------------------------------------
 PbsUse('UserBuild/BuildSequenceMiner') ;
 PbsUse('UserBuild/Multiple_O_Compile') ;
 
 sub Build
 {
 my ($build_result, $build_message) = BuildSequenceMiner
                        (
                          [@_]
                        , [\&Multiple_O_Compile]
                        ) ;
                        
 PrintInfo("Build done.\n") ;
 
 return($build_result, $build_message) ;
 }

Let's build! ^

 [nadim@khemir argus2_test]$ time pbs

Ananova - Wife pays man salary to keep quiet. ^

Pbs offers switches to lower output verbosity.

Debugging ^

One PBS requirements was to help the PBsfile writer to find errors in the build system he defines. PBS has an army of switches to help you get insight in how the dependency tree is constructed.

Simple stuff takes you a long way

When things get stubborn.

Logging

Serious Visualization ^

How did I get there? ^

file://home/nadim/Dev/PerlModules/PerlBuildSystem-0.25/snapshots/00000.png

Debugging Hooks ^

Hooks

  #DEBUG HOOK (see PBS::Debug)
  my %debug_data = 
        (
          TYPE                   => 'BUILD'
        , CONFIG                 => $file_tree->{__CONFIG}
        , NODE_NAME              => $file_tree->{__NAME}
        , NODE_BUILD_NAME        => $build_name
        , DEPENDENCIES           => \@dependencies
        , TRIGGERED_DEPENDENCIES => \@triggered_dependencies
        , ARGUMENTS              => \$arguments
        , NODE                   => $file_tree
        ) ;
        
  #DEBUG HOOK, jump into perl debugger is so asked
  $DB::single = 1 if(PBS::Debug::CheckBreakpoint(%debug_data, PRE => 1)) ;

Breakpoints

 AddBreakpoint
        (
          'hi'
        , DEPEND => 1
        , PRE => 1
        #~ , USE_DEBUGGER => 1
        #~ , ACTIVE => 1
        , ACTIONS =>
                [
                sub
                        {
                        PrintDebug "Hi there.\n" ;
                        }
                ]
        ) ;

types:

filters :

Using the perl debugger

Take us to Warp 1 Mr Sulu ^

PBS takes 6 seconds, in a test project, to do nothing vs 0.2 for gmake. OK, gmake doesn't do what PBS does but for the day to day work, 6 seconds are just too much!

Optimizing

Warp!

 [nadim@khemir argus2_test]$ rm nadim_out/nisse/nisse.o
 [nadim@khemir argus2_test]$ time pbs -warp

Warp, is it possible?

http://www.grc.nasa.gov/WWW/PAO/warp.html

Warp 2? ^

What can be done to run faster?

Peeping at the code ^

Future development ^

Problems with PBS

So what's next?

Contact: nadim@khemir.net.

syntax highlighting: