NAME

Test::AutoBuild - Automated build engine controller

SYNOPSIS

  use Test::AutoBuild;
  use Config::Record;

  my $config = new Config::Record (file => $filename);
  my $builder = new Test::AutoBuild (config => $config);

  my ($status, $log) = $builder->run($timestamp);

  if ($status) {
     print STDERR $log, "\n";
  }

  exit $status;

DESCRIPTION

This module provides the build controller, tieing together various subsystems to form an integrated engine. It is wholely reponsible for loading the various runtime objects (stages, modules, repositories, package types, monitors, publishers) based on their definitions in the configuration file and then invoking the build. This object does not, however, contain any logic pertaining to how the build is run, since this is all delegated to the stages defined in the configuration file.

SETUP

After installing the modules, the first setup step is to create an unprivileged user to run the build as. By convention the user is called 'builder', in a similarly named group and a home directory of /var/lib/builder. So as root, execute the following commands:

 $ groupadd builder
 $ useradd -g builder -m -d /var/lib/builder builder

NB, with the combined contents of the source checkout, the cache and the virtual installed root, and HTTP site, the disk space requirements can be pretty large for any non-trivial software. Based on the applications being built, anywhere between 100MB and many GB of disk space make be neccessary. For Linux, making /var/lib/builder a dedicated partition with LVM (Logical Volume Manager) will enable additional space to be easily grafted on without requiring a re-build.

The next step is to create the basic directory structure within the user's home directory for storing the various files. There are directories required for storing the source code, a virtual root directory for installing files, a build archive, package spool directories, and publishing directories for HTTP and FTP servers. To facilitate quick setup, a script is provided to create all the required directories. Run this script as the unprivileged user

  $ su - builder
  $ auto-build-make-root /var/lib/builder

It will display a list of all the directories it creates, but for advance reference, they are

   /var/lib/builder/install-root
   /var/lib/builder/source-root
   /var/lib/builder/log-root
   /var/lib/builder/build-archive

   /var/lib/builder/package-root
   /var/lib/builder/package-root/rpm
   /var/lib/builder/package-root/rpm/BUILD
   /var/lib/builder/package-root/rpm/RPMS
   /var/lib/builder/package-root/rpm/RPMS/noarch
   /var/lib/builder/package-root/rpm/RPMS/i386
   /var/lib/builder/package-root/rpm/RPMS/i486
   /var/lib/builder/package-root/rpm/RPMS/i586
   /var/lib/builder/package-root/rpm/RPMS/i686
   /var/lib/builder/package-root/rpm/RPMS/x86_64
   /var/lib/builder/package-root/rpm/RPMS/ia32e
   /var/lib/builder/package-root/rpm/RPMS/ia64
   /var/lib/builder/package-root/rpm/RPMS/sparc
   /var/lib/builder/package-root/rpm/SPECS
   /var/lib/builder/package-root/rpm/SOURCES
   /var/lib/builder/package-root/rpm/SRPMS
   /var/lib/builder/package-root/zips
   /var/lib/builder/package-root/tars
   /var/lib/builder/package-root/debian

CONFIGURATION

The configuration file determines all aspects of operation of the build engine, from the modules built, through the package types detected, archival method, to build workflow stages, and much more. The example build configuration file installed by default should provide a fully functional build instance running under /var/lib/builder, which is capable of building Test-AutoBuild, versions 1.0.x and 1.1.x, along with the AutoBuild-Applet. A good sanity check for correct installation, is to ensure that the example build configuration succeeds when run.

The configuration file is split into a number of logical groups, which will be considered in turn below. The minimal level of configuration to get started involves editing the list of modules, along with the source repository definitions.

General runtime

The following options define miscellaneous aspects of the build engine runtime environment.

root

The root option is a grouping under which core directories of the build engine are defined.

  root = {
    ... nested options...
  }

The following nested options are permitted with the root option

source

The location into which modules' source code will be checked out from version control. If not specified this option defaults to the location $HOME/source-root

  root = {
    ...
    source = /var/lib/builder/source-root
    ...
  }

Thus, a module with a name of 'dbus-dev' would be checked out into the directory

  /var/lib/builder/source-root/dbus-dev

install

The location into which a module's build process would install files to be used by dependant modules later in the build cycle. This location is made available to a module's build control file via the environment variable $AUTOBUILD_INSTALL_ROOT. If not specified this option defaults to the location $HOME/install-root

  root = {
    ...
    install = /var/lib/builder/install-root
    ...
  }

Consider, for example, a module 'mozilla' which depends on a library 'openssl'. The 'openssl' module would be listed as a dependant module so that it is built first. The build of 'openssl' would install itself into the install root, perhaps by passing the 'prefix' argument to a configure script:

  ./configure --prefix=$AUTOBUILD_INSTALL_ROOT

The later build of mozilla, would then build against this version of openssl, by using

  ./configure --with-openssl=$AUTOBUILD_INSTALL_ROOT

package

The location in which a module's build process will create any binary packages it generates, for example RPMs, or Debian packages. The packages are typically placed into a package type specific sub-directory. This location is made available to a module's build control file via the environment variable $AUTOBUILD_PACKAGE_ROOT. If not specified, this option defaults to the location $HOME/package-root

  root = {
    ...
    package = /var/lib/builder/package-root
    ...
  }

Consider, for example, a module which generates an RPM, of itself. The $AUTOBUILD_PACKAGE_ROOT directory would be used to set the '_topdir' macro for the RPM build process

  rpmbuild --define '_topdir $AUTOBUILD_PACKAGE_ROOT/rpm' -ta foo.tar.gz

log

The location in which the output from a module's build control file will be spooled during execution. If not specified, this option defaults to the location $HOME/log-root. The control file's standard output and error streams will be combined into one.

  root = {
    ...
    log = /var/lib/builder/log-root
    ...
  }

admin-email

The email address of the build engine administrator, typically linked from the bottom of the HTML status pages. This is also the address spammed with build status alerts if the Test::AutoBuild::Stage::EmailAlert module is in use.

  admin-email = admin@example.com

admin-name

The full name of the build engine administrator, typically displayed on the bottom of the HTML status pages.

  admin-name = John Doe

log4perl

A configuration block controlling the output of debug information during execution of the build. The data here is passed straight through to the init method in the Log::Log4perl module, so consult that module's manual page for possible configuration options. The example setting, enables display of progress through the build workflow. To get maximum possible debug information, change the log4perl.rootLogger option to 'DEBUG' instead of 'WARN'.

  log4perl = {
    log4perl.rootLogger = WARN, Screen

    # To get progress updates
    log4perl.logger.Test.AutoBuild.Monitor.Log4perl = INFO

    log4perl.appender.Screen        = Log::Log4perl::Appender::Screen
    log4perl.appender.Screen.stderr = 1
    log4perl.appender.Screen.layout = Log::Log4perl::Layout::SimpleLayout
  }

counter

This configuration block determines the module used to generate the unique build cycle counter.

  counter = {
    ..nested options..
  }

The nested options allowed within this block are

module

The full package name of the subclass of Test::AutoBuild::Counter used to generate the build cycle counter. Consult that module for a list of known implementations. The exmaple configuration sets the build counter to match the timestamp taken at the start of the build cycle.

  module = Test::AutoBuild::Counter::Timestamp

options

This is defines a set of options as key, value pairs which are passed to the counter object created. The valid keys for this are specific to the package specified in the module parameter above, so consult the manual page corresponding to the module defined there. If using the Test::AutoBuild::Counter::ChangeList class, and there is a source repository named, mysvn, one would set an option such as

  options = {
    repository = mysvn
  }

Source repositories

The repositories configuration block, defines the source repositories from which modules are checked out. The keys in the block form the short name of the repository, and is referenced later when defining the paths for modules' source checkout

  repositories = {
    myrepo = {
      ... definition of myrepo ...
    }
    otherrepo = {
      ... definition of otherrepo ...
    }
  }

Within each repository definition the following options are supported

label

The free-text display name of the repository, typically used in the HTML status pages.

module

The full package name of a subclass of Test::AutoBuild::Repository which implements the checkout procedure for the repository type. There are repository types for common version control systems such as Subversion, CVS, Perforce and GNUArch, as well as a simple non-versioned repository. Refer to the Test::AutoBuild::Repository module for a list of known repository types and their corresponding package names.

  module = Test::AutoBuild::Repository::Subversion

env

Lists a set of environment variables which will be set whenever running any repository commands. The possible environment variable names vary according to the type of repository, so refer to the manual page for the repository module defined in the module option. For example, the CVS repository type uses the CVSROOT environment variable to specify the repository location.

  env = {
    CVSROOT = :pserver:nonymous@cvs.gna.org:/cvs/testautobuild
  }

option

A set of configuration options specific to the type of repository configure. Again, refer to the manual page for the repository module defined in the module option. For example, the GNU Arch repository type supports the 'archive-name' option

  options = {
    archive-name = lord@emf.net--2004
  }

Modules

The modules configuration block defines the list of modules to be checked out of source control and built. The keys in the block form the short names for the modules, used, for example, in creating filenames for assets relating to the module, and the name of the checkout directory under the source root. If building multiple branches of a module, it is common to post-fix the module name with a version / branch name.

  modules = {
    mymod-1.0 = {
      .. definition of mymod version 1.0..
    }
    mymod-dev = {
      .. definition of mymod development snapshot...
    }
  }

Within the configuration block of an individual module the following options are permitted

label

The free-text display name for the module, typically used in HTML status pages, and email alerts.

  label = Test-AutoBuild (Development branch)

source

This block defines the repository containing the source to be checked out for the module. There are two keys in the block, the value associated with the key repository, is the name of a repository previously defined in the config file. The value associated with the key path is the path to checkout within the repository. The syntax for path values is dependant on the type of repository being accessed. For details refer to the manual pages for the corresponding modules:

CVS: Refer to Test::AutoBuild::Repository::CVS
GNU Arch: Refer to Test::AutoBuild::Repository::GNUArch
Subversion: Refer to Test::AutoBuild::Repository::Subversion
Perforce: Refer to Test::AutoBuild::Repository::Perforce
Mercurial: Refer to Test::AutoBuild::Repository::Mercurial
SVK: Refer to Test::AutoBuild::Repository::SVK
Local disk: Refer to Test::AutoBuild::Repository::Disk

An example config entry for a module checked out of CVS is

    source = {
        repository = gna-cvs
        path = /testautobuild
    }

sources

If a module's source is split amongst several locations, this block is used instead of the source block. It allows defintion of a list of source paths to checkout. It is a list, where each entry matches the format of the source parameter. For example

  sources = (
    {
      repository = gna-cvs
      path = /testautobuild
    }
    {
      repository = local-disk
      path = /testautobuild-autobuild.sh -> autobuild.sh
    }
  )

NB, not all repository types play nicely together when checking out from multiple paths. Consult manual pages for individual repository types for futher information

groups

Lists the groups to which the module belongs. The values in the list must be group names, specified earlier in the top level groups configuration block.

  groups = (
    software
    perl
  )

env

Defines a set of environment variables which will be set whenever running the build/test control files for the module. The only restriction on variables set here, are that none should be named with the prefix AUTOBUILD_, otherwise they are liable to be overridden by variables set by the build engine.

  env = (
    SKIP_TESTS = 1
  )

options

The options parameter is used to specify module specific data which will be used by stages in the workflow engine. Consult the manual pages for individual stages in use, for further details on which options are possible. The most common option is control-file which can be used to override the default name of the command to invoke to run the build. For compatability with version 1.0.x of autobuild, this should be set to 'rollingbuild.sh'

  options = {
    control-file = rollingbuild.sh
  }

links

The links configuration block defines a simple list of hyperlinks relating to the module. This is typically used to provide links to a graphical front end for the source repository, or a link to a project homepage. The two keys with the block are href and label

  links = (
    {
      href = http://www.autobuild.org/
      label = Project homepage
    }
    {
      href = http://cvs.gna.org/viewcvs/testautobuild/testautobuild/?only_with_tag=RELEASE_1_0_0
      label = Browse source code
    }
  )

artifacts

The artifacts configuration block defines a list of build artifacts which will be published to the distribution sites. This is typically used to provide access to items such as build reports on code coverage, code complexity, bug analysis, etc, or metadata files such as the module's README, or copyright notices. With the block, the src parameter is a filename glob relative to the base of the module's code checkout; the dst parameter is the name of the destination file (or directory if the source glob matches multiple files), and will also form the URL string; the label key gives a label for hyperlinks to the artifact, and finally the publisher is the name of a file publisher, as defined in the top level publishers config block.

  artifacts = (
    {
      src = README
      dst = README
      label = Readme
      publisher = copy
    }
    {
      src = blib/coverage/*
      dst = coverage/
      label = Code Test & POD coverage Reports
      publisher = copy
    }
  )

Groups

The following options define grouping of modules, primarily used for grouping modules in the HTML status display. The keys in the configuration block for the short group name, used when defining group membership in the module configuration.

  groups = {
    perl = {
      label = Perl modules
    }
    software = {
      label = Software
    }
    docs = {
      label = Documentation
    }
  }

The following options are allowed within each group definition

label: The free-text display name of the group

Package types

The following options define binary package types to detect and publish.

Publishers

The following options define mechanisms for publishing files to distribution directories.

Platforms

The following options define aspects of the host platform

Build archive

The following options define the mechanism used for archiving module build output between build cycles.

Workflow stages

The following options defined the workflow followed for a build cycle

METHODS

$builder = Test::AutoBuild->new(config => $config);

Creates a new builder instance, loading configuration parameters from the $config parameter, whose value is an instance of the Config::Record module.

$config = $builder->config([$name, [$default]]);

If invoked with no arguments returns the Config::Record object storing the builder configuration. If invoked with a single argument, returns the configuration value with the matching name. An optional default value can be provided in the second argument

$builder->run();

Executes the build process. This is the heart of the auto build engine. It performs the following actions:

 * Reads the list of modules, source control repositories,
   package types and output modules from the configuration
   file
 * Initializes the build cache
 * Takes out an exclusive file lock to prevent > 1 builder
   running at the same time.
 * Changes the (nice) priority of the AutoBuild process
 * Checks the code for each module out of its respective
   source control repository.
 * Does a topological sort to determine the build order
   for all modules
 * For each module to be built:
    - Take a snapshot of the package & virtual root install
      directories
    - Change to the top level source directory of the module
    - Run the rollingbuild.sh script
    - Take another snapshot & compare to determine which
      files were install in the virtual root & which packages
      were generated
    - Save the intsalled files and packages in the cache.
 * Invoke each requested output module, for example, HTML
   status generator, package & log file copiers, email
   alerts

my \%groups = $autobuild->load_groups();

Creates the Test::AutoBuild::Publisher objects for each publisher defined in the build configuration.

my \%publishers = $autobuild->load_publishers();

Creates the Test::AutoBuild::Publisher objects for each publisher defined in the build configuration.

my \%monitors = $autobuild->load_monitors();

Creates the Test::AutoBuild::Monitor objects for each monitor defined in the build configuration.

my \%platforms = $autobuild->load_platforms();

Creates the Test::AutoBuild::Platform objects for each platform defined in the build configuration.

my \@repositories = $autobuild->load_repositories();

Creates the Test::AutoBuild::Repository objects for each repository defined in the build configuration.

my \%package_types = $autobuild->load_package_types();

Creates the Test::AutoBuild::PackageType objects for each package type defined in the build configuration.

my \%modules = $autobuild->load_modules();

Creates the Test::AutoBuild::Module obkjects for each module defined in the build configuration.

TODO

The task tracker on the Gna! project site (www.autobuild.org) contains a list of all things we'd like to do.

Oh and 100% Pod & code test coverage - Devel::Cover WILL EAT YOUR BRAAAAANE!

BUGS

Probably a few, so report them to the bug tracker linked from the Gna! project site www.autobuild.org.

AUTHORS

Daniel P. Berrange, Dennis Gregorovic

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

SYNOPSIS

DESCRIPTION

SETUP

CONFIGURATION

General runtime

Source repositories

Modules

Groups

Package types

Publishers

Platforms

Build archive

Workflow stages

METHODS

TODO

BUGS

AUTHORS

COPYRIGHT

SEE ALSO

NAME

SYNOPSIS

DESCRIPTION

SETUP

CONFIGURATION

General runtime

Source repositories

Modules

Groups

Package types

Publishers

Platforms

Build archive

Workflow stages

METHODS

TODO

BUGS

AUTHORS

COPYRIGHT

SEE ALSO

Module Install Instructions