NAME
FleetConf - An multi-agent, configuration management tool
SYNOPSIS
perl Build.PL install_base=/master/install/dir/FleetConf
./Build
./Build test
./Build install
vim /master/install/dir/FleetConf/etc/fleet.conf
/master/install/dir/FleetConf/bin/fleetconf -v
/usr/local/Fleetconf/bin/fleetconf
DESCRIPTION
This is a highly configurable, Perl-based, multi-agent, configuration
management tool. It was originally designed by the author to handle the
account management tasks for a mixed platform environment. In order to
create accounts, he needed to create directory users on a Microsoft
Windows Server 2003, create home directories and establish quotas on a
Linux file server, add the user to mailing lists on another Linux mail
server, monitor the process, and notify the user when the operations
were complete.
This system has been further generalized to perform nearly any common
configuration task needed to be performed across multiple systems. It's
meant to perform actions similar to (and is being used by the author to
replace) cfengine.
HOW DOES IT WORK?
The front-end for the tool is fleetconf. Though, the real work is all
started within the Perl module named FleetConf and through another set
of Perl modules used by FleetConf::Agent. (Therefore, if FleetConf
doesn't quite do it right for your needs, it should be a simple matter
to modify fleetconf or create a new similar script to perform the same
purpose.) For details on how to use fleetconf see it's documentation.
Essentially, the FleetConf system starts by determining the root
directory of FleetConf. Using fleetconf, this is determined by moving
upward to the parent directory that script is being run from. Within
this root, the system opens and reads the contents of etc/fleet.conf,
which contains some initial settings (see "CONFIGURATION" below).
Then, one or more plugin directories are loaded. Each plugin is a Perl
file with the suffix ".pl" or ".pm" and may do whatever it likes to
modify the internals of the system. Usually, this involves defining new
agent language commands or defining helper functions (in the
"FleetConf::Functions" namespace) or helper commands (in the
"FleetConf::Commands" namespace). However, these can be used to modify
any aspect of the published API they want.
Following this, a series of "workflows" are loaded, as defined in the
configuration file (and possibly plugins). Agents use the workflows to
determine what to do and also to record their progress in doing it.
Next, one or more agent directories are loaded. All files ending in
".agent" are read using the agent parser. The agent parser compiles
every agent script found. Each agent is defined in a fairly simple
command-based language that can be extended using plugins. For a full
definition of the language see FleetConf::Agent::Parser, and any plugins
you have loaded (the included plugins are linked from
FleetConf::Agent::Parser).
Finally, each agent is run. Each agent runs over each record found in
the given workflow system and may perform an action permitted by the
agent language. Each iteration involves running the agent in 5 phases
(plus a phase before iteration begins). It is likely that future
versions will feature an additional phase at the end and perhaps other
phases. For a full description of running and the phases see
FleetConf::Agent. For a description of which commands run in which
phases, you'll need to see the FleetConf::Agent::Parser and command
plugin documentation.
HOW DO I USE IT?
The first step, of course, is to install it. Then, you will need to
configure it and write some useful agents. Then, you need to run it all
over the place to copy it locally.
For information on installation, see "INSTALLATION". For information on
configuration, see "CONFIGURATION". For information to get started with
agents, see FleetConf::Agent.
FleetConf API
The public API is documented to allow contributors to learn about the
system, for users to create plugins that take full advantage of the API,
and to keep the author from forgetting what something is supposed to do.
In generally, I try to keep the documentation up to date with the actual
program, but there will be lapses. Please let me know about them by
filing a bug at <http://rt.cpan.org/> if you discover such a lapse.
API OVERVIEW
The FleetConf object exists as a singleton created sometime after
startup. (For those who may not know, a "singleton" is approximately
what it sounds like: it is an object for which there can be only one for
each Perl process. That means that there should only ever be one of
these objects-per-run of fleetconf.) This singleton is accessed (and,
during the first call, created) by calling the "instance" method of
FleetConf.
Prior to calling the "instance" method you will probably want to
configure the global variables, which are used to determine certain
parts of the configuration.
GLOBAL VARIABLES
NOTE: Please be aware that these variables may change. These were an
easy mechanism of communication between fleetconf and FleetConf
originally, but aren't very good solutions. Future revisions may include
accessors from the FleetConf singleton so that their effects can be
modified after startup. Currently, if you set these after the "instance"
method is called, that change may have no effect. However, it's likely
that I will keep these around and use some devious "tie"ing to allow
them to be effective. So perhaps my warning doesn't matter anyway.
This documentation shows the fully qualified name of each global
variable and the initial/default value of that variable (if any).
$FleetConf::verbose = 0;
The verbose setting is used to determine how much information to
output while running. All output *should* be routed through the
logger (see $FleetConf::log below), which is configured according to
the value of this setting when "initialize" is called.
When this setting is set to "0", the default log level of "warning"
or "error" is used, meaning that anything below that threshold is
not output. (See $FleetConf::no_warnings for more information.) If
verbose is set to "1", then the log level is set to "notice". If
verbose is set to "2", the log level is set to "info". If verbose is
set to "3", the log level is set to "debug". Higher levels will
result in greater amounts of debug information being displayed. At
this time, I believe level "4" is the highest implemented debug
level, but more might be used in the future.
$FleetConf::pretend = 0;
All plugins should be careful to check the value of this variable.
If this value is set to a true (non-zero) value, then no changes
should be made to the system, the workflow, or anything. Plugins and
commands should merely report back as if they were doing that work
(or may log messages stating that we're pretending to do something,
whatever is most appropriate).
This flag allows the user to safely run and see what would happen
without actually causing anything to happen.
Setting this value to will cause verbose to be set to at least a
value of "1" (so that log level is set to "notice").
$FleetConf::no_warnings = 0;
In some cases, it might be desireable to hide warnings from standard
output. Setting this to a true (non-zero) value will mean that the
log level will be set to "error" rather than "warning" as long as
neither $FleetConf::verbose nor $FleetConf::pretend are non-zero.
$FleetConf::fleetconf_root = Cwd::getcwd;
This value is very important to set if you are writing an
alternative to the fleetconf front-end. This tells FleetConf which
directory to use as the basis for all relative file names and where
to find the configuration file (i.e., etc/fleet.conf inside of this
directory).
it defaults to the current working directory of the current Perl
interpreter, but this is a poor default.
$FleetConf::log = ...;
This log mechanism is deprecated. Please use FleetConf::Log instead.
This is still used, but only internally.
$FleetConf::log_file;
Name the log file to log to. If none is given, then logging only
goes to standard error. This file will be appended to rather than
overwritten.
$FleetConf::log_stderr = 1;
If this is set to a false value, then logging will not be printed to
the terminal on standard error.
%FleetConf::globals;
This global is setup initially from the "globals" configuration
option. It may contain any information a plugin or agent wants to
store here. It may be modified by any agent or plugin and those
changes stay until changed by a later agent or plugin or when
FleetConf quits. The contents of this variable are not saved between
runs of FleetConf.
METHODS
The most important method is the "instance" method, which returns the
singleton FleetConf instance that is used to call most of the other
methods.
$fleetconf = FleetConf::instance
$fleetconf = FleetConf->instance
This method returns the singleton reference to the FleetConf object.
This is the starting point for all things FleetConf in the API. See
the rest of the method documentation below to find out what you can
do with it.
The first time this method is called, it will perform the first four
phases of loading based upon the globals (see "GLOBAL VARIABLES"
above).
Prologue
Before doing anything it will determine the log level that
should be used to output to the screen (via standard error). It
then resets the logger to use this log level on the screen
(prior to this call, the logger will output anything to the
screen as it's rare that it should be used at all prior to this
point).
It also performs a late loading of the FleetConf::Agent library
(which shouldn't, in general, be loaded directly by any plugin
or application). This late loading happens so that we can
capture the output of the parser trace for debugging purposes.
Phase 1: Configuration
Loading the configuration happens next. See "CONFIGURATION" for
details on what is stored there.
Phase 2: Plugins
We search all directories specified by the plugin search path
for any file ending in ".pl" or ".pm" and we run that file as if
it were Perl. We report as errors or warnings any plugin that
fails to load correctly, but we otherwise ignore them.
Most of the time, FleetConf ignores errors and continues despite
them because it's possible that unrelated parts of the system
should run anyway. If an error should stop some other part of
the system from running a plugin should make sure and fail
cleanly itself and alter the state of the system somehow to
prevent the other system from failing (the %FleetConf::globals
would be a good place to do this).
Phase 3: Workflows
Based upon the configuration in the "workflows" variable, we
will load workflow instances into the system. Agents then
request named workflow instances to determine how they proceed.
One workflow instance is ubiquitous and loaded regardless of any
other configuration. This workflow is called "Null" and is
defined as a basic instance of FleetConf::Workflow::Null. This
module basically always returns a single empty workflow record
and uses the $FleetConf::log logger to record changes. See
FleetConf::Workflow::Null for more information.
Phase 4: Agents
Finally, we search all of the agent include directories for
files ending with ".agent" and compile the agents there
according to the "load_file" method of FleetConf::Agent. (See
that module for more documentation on this process.)
At this point everything that will be loaded is loaded, but (unless
a plugin does something) no action should have been taken yet. The
returned object is ready to continue.
$config = FleetConf::configuration
$config = FleetConf->configuration
$config = $fleetconf->configuration
This method can be used to fetch the configuration file after it is
read in. If this method is not called on a FleetConf instance, the
"instance" method will be called to get one and the configuration
method called on that (i.e., don't call this if you aren't ready to
call "instance").
For details on what this variable will contain, see "CONFIGURATION".
$fleetconf->select_agents(@selectors)
This method allows the user to specify which agents should be run.
In some cases, it may be desireable to limit which agents are run
externally (i.e., agents can internally refuse to run based on the
situation as well). If only some of the agents should be run, then
this method should be used to select the agents.
Each element of @selectors is a subroutine (code reference) that
takes an FleetConf::Agent object reference as it's only argument. An
agent will be used if *any* selector returns a true (non-zero) value
when pased that agent.
This call affects any call to "run_agents" following until this
method is called again. Each call to this method replaces the entire
list of selectors. Calling this method with no arguments will cause
all agents to be selected if immediately followed by a call to
"run_agents".
@agents = $fleetconf->agents
This method returns a list of agents that are currently selected
(according to the last call to "select_agents") or all agents if no
selectors are set. See "select_agents" for information on how to
select agents.
$fleetconf->run_agents
Runs all the agents that have been selected (i.e., those that would
be returned by "agents"). This basically means that each agent is
loaded and the "run_foreach" method is called with the requested
workflow passed.
CONFIGURATION
The configuration is stored as a YAML file. Please see the documentation
of YAML for information on how YAML works. The top-level of the
FleetConf configuration file, fleet.conf, is a hash containing keys
pointing to the different configuration directives. At this time, there
are very few directives.
For a basic and simple configuration file illustrating these directives,
see the file etc/fleet.conf.sample included in the distribution.
Each are described here:
agent_include
This is an array containing the name of the directories to search
for agents.
For example:
agent_include:
- agents
- /etc/FleetConf/agents
Here, all agents found in the directory or subdirectories of agents
in the FleetConf root directory will be searched first and then
those found in /etc/FleetConf/agents will be loaded second.
plugin_include
This is an array containing the name of the directories to search
for plugins.
For example:
plugin_include:
- plugins
- /etc/FleetConf/plugins
Here, all agents found in the directory or subdirectories of plugins
in the FleetConf root directory will be searched first. Then, those
found in /etc/FleetConf/plugins will be leaded second.
workflows
This option contains a hash of the all the workflows to load. Each
hash key is the name of the workflow instance. Each of these should
then be a hash containing two keys, "class" and "args". The "class"
key is a scalar naming the Perl class that will be used to create
the workflow instance (and should already be loaded as a plugin).
The "args" will be set to whatever the "new" method of the given
"class" expects to set it up.
For example:
workflows:
new_jobs:
class: FleetConf::Workflow::RT
args:
proxy: https://example.com/cgi-bin/rt-soap-server.pl
query: Queue='Jobs' AND Status='new'
ssl_client_certificate: /etc/ssl/client/rt-workflow.crt
ssl_client_key: /etc/ssl/client/rt-workflow.key
open_jobs:
class: FleetConf::Workflow::RT
args:
proxy: https://example.com/cgi-bin/rt-soap-server.pl
query: Queue='Jobs' AND Status='open'
ssl_client_certificate: /etc/ssl/client/rt-workflow.crt
ssl_client_key: /etc/ssl/client/rt-workflow.key
This will instantiate two workflow instances: one named "new_jobs"
and another named "open_jobs". If a workflow requires more
complicated work to instantiate it, that work may be done with a
plugin (though, the way to do that isn't part of teh published API
yet as this isn't necessarily safe if you upgrade in the future).
globals
This option sets the initial value of %FleetConf::globals. Thus, it
should, obviously, be a hash.
For example:
globals:
master_path: /common/admin/FleetConf
local_path: /usr/local
This would set $FleetConf::globals{master_path} to
"/common/admin/FleetConf" and $FleetConf::globals{local_path} to
"/usr/local".
INSTALLATION
Installation follows the typical Module::Build template with one
important caveat. FleetConf is meant to operate in a more controlled
environment, so "install_base" is automatically set to
/usr/local/FleetConf. That is, by doing the typical (such as installing
with CPAN):
perl Build.PL
./Build
./Build test
./Build install
The files will all be dropped within /usr/local/FleetConf and
subdirectories within that folder. It is strongly recommended that you
install FleetConf into it's very own directory, which is why you will
receive a complaint if you try to install into a directory not named
"FleetConf", just to try and make sure you're aware of this if you have
not yet read the docs.
Thus, the more typical installation will look like this:
perl Build.PL install_base=/some/master/root/FleetConf
./Build
./Build test
./Build install
Just in case you hate using capitals, no complaint will occur if you
install using all lower case, e.g., /usr/local/fleetconf.
After installation, you will need to create a file named etc/fleet.conf
inside the installation directory and add any agents you like.
At this point, one agent is installed automatically, update.agent, which
will update a local installation from a global one. Delete this if you
don't want it. Future versions might include this as an
update.agent.sample or something similar. You will need to check the
agent folder and make sure you have all the agents you want there. You
may also want to add plugins.
The rest is up to you. At this point, FleetConf has no standard policies
on how things should be done. This will likely remain the case until
either I find something useful, or a user community starts choosing
those policies. I have no idea what would be best in general or what
tricks might be handy.
SEE ALSO
FleetConf::Agent, FleetConf::Agent::Parser, FleetConf::Commands,
FleetConf::Functions, FleetConf::Workflow::Null,
FleetCOnf::Workflow::RT, Log::Dispatch, Module::Build, CPAN
AUTHOR
Andrew Sterling Hanenkamp, <hanenkamp@users.sourceforge.net>
COPYRIGHT AND LICENSE
Copyright 2005 Andrew Sterling Hanenkamp. All Rights Reserved.
FleetConf is licensed and distributed under the same terms as Perl
itself.