NAME

helm - Easy server and cluster automation

SYNOPSIS

    helm TASK [OPTIONS]
    helm help tasks
    helm help [TASK]

    # patch the same file on all of the machines in your cluster 
    helm patch --file my_fix.patch --target /opt/cool_system/do_stuff.pl

    # run a command on only the servers which run memcache
    helm run "pidof memcached" --roles memcache 

    # copy a file to every server in your cluster
    helm put --local foo.tar.gz --remote /tmp/bar.tar.gz

    # rsync a local folder to specific servers
    helm rsync_put --local lib/important --remote /tmp/important_libs --servers web1,web2,web4

DESCRIPTION

helm is command-line utility to make it easy to automate system tasks for individual machines, a cluster of machines or a subset of machines in a cluster. It has the following features:

Combine multiple commands into a single tasks and to have groups of related tasks gathered together.
Uses SSH as the transport layer for tasks and uses SSH keys for automatic authorization.
Simple optional configuration file describing your cluster, allowing tasks to target a single machine, multiple machines which share the same role, or all machines in your cluster. Can also be extended to pull configuration from more complicated sources like LDAP, etc.
Logging of each action performed to multiple channels: console, log file, irc, email, etc.
Interact with the remote processes via STDIN, STDOUT and STDERR.
Convenient .helmrc file to reduce the number of options you need to pass on every invocation of helm.
Locking on the client and/or server so that multiple invocations of helm aren't running at the same time.

OPTIONS

helm has several options that can be specified for all commands. Any remaining arguments are passed to the task that was invoked. These global options are:

--user

By default, helm assumes you're running the comands as the current local. This works really well with SSH keys set up so that you don't have to try and type in your password on lots of login attempts. However, if you want to change the user, this option allows for that. You will however be prompted for a password if you don't have SSH keys properly set up.

--servers

Specifically list out the hostnames of the servers to send the task to. These can either be full hostnames (or an unambiguous abbreviation of your servers as defined in your CONFIGURATION). Multiple hostnames must me specified in a comma separated list.

If no servers (or roles) are specified, then the task will be performed on all servers (as defined in your CONFIGURATION).

    # using full hostnames 
    --servers 192.168.0.23,192.168.0.24,web1.company.com

    # using abbreviations from config file 
    --servers web1,db2

    # using abbreviations from config file and ranges
    --servers web[1-3],db[1-2]

--roles

Instead of specifying the servers explicitly, you can instead specify which server roles you want to target (as defined in your CONFIGURATION). Multiple roles must be specified in a comma separated list.

If no roles (or servers) are specified, then the task will be performed on all servers (as defined in your CONFIGURATION).

    # single role 
    --roles web

    # multiple roles
    --roles web,cache

--exclude

Specifically exclude servers from the list to be used. This can be combined with both --servers and --roles and even by itself to exclude certain servers from the list of all servers as defined in your CONFIGURATION).

    # combined with --servers
    --servers web[1-10] --exclude web3

    # combined with --roles, using ranges
    --roles web --exclude web[4-5]

--exclude-roles

Specifically exclude servers with certain roles from the list to be used. This can be combined with both --servers and --roles and even by itself to exclude certain servers from the list of all servers as defined in your CONFIGURATION).

    # combined with --servers: any web[1-10] servers that aren't also caches
    --servers web[1-10] --exclude-roles cache

    # combined with --roles: any web servers that aren't also proxies
    --roles web --exclude-roles proxy

--parallel

Execute tasks in parallel on the remote servers. The default is to execute in serial instead.

--parallel-max

The maximum number of parallel processes to be running at the same time if --parallel is used. The default is 100.

--config

Which resource to use for pulling cluster configuration information (see CONFIGURATION).

    --config helm:///etc/helm.conf

--log-level

The log level used. Can be one of debug, info, warn, and error. Defaults to info.

--log

Log messages to a specific channel. Multiple channels can be specified by specifying this option multiple times. The value of this option is the URI of the channel to be used. See "LOG CHANNELS" for more details.

    # send log messages to a comany tech IRC channel 
    --log irc://helm@irc.company.com/tech

    # log messages to a file and to email address
    --log file:///var/log/helm.log --log mailto:helm@company.com?from=helm@company.com

Log messages are also sent to STDERR unless the --quiet option is also passed.

--load

Allows you to load other 3rd party plugin modules to extend helm functionality. This could be to add more log channels, different configuration loading, etc. The value of this option is the full Perl module name of the plugin. This can be specified multiple times.

    # load a hypothetical Yammer log plugin 
    --load Helm::Log::Channel::Yammer

    # load hypothetical custom LDAP configuration and
    --load Helm::Conf::Loader::CompanyLDAP

See "WRITING HELM PLUGINS" for more information.

--sudo

The user that should be used to perform the commands on the remote server. The actual SSH connection will be made using the current user's SSH keys, but then once the connection is made to the remote server, it's sometimes useful for the commands to be run as a different user. We also try to make sure that things like file permissions (on tasks like put and patch) are also handled so that the resulting files are owned by this sudo user.

--lock

This options allows you to have control over whether concurrent helm processes can be running on either the local or remote servers. The value can be one of: none, local, remote, both.

--no-rc-file

Suppress the default loading of the ".helmrc file" file.

--port

The port to use for SSH on all of the remote servers. Defaults to the standard (22).

--timeout

The timeout in seconds to give the ssh connections. Default is 30 seconds.

--continue-with-errors

If a command fails executing against this will normally cause execution to halt and not run against any other remote servers. Using this option will tell helm to continue on to the server(s) even if a previous one has errors.

--quiet

Suppress the default logging to STDERR

--man

Display this documentation

--version

Display the version of Helm installed

--dump-config

Display a dump of the configuration data as understood by Helm

--debug

Tell helm to dump out verbose information about what it is doing internally to a log file named helm.debug in the current working directory. This is different than the --log-level since that is mainly for end user logging of individual task actions. This flag is meant to debug the internals of helm, including the logging subsystem, which is why it's a separate flag.

--sleep

Sleep a given number of seconds in between each server.

TASKS

Helm comes with the following built-in tasks

To get more information on how to run each task, simply use helm help:

    helm help run

run

This is the simplest of helm commands and will run some arbitrary command on the remote server.

get

Get a file from a remote server and put it on the local machine.

put

Put a local file onto a remote server.

rsync_put

Put a local file onto a remote server using rsync rather than a full copy.

patch

Patch a file on a remote server with a patch on the local machine.

unlock

Clear out any stuck Helm locks on a remote server.

CONFIGURATION

By default, helm doesn't use a configuration file, but certain features require it (using roles, server abbreviations, etc) so it's best to have one. You can tell helm which configuration resource to use by using the --config option. Currently, only the helm:// URI scheme is supported.

    --config helm:///etc/helm.conf

A configuration file will look something like this:

    <Server web[1-5].company.com>
        Role web
    </Server>

    <Server db1.company.com>
        Role db Role db_master
    </Server>

    <Server db2.company.com>
        Role db Role db_slave
    </Server>

This configuration would define 7 servers (web1.company.com, web2.company.com, web3.company.com, web4.company.com, web5.company.com, db1.company.com and db2.company.com). It defines 4 different roles (web, db, db_master, db_slave).

helm currently just supports a single configuration resource format (helm://), but the internals are flexible enough that more formats could be supported in the future, including other configuration methods like LDAP, etc.

If, for instance you wanted to support a URI like:

    --config ldap://ldap.company.com

See "WRITING HELM PLUGINS" for more information.

If you are having problems getting your configuration right, you can pass the --dump-config option to tell helm to display what it thinks things are configured to be.

LOG CHANNELS

helm can be told to send various levels of log data to different channels. By default we support a log file, IRC and email logs. We also support various log levels (see the --log-level option).

You can specify which channel is used by giving a URI which indicates what type of channel and where to send the log. The following URI schemes are supported:

file://

This is basically a log file where messages are immediately appended.

    --log file:///var/log/helm.log

irc://

NOTE: IRC logging is still experimental and quite tempermental. Improvements are welcome.

This is an IRC channel where messages are immediately sent. For example to send messages to the sysadmin IRC channel on irc.company.com using the user michael and the password s3cr3t you would have:

    --log irc://michael@irc.company.com/sysadmin?pass=s3cr3t

mailto:

Similar to Mail-To links in HTML documents, this just specifies an email address to log. Log messages aren't sent immediately, but are instead queued up to be sent once the command has been completed.

    --log mailto:michael@company.com?from=helm@company.com

Plugins can be written to load allow other log channels. See "WRITING HELM PLUGINS" for more information on how this is done.

THE .helmrc FILE

The .helmrc file contains command-line options that are prepended to the command line before processing. Multiple options may live on multiple lines. Lines beginning with a # are ignored. A .helmrc might look like this:

    # always log to a file 
    --log file:///var/log/helm.log

    # always load our custom plugins 
    --load Helm::Conf::Loader::CompanyLDAP 
    --load Helm::Log::CompanyYammer
    --load Company::CustomHelmTasks

helm looks for the rc file first in the current directory and then in your home directory. You can specify another location with the HELMRC environment variable.

If --no-rc-file is specified on the command line, the .helmrc file is ignored.

WRITING HELM PLUGINS

Helm can be extended in many ways to make it more convenient for your projects. Helm has 3 customization points where plugins can be written and registered to interact: task, log and configuration. Using the --load option, you can tell Helm about your custom or 3rd party Perl modules that you would like to load. Each plugin module must register itself with Helm along with the type of events it will handle using the Helm->register_module method. For example, if I were creating a new plugin module for a custom task named "spiffy" I might invoke Helm like:

    helm spiffy --load MyCompany::Helm::Task::spiffy

And my Perl module might look something like this:

    package MyCompany::Helm::Task::spiffy;
    use strict;
    use warnings;
    use Helm;
    use Moose;
    extends 'Helm::Task';
    Helm->register_module('task', spiffy => 'MyCompany::Helm::Task::spiffy')

    sub validate {
        my $self = shift;
        # custom validation
    }

    sub execute {
        my ($self, %args) = @_;
        # do something spiffy
    }

    1;

Most likely you'd put the --load statment in your .helmrc file so you wouldn't have to worry about it again. Now we'll get into the details of what is expected of each type of plugin.

Task Plugins

A task plugin should inherit from Helm::Task and implement the following methods:

validate

This method would perform any custom validation needed before the task is executed against any servers. Normally this involves validating the custom options this task might use. This method only receives a single object, the task itself. As an example, lets say you want a --nifty option to your spiffy plugin above:

    helm spiffy --nifty foo

Then your validation method might look something like this:

    sub validate {
        my $self          = shift;
        my $helm          = $self->helm;
        my $extra_options = $helm->extra_options;
        my $nifty         = $extra_options->{nifty};
        $helm->die("You need to provide a --nifty option!") unless $nifty;
        $helm->die("--nifty option ($nifty) is not a valid value") unless $nifty =~ /^fo+/;
    }

execute

This is the meat of your task plugin and is where the work happens. This method is called once for every server the task is being executed against. It receives the following named arguments:

ssh: A Net::OpenSSH object with an already open SSH connection to the server in question.
server: A Helm::Server object for the currently executing task.

As long as your method doesn't die (or preferrably calls $helm->die(), then we will assume that all was fine and dandy in the execution of the task.

Log Plugins

Log plugins can offer new channels for logging Helm output based on the URI given to Helm. For instance, if you wanted to send SMS logging of critical messages only, you might invoke helm with a logger like:

    helm foo --log sms:+15105550101 --load MyCompany::Helm::Log::SMS

And then your SMS module might look something like

    package MyCompany::Helm::Log::SMS;
    use strict;
    use warnings;
    use Helm;
    use Moose;
    extends 'Helm::Log::Channel';
    Helm->register_module('log', sms => 'MyCompany::Helm::Log::SMS');

    sub initialize   {}
    sub finalize     {}
    sub start_server {}
    sub end_server   {}
    sub debug        {}
    sub info         {}
    sub warn         {}
    sub error        { 
        my ($self, $msg) = @_;
        # send SMS message
    }
    1;

In this example, we don't care about anything except errors (since SMS messages cost money and would get really annoying for anything with frequency). Log plugins need to inherit from Helm::Log::Channel and implement the following methods:

intialize
finalize
start_server
end_server
debug
info
warn
error

Configuration Plugins

Configuration plugins can implement new ways to load configuration data about your servers based on the URI given to helm. For instance, if you wanted to load the list of your servers and their roles from a company LDAP server, you might invoke Helm like:

    helm foo --config ldap://ldap.company.com --load MyCompany::Helm::Conf::ldap

Helm will look for the last module registered to handle the ldap scheme of that url. That module might look like:

    package MyCompany::Helm::Conf::ldap;
    use strict;
    use warnings;
    use Helm;
    use Moose;
    extends 'Helm::Conf::Loader';
    Helm->register_module('configuration', ldap => 'MyCompany::Helm::Conf::ldap')

    sub load {
        my ($self, %args) = @_;
        my $uri = $args{uri};
        # poke around in our LDAP server and create a list of Helm::Server objects
        my @servers = ...;
        # then return a Helm::Conf object
        return Helm::Conf->new(servers => \@servers)
    }

    1;

A configuration loading plugin should inherit from Helm::Conf::Loader and implement the following methods:

load

This method is the backbone of a configuration plugin. It receives the following named arguments:

uri: A URI object representing the URI passed on the command line to be loaded by this configuration loader.
helm: The Helm object doing the loading.

This method must create a list of Helm::Server objects and use them to return a Helm::Conf object.

CAVEATS

Multi-Platform: This has been developed and tested on Linux (with bash as the shell on the remote hosts) only. Dealing with multiple platforms and writing multi-platform tasks has been punted to the future.
Alpha Software: This software is very ALPHA, which means it's interface is likely to change in the future. It's used currently and happily in production but please be aware if you start using it that you'll probably want to follow future versions carefully to make sure you aren't bitten by API changes as thing are flushed out.

TODO

In the not too distant future, we'd like to add the following features to Helm:

Add a capture option which allows stdout/stderr to be handled differently
Add a compare option which allows the output (stdout/stderr) to be compared between servers in an intelligent manner
A real exception system to avoid parsing error messages

To install Helm, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Helm

CPAN shell

perl -MCPAN -e shell
install Helm

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)