Brian Kelly > Net-FullAuto > Net::FullAuto

Download:
Net-FullAuto-0.99999999999904.tar.gz

Dependencies

Annotate this POD

CPAN RT

New  1
Open  1
View/Report Bugs
Module Version: 0.99999999999904   Source  

NAME ^

Net::FullAuto - Fully Automate ANY Process with *Persistent* SSH/SFTP from One Host

NOTE TO USERS ^

Please contact me or my team at the following email addresses -

and let us know of any and all bugs, issues, problems, questions as well as suggestions for improvements to both the documentation and module itself. We will make every effort to get back to you quickly.

Update the module from CPAN *often* - as we anticipate adding documentation and fixing bugs and making improvements often.

THANKS - and GOOD LUCK with your Net::FullAuto project(s)!

Brian Kelly, April 11, 2015

DISCLAIMER ^

 Beware that this software is provided "as is", and comes
 with no warranty of any kind, either express or implied.
 If you use the contents of this distribution, you do so
 at your own risk, and you agree to free the author(s) of
 any consequences arising from such use, either intended
 or otherwise.

SSH SYNOPSIS ^

SFTP SYNOPSIS ^

see METHODS section below

DESCRIPTION ^

Net::FullAuto  (aka FullAuto) is a Perl module and automation framework that transforms Perl into a true multi-host scripting language. It accomplishes this with multiple *PERSISTENT* SSH and SFTP connections to multiple hosts simultaneously. With FullAuto entire hosts are encapsulated in a single filehandle. Think of each filehandle as an always available SSH client (like PuTTY) and SFTP client (like WinSCP) that is available progammatically to the script.

The importance of PERSISTENT connections when attempting to programmatically control remote hosts cannot be over stated. Essentially, it means that FullAuto can automate almost EVERYTHING.

Think about that for a moment.

Talk is cheap. SEEING is believing. To see FullAuto in action, please download and explore the "Self Service Demonstration" at http://sourceforge.net/projects/fullauto. The demo contains an embedded YouTube video (https://youtu.be/gRwa1QoOS7M) explaining and showing the entire automated process of setting up a complex multi-host infrastructure in the Amazon EC2 Cloud. After watching or while watching the video, you can run the demo and standup your own clould infrastructure in just a few minutes. The Hadoop demo is particularly interesting and timely given the recent explosion of BIG DATA and the need to access it more powerfully.

Imagine a scripting language that can turn an entire network of hosts into a virtual single host? This is precisely what FullAuto does.

How is FullAuto different from programs like Chef (http://www.chef.io) and Puppet (http://www.puppetlabs.com) and Ansible (http://www.ansible.com) and Salt (http://www.saltstack.com)? All of which essentially claim the same ability and functionality?

Chef and Puppet and Salt require the use of agents on remote hosts. FullAuto has no such dependency as it is agent-less. It works against any ssh server implementation on any operating system. Ansible claims to be "agent-less" but actually has a dependency on the Python scripting language being available on the remote host, as well as requiring that the OpenSSH daemon on remote nodes be configured to utilize the ControlPersist feature. FullAuto has no such dependency (FullAuto does not even require Perl on the remote nodes!), and if any manual terminal program or utility can connect to a device via ssh or sftp or scp or even telnet or ftpFullAuto can connect as well - persistently!

FullAuto utilizes ssh and sftp (can also use telnet and ftp, though for security reasons, this is NOT recommended) to bring the command enviroments of any number of remote computers (OS of remote computer does not matter), together in one convenient scripting space. With FullAuto, you write code once, on one computer, and have it execute on multiple computers simultaneously, in an interactive dynamic fashion, as if the many computers were truly one.

FullAuto is powerful. FullAuto can be run by a user in a Menu driven, interactive mode (using the Term::Menus module - also written by Brian Kelly), or via UNIX or Linux cron or Windows Scheduler or Cygwin cron in a fully automated (and secure) fashion.

Example: A user needs to pull data from a database, put it in a text file, zip and encrypt it, and then transfer that file to another computer on the other side of the world via the internet - in one step, and in a secure fashion.

FullAuto is the answer. Assume FullAuto is installed on computer one, the database is on computer two, and the remote computer in China is computer three. When the user starts the script using Net::FullAuto, FullAuto will connect via ssh and sftp (simultaneously) to computer two, and via sftp to computer three. Using a sql command utility on computer two, data can be extracted and piped to a text file on computer two. Then, FullAuto will run the command for a zip utility over ssh on computer two to compress the file. Next (assume the encryption software is on computer one) FullAuto will transfer this file to computer one, where it can be encrypted with licensed encryption software, and then finally, the encrypted file can be transferred to computer three via sftp. Email and pager software can be used for automated notification as well.

Example: The same process above needs to run at 2:00am unattended.

A script using FullAuto can be run via cron (or any other scheduler) to perform the same actions above without user involvement.

FullAuto is reliable and fault tolerant. Each individual command run on a remote computer returns to FullAuto BOTH stdout (output) and stderr (error messages). With this feature, users and programmers can write code to essentially trap remote errors "locally" and respond with any number of error recovery approaches. Everything from sending an e-mail, to re-running the command, to switching remote computers and much more is available as error handling options. The only limits are the skills and ingenuity of the programmers and administrators using FullAuto. If FullAuto loses a connection to a remote host, automatic attempts will be made to re-connect seemlessly - with errors reported when the configured number of attempts fail.

FullAuto is easy. FullAuto uses a mix of traditional and object-oriented features to maximize ease of use and maintenance. Due to the unique nature of distributed computing, combined with the need for ease of maintaining a lot of configuration information (i.e. ip addresses, host names, login ID's, passwords, etc), along with any number of routines or processes, as well as the need for robust security, FullAuto has a unique layout and architechture. Normally in perl, programmers segregate functional code in separate script files or perl modules or packages. FullAuto supports this as well, but advocates keeping process code confined to a single routine in a kind of "process library" file. This is in order that FullAuto can provide additional built-in features like a command-handle to the local machine without having to explicitly create it. Or, being able to connect to a remote host with syntax as simple as:

 $computer_one=connect_ssh('COMPUTER_ONE');

Commands also are easy:

 ($stdout,$stderr,$exitcode)=$computer_one->cmd('ls -l');

And no cleanup is necessary - FullAuto handles this AUTOMATICALLY.

This is a complete routine or process:

 sub ls_one {

    my ($computer_one,$stdout,$stderr,$exitcode); # Scope Variables

    $computer_one=connect_ssh('COMPUTER_ONE');      # Connect to Remote Host

    ($stdout,$stderr,$exitcode)=$computer_one->cmd('ls -l');  # Run Command

    if ($stderr) {                                  # Check Results
       print "We Have and ERROR! : $stderr\n";
    } else {
       print "Output of ls command from Computer One:\n\n$stdout\n\n";
    }
 }                                                  # DONE!! 

As with most things in life, what many or most consider a blessing, others consider a curse. Perl's motto is "There's more than one way to do it." (TIMTOWTDI) Not everyone thinks this is utopia. Perl also attempts "to make easy tasks easy and difficult tasks possible." FullAuto - written in Perl - *IS* Perl. It is essentially a Perl extension and therefore adheres to the same goals as Perl itself: i.e. - there's no "one" correct way to use FullAuto.

FullAuto is secure. It uses ssh and sftp for communication accross hosts and devices. FullAuto uses password-less key exchange, but also uses powerful encryption to store passwords to remote resources, when key exchange is not available.

When using passwords with FullAuto (again - password-less key exchange is ALWAYS recommended as it is more secure, passwords should only be used when key exchange is not available), a user on the first iteration of a process will be prompted to enter a password for each and every remote resource (or even local resource, since FullAuto can and does use ssh to acquire enhanced user-rights on the local computer.) Every following iteration will then prompt for a password only once (or a password can even be passed in via command line or method argument) with every other needed password retrieved from an encrypted datafile which utilizes the user's main login password as the "salt" (not to be confused with the program Salt mentioned earlier).

For added security, and enhanced user functionality, FullAuto can be installed on UNIX and Linux based hosts to use setuid. (Windows/Cygwin does not support "setuid" - so this feature is not available on Windows computers. This is the ONLY Windows limitation.) With FullAuto setup to use setuid, users can be configured to run complex distributed processes in a secure fashion without the permissions actually needed by the remote (or even local) resources. On top of that, it is possible to create a process administered by numerous individuals such that no one person knows or has access to all passwords. For example, a database administrator on a remote computer can "loan" his username and password to drop a table (for instance) for a process that will be run by another user remotely. During the first iteration, after the user enters her/his password, the DB can then (when prompted), enter his/her password which will then be encrypted locally with the user's password as the salt. With the encrypted datafile and perl code protected from user write (or even read) access via setuid on UNIX and Linux hosts (setup and administered by yet another individual or group - such as the root user), there is no way for either the DB to discover the user's password, or the user to discover the DB's password. Even the root user of the local computer running FullAuto will not be able to discover these passwords. (When setuid is setup and used properly). This setup will allow users to run FullAuto processes without access to the passwords controlling remote access, or for that matter, the code running those processes.

Reasons to use this module are:

SETUP ^

Net::FullAuto requires some preliminary setup before it can be used.

Setup Checklist

Setup Components

              BEGIN { our $fa_hosts='/home/user/my_hosts.pm' }
              use Net::FullAuto;
              . . . 
              BEGIN { our $fa_code='/home/user/my_code.pm' }
              use Net::FullAuto;
              . . .

 NOTE: It is common to use BOTH location variables together:

        BEGIN { our $fa_code='/home/user/my_code.pm';
                our $fa_hosts='/home/user/my_hosts.pm' }
        use Net::FullAuto;
        . . . 
        package fa_hosts;

        require Exporter;
        use warnings;
        our @ISA = qw(Exporter);
        our @EXPORT = qw(@Hosts);

        @Hosts=(
        #################################################################
        ##  Do NOT alter code ABOVE this block.
        #################################################################
        ## -------------------------------------------------------------
        ##  ADD HOST BLOCKS HERE:
        ## -------------------------------------------------------------

           {
               Label     => 'REMOTE COMPUTER ONE',
               IP        => '198.201.10.01',
               Hostname  => 'Linux_Host_One',
           },
           {
               Label     => 'REMOTE COMPUTER TWO',
               IP        => '198.201.10.02',
               Hostname  => 'Linux_Host_Two',
           },

        #################################################################
        ##  Do NOT alter code BELOW this block.
        #################################################################
        );

        ## Important! The '1' at the Bottom is NEEDED!
        1
        package fa_code;

        require Exporter;
        use warnings;
        our @ISA = qw(Exporter Net::FullAuto::FA_Core);
        use Net::FullAuto::FA_Core;

        #################################################################
        ##  Do NOT alter code ABOVE this block.
        #################################################################

        sub hello_world {

            print $localhost->cmd('echo "hello world"');

        }

        sub remote_hostname {

            my ($computer_one,$stdout,$stderr,$exitcode);   # Scope Variables

            $computer_one=connect_ssh('REMOTE COMPUTER ONE'); # Connect to
                                                     # Remote Host via ssh

            ($stdout,$stderr,$exitcode)=$computer_one->cmd('hostname');

            print "REMOTE ONE HOSTNAME=$stdout\n";

        }

        ########### END OF SUBS ########################

        #################################################################
        ##  Do NOT alter code BELOW this block.
        #################################################################

        ## Important! The '1' at the Bottom is NEEDED!
        1 

 fa_hosts.pm HOST BLOCK KEY ELEMENTS ^

 fa_code.pm METHODS ^

EXAMPLES ^

AUTHOR ^

Brian M. Kelly <Brian.Kelly@fullauto.com>

COPYRIGHT ^

Copyright (C) 2000-2015

by Brian M. Kelly.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU Affero General Public License. (http://www.gnu.org/licenses/agpl.html).

syntax highlighting: