
Net::FullAuto - Perl Based Secure Distributed Computing Network Process Automation Utility

This is a BETA RELEASE of Net::FullAuto.
Please contact me at my email address -
and let me know of ANY and ALL bugs, issues, problems, questions as well as suggestions for improvements to both the documentation and module itself. I will make every effort to get back to you quickly.
Update the module from CPAN *often* - as I anticipate adding documentation and fixing bugs and making improvements often.
THANKS - and GOOD LUCK with your Net::FullAuto project(s)!
Brian Kelly, August 31, 2012

Beware that this software is provided "as is", and comes with no warranty of any kind, either express or implied. If you use the contents of this distribution, you do so at your own risk, and you agree to free the author(s) of any consequences arising from such use, either intended or otherwise.

use Net::FullAuto;
see METHODS section below

Net::FullAuto is a Perl based Secure Distributed Computing Network Process Automation Utility. It's a MOUTHFUL - and it IS everything implied in it's description. Net::FullAuto is a command environent based implementation that truly embodies the term "The Network **IS** the Computer!!"
Net::FullAuto utilizes ssh and sftp (can also use telnet and ftp, though for security reasons, this is NOT recommended) to bring the command enviroments of any number of remote computers (OS of remote computer does not matter), together in **ONE** convenient scripting space. With Net::FullAuto, you write code once, on one computer, and have it execute on multiple computers simultaneously, in an interactive dynamic fashion, AS IF the many computers were truly ONE.
Net::FullAuto is POWERFUL. Net::FullAuto can be run by a user in a Menu driven, interactive mode (using the Term::Menus module - also written by Brian Kelly), OR via UNIX or Windows/Cygwin cron in a fully automated (and secure) fashion.
Example: A user needs to pull data from a database, put it in text file, zip and encrypt it, and then transfer that file to another computer on the other side of the world via the internet - in ONE step, and in a SECURE fashion.
Net::FullAuto is the ANSWER! Assume Net::FullAuto is installed on computer one, the database is on computer two, and the remote computer in China is computer three. When the user types the script using Net::FullAuto, Net::FullAuto will connect via ssh AND sftp (simultaneously) to computer two, and via sftp to computer three. Using an sql command utility on computer two, data can be extracted and piped to a text file on computer two. Then, Net::FullAuto will run a command for a zip utility over ssh on computer two to zip the file. Next (assume the encryption software is on computer one) Net::FullAuto will transfer this file to computer one, where it can be encrypted with licensed encryption software, and then finally, the encrypted file can be transferred to computer three via sftp. Email and pager software can be used for automated notification as well.
Example: The same process above needs to run at 2:00am unattended.
No Problem! A script using Net::FullAuto can be run via cron to perform the same actions above without user involvement.
Net::FullAuto is RELIABLE and FAULT TOLERANT. Each individual command run on a remote computer returns to Net::FullAuto BOTH stdout (output) and stderr (error messages). With this feature, users and programmers can write code to essentially trap remote errors "locally" and respond with a host of error recovery approaches. Everything from sending an e-mail, to re-running the command, to switching remote computers and much more is available as error handling options. The only limits are the skills and ingenuity of the programmers and administrators using Net::FullAuto. If Net::FullAuto loses a connection to a remote host, automatic attempts will be made to re-connect seemlessly - with errors reported when the specified number of attempts fail.
Net::FullAuto is EASY. Net::FullAuto uses a mix of traditional and object-oriented features to maximize ease of use and maintenance. Due to the unique nature of distributed computing, combined with the need for ease of maintaining a lot of configuration information (i.e. ip addresses, host names, login ID's, passwords, etc), along with any number of *routines* or *processes*, as well as the need for robust security, Net::FullAuto has a unique layout and architechture. Normally in perl, programmers segregate functional code in separate script files or perl modules or packages. Net::FullAuto supports this as well, but advocates keeping *process* code confined to a single routine in a kind of "process library" file. This is in order that Net::FullAuto can provide additional built-in features like a command-handle to the local machine without having to explicitly create it. Or, being able to connect to a remote host with syntax as simple as:
$computer_one=connect_ssh('COMPUTER_ONE');
IT REALLY IS THAT EASY!
Commands also are easy:
($stdout,$stderr)=$computer_one->cmd('ls -l');
And NO CLEANUP is necessary - Net::FullAuto handles this AUTOMATICALLY.
This is a COMPLETE *routine* or *process*:
sub ls_one {
my ($computer_one,$stdout,$stderr); # Scope Variables
$computer_one=connect_ssh('COMPUTER_ONE'); # Connect to Remote Host
($stdout,$stderr)=$computer_one->cmd('ls -l'); # Run Command
if ($stderr) { # Check Results
print "We Have and ERROR! : $stderr\n";
} else {
print "Output of ls command from Computer One:\n\n$stdout\n\n";
}
} # DONE!!
AGAIN - IT REALLY IS THAT EASY!
As with most things in life, what many or most consider a blessing, others consider a curse. Perl's motto is "There's more than one way to do it." (TIMTOWTDI) Not everyone thinks this is utopia. Perl also attempts "to make easy tasks easy and difficult tasks possible." Net::FullAuto - written in perl - *IS* PERL. It is essentially a perl extension and therefore adheres to the same goals as perl itself: i.e. - there's no "one" correct way to use Net::FullAuto.
Net::FullAuto is SECURE. It uses ssh and sftp for communication accross computers, and uses powerful encryption to store passwords to remote resources. When running Net::FullAuto, a user on the first iteration of a process will be prompted to enter a password for each and every remote resource (or even local resource, since Net::FullAuto can and does use ssh to acquire enhanced user-rights on the local computer.) Every following iteration will then prompt for a password ONLY ONCE (or a password can even be passed in via command or method arguement) with every other needed password retrieved from an encrypted datafile which utilizes the user's main login password as the "salt".
For added security, and enhanced user functionality, Net::FullAuto can be installed on UNIX computers to use setuid. (Windows/Cygwin does not support "setuid" - so this feature is not available on Windows computers. This is the ONLY Windows limitation.) With Net::FullAuto setup to use setuid, users can be configured to run complex distributed processes in a secure fashion without the permissions ACTUALLY needed by the remote (or even local) resources. On top of that, it is possible to create a process administered by numerous individuals such that NO ONE PERSON KNOWS OR HAS ACCESS TO ALL THE PASSWORDS. For example, a database administrator on a remote computer can "loan" his username and password to drop a table (for instance) for a process that will be run by another user remotely. During the first iteration, after the user enters her/his password, the DB can then (when prompted), enter his/her password which will then be encrypted LOCALLY with the user's password as the salt. With the encrypted datafile and perl code protected from user write (or even read) access via setuid on UNIX computers (setup and administered by yet another individual or group - such as the root user), there is no way for either the DB to discover the user's password, or the user to discover the DB's password! Even the root user of the local computer runningNet::FullAuto will not be able to discover these passwords! (When setuid is setup and used PROPERLY). This setup will allow users to run FullAuto processes WITHOUT access to the passwords controlling remote access, or for that matter, the CODE running those processes!
Reasons to use this module are:
In the file fa_hosts.pm add the connection information for the remote computer (This will suffice for all following examples):
{
'Label' => 'COMPUTER_ONE',
'IP' => '10.200.210.37',
'HostName' => 'compter_one.w2k.fullautosoftware.net',
'Login' => 'bkelly',
'LogFile' => "/cygdrive/d/fullauto/logs/FAlog${$}d".
"${FA_Core::invoked[2]}".
"${FA_Core::invoked[3]}.txt",
},
In the file fa_code.pm add the *process* subroutine code:
sub ps_one {
my ($computer_one,$stdout,$stderr); # Scope Variables
$computer_one=connect_ssh('COMPUTER_ONE'); # Connect to
# Remote Host via
# ssh only
($stdout,$stderr)=$computer_one->cmd('ps -e'); # Run Command
if ($stderr) { # Check Results
print "We Have and ERROR! : $stderr\n";
} else {
print "Output of ps -e command from Computer One:".
"\n\n$stdout\n\n";
}
}
Run script using Net::FullAuto (Hint: the --<...>-- line are instructions and are not displayed when the program actually runs)
--< 1 >-<Type Command and <ENTER> >---------------------------
fullauto --code ps_one
--< The user sees: >------------------------------------------
STARTING FULLAUTO on Wed Jun 6 12:27:08 2007 Starting fullauto . . . Running in TEST mode computer_one Login <bkelly> :
--< 2 >-<ENTER>-(Hint: since 'Login' was specified in fa_hosts.pm 'bkelly' appears as the default)-
Password:
--< 3 >-<Type Password and <ENTER> >--------------------------
--> Logging into localhost via ssh . . .
Logging into computer_one.w2k.fullautosoftware.net via ssh . . .
Output of ps -e command from Computer One:
PID TTY TIME CMD
1 ? 00:00:03 init
2 ? 00:00:00 migration/0
3 ? 00:00:00 ksoftirqd/0
80 ? 00:00:00 aio/0
2805 ? 00:00:08 syslogd
2820 ? 00:00:00 irqbalance
2839 ? 00:00:00 portmap
2859 ? 00:00:00 rpc.statd
2891 ? 00:00:00 rpc.idmapd
2949 ? 00:00:00 ypbind
2969 ? 00:00:45 nscd
2987 ? 00:00:01 smartd
2997 ? 00:00:00 acpid
3059 ? 00:00:00 xinetd
3072 ? 00:00:14 ntpd
3092 ? 00:00:19 sendmail
3111 ? 00:00:00 gpm
3121 ? 00:00:03 crond
3153 ? 00:00:00 xfs
3172 ? 00:00:00 atd
3188 ? 00:00:00 dbus-daemon-1
3201 ? 00:05:09 hald
3210 tty1 00:00:00 mingetty
1432 ? 00:02:34 rvd
14675 ? 00:00:00 kdbd
17052 ? 00:00:00 postmaster
24389 ? 00:00:00 chatserv_d
16463 ? 00:00:06 java
11700 ? 00:04:48 cmefx
905 ? 00:00:00 automount
563 ? 00:00:00 sshd
564 pts/30 00:00:00 bash
641 pts/30 00:00:00 ps
642 pts/30 00:00:00 sed
FULLAUTO COMPLETED SUCCESSFULLY on Wed Jun 6 12:28:30 2007
zip and transfer a remote file from COMPUTER_ONE to your local computer and then unzip it:
In the file "usr_code.pm" add the *process* subroutine code:
sub get_file_from_one {
my ($computer_one,$stdout,$stderr); # Scope Variables
$computer_one=connect_host('COMPUTER_ONE'); # Connect to
# Remote Host via
# ssh *and* sftp
($stdout,$stderr)=$computer_one->cmd(
'echo test > test.txt'); # Run Remote Command
($stdout,$stderr)=$computer_one->cmd(
'zip test test.txt'); # Run Remote Command
if ($stderr) { # Check Results
print "We Have and ERROR! : $stderr\n";
} else {
print "Output of zip command from Computer One:".
"\n\n$stdout\n\n";
}
($stdout,$stderr)=$computer_one->get(
'test.zip'); # Get the File
if ($stderr) { # Check Results
print "We Have and ERROR! : $stderr\n";
} else {
print "Output of zip command from Computer One:".
"\n\n$stdout\n\n";
}
($stdout,$stderr)=$localhost->cmd(
'unzip test.zip'); # Run Local Command
}
Run script using Net::FullAuto (Hint: the --< # >-- line are instructions and are not displayed when the program actually runs)
--< 1 >-<Type Command and <ENTER> >---------------------------
fullautoS< >--codeS< >get_file_from_one
--< The user sees: >------------------------------------------
STARTING FULLAUTO on Wed Jun 6 12:27:08 2007 Starting fullauto . . . Running in TEST mode computer_one Login <bkelly> :
--< 2 >-<ENTER>-(Hint: since 'Login' was specified in fa_hosts.pm 'bkelly' appears as the default)-
Password:
--< 3 >-<Type Password and <ENTER> >--------------------------
--> Logging into localhost via ssh . . .
Logging into localhost via ssh . . .
Logging into computer_one.w2k.fullautosoftware.net via sftp . . .
Logging into computer_one.w2k.fullautosoftware.net via ssh . . .
Output of zip command from Computer One:
updating: test.txt (stored 0%)
get "/tmp/test.zip"
Fetching /tmp/test.zip to test.zip
/tmp/test.zip 0% 0 0.0KB/s --:-- ETA
/tmp/test.zip 100% 153 0.2KB/s 00:00
Output of zip command from Computer One:
Fetching /tmp/test.zip to test.zip
/tmp/test.zip 100% 153 0.2KB/s 00:00

Net::FullAuto requires some preliminary setup before it can be used.
Net::FullAuto requires a local installation of perl5. To test for perl, type perl -v at the UNIX/Linux command prompt:
$ perl -v This is perl, v5.8.7 built for x86_64-linux-thread-multi Copyright 1987-2005, Larry Wall Perl may be copied only under the terms of either the Artistic License or the GNU General Public License, which may be found in the Perl 5 source kit. Complete documentation for Perl, including FAQ lists, should be found on this system using `man perl' or `perldoc perl'. If you have access to the Internet, point your browser at http://www.perl.org/, the Perl Home Page.
Net::FullAuto is easiest to set up with a working CPAN utility. To test for cpan, type cpan -v at the UNIX/Linux command prompt:
$ cpan -v /usr/bin/cpan script version 1.9, CPAN.pm version 1.9402
If you don't have a working cpan, then perhaps you can find some assitance here:
http://www.thegeekstuff.com/2008/09/how-to-install-perl-modules-manually-and-using-cpan-command/
Otherwise, you will have to manually install the additional modules from the CPAN ( Comprehensive Perl Archive Network - http://www.cpan.org ) that are required by Net::FullAuto. Note also that a number of required modules - IO::Pty, will require the use of a C compiler on your local host. The following CPAN modules are required by Net::FullAuto :
- Module::Build - Module::Load::Conditional - Sort::Versions - Crypt::CBC - Crypt::DES - Digest::MD5 - Digest::SHA - Getopt::Long - HTTP::Date - IO::Pty - IO::Capture::Stderr - IO::CaptureOutput - Capture::Tiny - Test::Deep - LWP - Mail::Internet - Mail::Sender - MemHandle - MLDBM - MLDBM::Sync - MLDBM::Sync::SDBM_File - Net::Telnet - Term::ReadKey - Term::RawInput - Term::Menus >= Version 2.16 - Tie::Cache - Email::Sender - MIME::Entity - String::Random - Data::Password::Check - BerkeleyDB - YAML - URI
Net::Fullauto requires IO::Pty, and this module needs to be compiled locally with a C compiler. Check for an installation of cc or gcc .
$ gcc --version gcc (GCC) 4.5.3 Copyright (C) 2010 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
cpan, install Net::FullAuto.$ cpan cpan shell -- CPAN exploration and modules installation (v1.9402) Enter 'h' for help. cpan[1]> install Net::FullAuto
fa_hosts.pm file.See the fa_hosts.pm Setup and Location section below.
usr_code.pm file.See the fa_code.pm Setup and Location section below.
fa_hosts.pm Setup and Locationfa_hosts.pm. This file can be located in one of two places. There is a default fa_hosts.pm file included with the distribution, and you can locate it wherever Net::FullAuto was installed. Usually this is in the /lib directory under /usr or /usr/local/. A typical location would be /usr/local/lib/perl5/site_perl/5.8/Net/FullAuto/fa_hosts.pm. Hosts blocks can be added directly to this file (provided that file is given write permissions: i.e. chmod u+w fa_hosts.pm)fa_code.pm Setup and LocationNet::FullAuto requires the use of a fa_code.pm module file. This file can be located in one of two places. There is a default fa_code.pm file included with the distribution, and you can locate it wherever Net::FullAuto was installed. Usually this is in the /lib directory under /usr or /usr/local/. A typical location would be /usr/local/lib/perl5/site_perl/5.8/Net/FullAuto/fa_hosts.pm. Custom subroutines can be added directly to this file (provided that file is given write permissions: i.e. chmod u+w fa_code.pm)$fa_hosts location variablefa_hosts.pm files with the $fa_hosts variable.
IMPORTANT! - Be sure that this variable is defined in your invoking script. IT MUST BE PLACED IN A BEGIN {} block BEFORE the use Net::FullAuto; line:
BEGIN { our $fa_hosts='/home/user/my_hosts.pm' }
use Net::FullAuto;
. . .
$fa_code location variablefa_code.pm files with the $fa_code variable.
IMPORTANT! - Be sure that this variable is defined in your invoking script. IT MUST BE PLACED IN A BEGIN {} block BEFORE the use Net::FullAuto; line:
BEGIN { our $usr_code='/home/user/my_code.pm' }
use Net::FullAuto;
. . .
fa_hosts' configuration module file does NOT need to be named fa_hosts.pm . Any name can be used, so long as the internal package name is the same as the file name. For example, a file named host_blocks.pm needs to have the line package host_blocks; as the first line of the file.NOTE: It is common to use BOTH location variables together:
BEGIN { our $usr_code='/home/user/my_code.pm';
our $fa_hosts='/home/user/my_hosts.pm' }
use Net::FullAuto;
. . .
fa_hosts.pm File Contentsfa_hosts.pm showing two host blocks with minimal configuration: package fa_hosts;
require Exporter;
use warnings;
our @ISA = qw(Exporter);
our @EXPORT = qw(@Hosts);
@Hosts=(
#################################################################
## Do NOT alter code ABOVE this block.
#################################################################
## -------------------------------------------------------------
## ADD HOST BLOCKS HERE:
## -------------------------------------------------------------
{
Label => 'REMOTE COMPUTER ONE',
IP => '198.201.10.01',
Hostname => 'Linux_Host_One',
},
{
Label => 'REMOTE COMPUTER TWO',
IP => '198.201.10.02',
Hostname => 'Linux_Host_Two',
},
#################################################################
## Do NOT alter code BELOW this block.
#################################################################
);
## Important! The '1' at the Bottom is NEEDED!
1
fa_code.pm File Contentsfa_code.pm showing two simple subroutines: package fa_code;
require Exporter;
use warnings;
our @ISA = qw(Exporter Net::FullAuto::FA_Core);
use Net::FullAuto::FA_Core;
#################################################################
## Do NOT alter code ABOVE this block.
#################################################################
sub hello_world {
print $localhost->cmd('echo "hello world"');
}
sub remote_hostname {
my ($computer_one,$stdout,$stderr); # Scope Variables
$computer_one=connect_ssh('REMOTE COMPUTER ONE'); # Connect to
# Remote Host via ssh
($stdout,$stderr)=$computer_one->cmd('hostname');
print "REMOTE ONE HOSTNAME=$stdout\n";
}
########### END OF SUBS ########################
#################################################################
## Do NOT alter code BELOW this block.
#################################################################
## Important! The '1' at the Bottom is NEEDED!
1
fa_hosts.pm HOST BLOCK KEY ELEMENTS 
Label - string to identify host block (This is a REQUIRED Element)
Label =>'Any_Unique_String',
The Label Key Element is the method by which Net::FullAuto locates the connection information in the fa_hosts.pm file.
IP => '198.201.10.01',
Hostname => 'Remote_Host_One',
LoginID =>'Username'
LogFile => "/tmp/FAlog${$}d" . "$Net::FullAuto::FA_Core::invoked[2]" . "$Net::FullAuto::FA_Core::invoked[3].txt",
usr_code.pm METHODS 
connect_secure - connect to remote host via ssh & sftp
($secure_host_object,$error) = connect_secure('HOSTLABEL');$secure_host_object = connect_secure('HOSTLABEL');
The $secure_host_object represents both ssh AND sftp connections
together in ONE
object. The HOSTLABEL tag is a label to an
anonymous hash block defined in the file fa_hosts.pm.
(See the fa_hosts section for instructions on configuring host
connection information.)
The important thing to understand, is that there is no other code
needed to connect to remote
hosts. Net::FullAuto handles all
connection details, such as dynamic remote-prompt discovery,
AUTOMATICALLY. No need to define or even know what the remote
prompt is. This feature
'alone' is a major departure from most
other scriptable remote command and file transfer utilities.
connect_ssh - connect to remote host via ssh
($ssh_host_object,$error) = connect_ssh('HOSTLABEL');
$ssh_host_object = connect_ssh('HOSTLABEL');
connect_sftp - connect to remote host via sftp
($sftp_host_object,$error) = connect_sftp('HOSTLABEL');
$sftp_host_object = connect_sftp('HOSTLABEL');
connect_host - connect to remote host via ssh OR telnet and sftp OR ftp
($host_object,$error) = connect_host('HOSTLABEL');
$host_object = connect_host('HOSTLABEL');
connect_insecure - connect to remote host via telnet & ftp
($insecure_host_object,$error) = connect_insecure('HOSTLABEL');$insecure_host_object = connect_insecure('HOSTLABEL');
The $insecure_host_object represents both telnet AND ftp
connections together in ONE
object. The HOSTLABEL tag
is a label to an anonymous hash block defined in the file
fa_hosts.pm.
(See the fa_hosts section for
instructions on configuring host connection information.)
connect_secure() whenever possible.connect_telnet - connect to remote host via telnet
($ssh_host_object,$error) = connect_telnet('HOSTLABEL');
$ssh_host_object = connect_telnet('HOSTLABEL');
connect_ssh() whenever possible.connect_ftp - connect to remote host via ftp
($ftp_host_object,$error) = connect_ftp('HOSTLABEL');
$ftp_host_object = connect_ftp('HOSTLABEL');
connect_sftp() whenever possible.connect_ssh_telnet - connect to remote host via ssh OR telnet
($host_object,$error) = connect_ssh_telnet('HOSTLABEL');
$host_object = connect_ssh_telnet('HOSTLABEL');
connect_telnet_ssh - connect to remote host via telnet OR ssh
($host_object,$error) = connect_telnet_ssh('HOSTLABEL');
$host_object = connect_telnet_ssh('HOSTLABEL');
connect_ssh() whenever possible.connect_sftp_ftp - connect to remote host via sftp OR ftp
($host_object,$error) = connect_sftp_ftp('HOSTLABEL');
$host_object = connect_sftp_ftp('HOSTLABEL');
connect_ftp_sftp - connect to remote host via ftp OR sftp
($host_object,$error) = connect_ftp_sftp('HOSTLABEL');
$host_object = connect_ftp_sftp('HOSTLABEL');
connect_sftp() whenever possible.connect_reverse - connect to remote host via telnet OR ssh and ftp OR sftp
($connect_reverse_object,$error) = connect_reverse('HOSTLABEL');
$connect_reverse_object = connect_reverse('HOSTLABEL');
cmd - run command line commands on the remote host
($cmd_output,$error) = $connect_secure_object->cmd('hostlabel');($ftp_cmd_output,$error) = $connect_secure_object->{_ftp_handle}->cmd('help');($sftp_cmd_output,$error) = $connect_sftp_object->cmd('help');

Brian M. Kelly <Brian.Kelly@fullautosoftware.net>

Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012
by Brian M. Kelly.
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License. (http://www.opensource.org/licenses/gpl-license.php).