jcmcken / parallel-ssh Goto Github PK

View Code? Open in Web Editor NEW

19.0 6.0 6.0 532 KB

PSSH provides parallel versions of OpenSSH and related tools.

License: Other

Makefile 0.05% Python 99.95%

parallel-ssh's Introduction

Parallel SSH

The parallel-ssh module and scripts implement parallelized versions of common SSH tasks including:

Arbitrary commands over SSH (the pssh utility)
File copying to and from remote servers over SSH (the pscp and pslurp utilities, respectively)
Syncing files over SSH (the prsync utility)
Forcefully killing processes over SSH (the pnuke utility)

This project utilizes the SSH binaries and libraries already installed on your system, making it ideal for administrators who do not want to manage external dependencies or for servers which need to be bootstrapped to make use of utilities (e.g. Chef, Puppet, Salt) that follow the master-agent paradigm (i.e. require agents to be installed in order to manage servers).

About This Project

Original work on parallel-ssh is by Brent Chun and Andrew McNabb.

This fork of parallel-ssh implements a number of improvements, including major refactoring of the libraries to make them more modular and easier to extend and the addition of numerous extra options for more easily managing, viewing, and saving the status of command runs.

To see a list of all changes, view the ChangeLog.

Examples

One of the most useful additions to the pssh utility in particular are the --script and --sudo options:

$ pssh -h hostfile.txt --script restart_iptables.sh --sudo

These options let you run arbitrary scripts (as root, if needed) on remote hosts, without having to properly escape or make special considerations for the contents of those scripts. All that's required is that the script contain a shebang line referencing an interpreter that exists on the remote host.

All of the utilities received some new common options, some of which include the --host-regexp and --sample-size options. More information can be found by simply typing [utility] --help. (All of the help output for the commands has been reorganized and made easier to read)

parallel-ssh's People

Contributors

Stargazers

Watchers

Forkers

freme glubbered cainiaocome gyf19 pfhayes cthart

parallel-ssh's Issues

Host files with blank lines raise IndexError during parsing

Can't seem to get this to run the commands in parallel even when using --par x

Seems that using --par 3 with --host "host1 host2 host3" still causes the command to be run on each host until completion before moving onto the next. I tested with command "date; sleep 120" and the dates returned are always 2 mins apart.

Implement a --pickled option to serialize results to a pickle file

Implement a --sqlite option to serialize results to a sqlite database

This sqlite database should be persistent across runs, using metadata such as datetime and user to distinguish results

Don't execute scripts directly, run them in a shell

For instances where /tmp is mounted noexec

Add a '-B/--progress-bar' option

Instead of printing all of the task messages, just print a progress bar

Implement --script-args companion option for --script

E.g.

pssh -h hostlist.txt --script do_something.sh --script-args "-y arg1 arg2 blah"

Implement A Manager subclass per Utility

Currently, return codes and summary output are pretty fuzzy. They work and are correct in most cases, but it would be nice to pin down the exact return codes each utility (ssh, scp, rsync, pkill) has and customize the summary output accordingly

Add a '-T/--test' option

Usage would be something like 'pssh $WHATEVER_OPTIONS --test 30', which would run the command against 30 hosts and then stop and ask you if you want to continue on the rest of the hosts. This would be to catch yourself before you destroy your servers with a mistyped command.

Python 2.6 has no _sqlite module

Implement workaround

Implement .psshrc config file for better defaults handling

Invoke summary print from manager instead of inside each bin script

RPM spec file missing BuildRoot directive

Needs something like this:

BuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)

A --sudo companion option for --script

Write the script in a safe place and execute it as root

Implement logging

Who ran what with which options
If a script is passed, save a copy (deduping based on checksum)

Add --logdir configuration option which will generate the following dir structure:
[logdir]/ --pssh/ ----run.log ----scripts/ ------[checksum]-[script1_name] ------[checksum]-[script2_name]

Add a switch in source that requires root to change the logdir parameter

Move summary printing logic into Manager class (and subclasses)

Summary printing is very clunky right now. Make psshlib/ui a little simpler and implement print_summary in each Manager class. Does a ui submodule make sense still?

Add a -F/--fork-hosts option

After running a command, two new host lists will be spit out, one containing all the servers that succeeded and one containing all servers that failed.

Sample usage:

pssh $OTHER_OPTS -h some_host_list.txt --fork-hosts=some_unique_id

... pssh runs ...

Two host files are generated:
* some_unique_id.faillist
* some_unique_id.passlist

This would allow you to easily run follow up tasks if servers fail the first time

Finish man pages for all utilities

Implement "--script" option for PSSH

Supply a script to PSSH. It should then SCP the script to each host, execute it, and remove the temporary script.

Minify scripts

There's a lot of redundancy in the scripts (pssh, pscp, etc.). Refactor to remove this redundancy

Implement --webpage option for PSSH

Create a simple, deployable webpage containing PSSH task data.

So something like...

$ pssh $OTHER_OPTS --webpage

Exported web slug to 'pssh-bf0ac3bb906a96481506dc91c051a8fe.tar.gz'

$ tar xfz pssh-bf0ac3bb906a96481506dc91c051a8fe.tar.gz
$ tree pssh/
pssh
|-- 01-13-2012-01:23:48.html
-- static |-- pssh.css -- pssh.js

1 directory, 3 files
$ cd pssh; python -m SimpleHTTPServer
Serving HTTP on 0.0.0.0 port 8000 ...

Feature request: forced copy on pscp like native linux cp

I use the cp -f option sometimes (for example to upgrade nginx without stopping) and it would be nice if pscp had the force option as well.

Is this something you could add? Thanks for considering.

ps. it would also be nice if pscp had the -i inline output to see exact result messages per server

Remove hashlib dependency for Python 2.4 compatibility

In psshlib.psshutil

Move more of the console printing to the UI submodule

Return status of Task not equal to 0 can also indicate a successful result.

Hi,

We use parallel-ssh in our company to check several Linux machines for updates in parallel.
It's a great tool, because we can start i.e. apt-get update or yum check-updates to check for and also
apt-get upgrade to apply updates in parallel. The uptodate status is even faster and available at a central place (i.e. a database) if there are a lot of machines. (We plan to make this project public open sourced available, but at the moment there are a lot of hardcoded ssh- user and machine names in some scripts, that prevents this. Look at the prepared github repo openlsus )

many prefaces here's the issue

As mentioned in the subject, against the regular use of the return status of a command,
Red Hat's "yum check-updates" returns 100 if there are updates available. (also documented in the man page)

We want to get the Status [SUCCESS] even for these unusual return status.
How can we get this (upstreamed) ? Should I create a patch or a pull request? Or can I have this only in a separate fork? - I want want to give the TASK an additional optional param which is the expected return status of the command. Or do you want to implement this by yourself.

Regards Marc

Add option groups to option parsers

Add option groups to option parsers to make help output more readable

Move -T/--test to --test-pool-size

This is in preparation for a --test-pool-regexp option that will take a regular expression as argument and only run against those hosts whose hostnames match (by default, the reg will be something permissive, like '.+' )

Add a --test-pool-regexp option

Takes a regular expression as argument and only runs against those hosts whose hostnames match. If combined with -T/--test (soon to be something different, maybe -T/--test-pool-size) will only run against a certain number of hosts before pausing to ask if it's OK to continue based upon the "test results"