Monday, May 28, 2012

Quick and dirty parallel ssh

Occasionally,  you may want to scp or run a job across many hosts.. Well there's parallel-[scp|ssh] for you.

However, if you are constrained or are out of time to install this wonderful tool, xargs might just save your day. So here goes my simple one/two liner. Say we want to scp /etc/passwd to a local dir, then we'll need to:
  • Give a unique name to the file to be copied over.
$ mkdir /tmp/bah/results -p && cd /tmp/bah
$ cat hosts | xargs -n 1 -P 50 -IH  ssh -i ~/.ssh/key.pem user@H 'cp /etc/passwd /tmp/passwd.`hostname` && echo `hostname` ok'
a.b.c.d ok
.....

  • Copy it over:)
 $ cat hosts | xargs -n 1 -P 50 -IH  scp  -i ~/.ssh/key.pem  user@H:/tmp/passwd.* results/
passwd.a.b.c.d                                                                    100% 1155     1.1KB/s   00:01
....
Total time is 9 seconds for 60 hosts... Not bad..
real    0m9.654s
user    0m2.120s
sys    0m0.276s
sh-4.2$ ls results/|wc
     60      60    1486

1 comment:

  1. Nice! In this case, 'hosts' in 'cat hosts' should be a file with hosts/IPs separated by newlines

    ReplyDelete