Securing automated rsync over SSH

Quoting the RSYNC homepage : “rsync is an open source utility that provides fast incremental file transfer.”

To make rsync both secure and automated (i.e : non-interactive), you can use SSH as the transport and set up a key pair. This is what will be discussed in this post, along with a few improvements.

Continue reading Securing automated rsync over SSH

Unix : shell tips

I ran into this into the following article, “Learn 10 good UNIX usage habits“. This article is mainly common sense, but there are interesting points, such as :

  • avoid piping when you can, in order to save performance (the classical construct grep | wc to count the lines is useless as most versions of grep can count with grep -c)
  • use awk to “grep” on a specific field of a line with “… | awk ‘$1 == “XXX”‘ which is cool and I never use
  • the find | xargs construct (I’d add “find -print0 | xargs -0″, useful if your find brings back filenames with a space inside …)

All in all it is worth a reading, if only to refresh your memory.

dsh : a distributed shell

clusterA common problem when you deal with a pool of servers (clusters or server farms, you name it) is to execute the same command line on each server. It is usual to solve this with a “for” construct such as :

for i in server1 server2 ; do ssh $i "uname -a"; done

But this is basically re-inventing the wheel everytime. Here comes Distributed Shell (DSH).
Continue reading dsh : a distributed shell