Welcome to r5backup, a backup script that use rsync to transfer the files. As long as a file was not modified, a hardlink to its prior location is made.

This software is licensed under the General Public License GNU
 
r5backup comes with 5 scripts:

r5backup    the main backup application
r5scheduler    a script that control a complete backup run over several clients
r5trigger    a simple script that trigger a backup by a remote client (obsolet)
r5launcher    a script that trigger a backup to the r5backup server
r5recover    a script to recover files to the originally host
r5admin        a script to create new configurations, monitor progress etc

Requirements

• ssh (server and client)
• rsync (server and client)
• netcat or nc (if you want to use r5trigger)
• awk and nawk for Solaris or gawk for Linux
• mailx for Solaris or mail for Linux
• updatedb and locate (if you want to boost recovery with an index)
• bc (a command line calculator)

and of cause a huge partition as a target directory for your
backups. Whatch this movie to see how to create a ext2 partition for r5backup

Installation

Please note, that all installation steps must be done as root. If you received a sfx script just execute it. It will detect your operation system and environment and install/configure all parts.

After the installation you will find binaries in /usr/local/bin/ (Linux) or /opt/r5backup/bin (Solaris). An library with some functions is in /usr/local/lib/r5backup.inc (or /opt/r5baclup/lib). Configuration files will be unter /etc/r5backup (or /opt/r5backup/etc) The scheduler r5scheduler is under /etc/init.d/r5scheduler and there are symlinks in the runlevel directories.

If you want to install r5backup by yourself extract the tgz archive and copy the files to the favored directories. E.g. the scripts to /usr/local/bin/ and the configuration files
to /etc/r5backup. Create the directories /var/log/r5backup and /var/spool/r5backup.

Then create a pair of ssh keys by typing (you may use a different name
for you key pair if you want)

# ssh-keygen -N "" -C "key for r5backup" -f /etc/r5backup/ssh-keys/r5backup-key

Now you will find several files in /etc/r5backup; e.g.

• /etc/r5backup/profiles/example.cfg           -> example configuration file
• /etc/r5backup/profiles/example.exclude       -> example exclude list
• /etc/r5backup/profiles/example.sh            -> example script that is executed remote
• /etc/r5backup/ssh-keys/r5backup-key          -> ssh secret key
• /etc/r5backup/ssh-keys/r5backup-key.pub      -> ssh public key
• /etc/r5backup/r5backup.cfg                   -> main configuaration file

In r5backup.cfg are common settings. These seetings can be overwritten in a client configuration file.

+-------------- r5backup.cfg ------------------------------------
|# Defaults for r5backup
|
|# should rsync use gzip
|ZIP=yes
|#ZIP=no
|
|# directory with configuration files (path to this file)
|DIR_CFG=/etc/r5backup
|
|# directory with r5backup functions
|R5LIB=/usr/local/lib/r5backup.inc
|
|# how to check if client is online
|CHECK=ping
|#CHECK=ping6
|#CHECK=ssh
|
|# verbose mode
|#VERBOSE=yes
|VERBOSE=no
|
|# verbose level for log messages
|LOGGER=verbose
|
|# additional ssh opts
|SSH_OPTS="-o StrictHostKeyChecking=no -o ConnectTimeout=60"
|
|# ssh port to use
|PORT=22
|
|# username for remote login. Use 'root' for most cases, 'administrator' for Windows/cygwin
|SSHUSER=root
|
|# rsh/dsa key for passwordless login
|HOSTKEY="/etc/r5backup/ssh-keys/r5backup-key"
|
|# the directory where backups must go to (unix fs only)
|TARGET=/space/r5backup
|
|# the file with the PID
|PIDFILE=/var/run/r5backup.pid
|
|# the file with the pid for the sleep process
|PID_SLEEP=/var/run/r5sleep.pid
|
|# white space seperated list of return codes for rsync to ignore
|RC_IGNORES="6 13 23 24"
|
|# create and use an index? (you need locatedb/updatedb)
|INDEX=yes
|
|# the directory for logs
|LOGDIR=/var/log/r5backup
|
|SSH="/usr/bin/ssh"
|
|# directory for tempoary files
|TEMPDIR=/tmp
|
|
|# Defaults for r5scheduler
|
|# hold that number of backups
|HOLD=35
|
|# roll back failed backups
|ROLLBACK=yes
|
|# email address for reports
|EMAIL="name@domain.tld"
|
|# the port where netcat wait for the trigger
|TRIGGERPORT=100666
|
|# sleep how lang to the next period (in seconds)
|DAY_SLEEP=83999
|
|# check filesystem before running backups
|FSCK=yes
|
|# the file system
|FSTYPE=ext2
|
|# some settings for the backup server
|OS=Linux
|
|# important binaries
|R5BINARIES=/usr/local/bin
|R5SSH=/usr/bin/ssh
|R5RSYNC=/usr/bin/rsync
|R5BC=/usr/bin/bc
|R5NC=/usr/bin/nc
|R5UPDATEDB=/usr/bin/updatedb
|R5LOCATE=/usr/bin/locate
|R5AWK=/usr/bin/gawk
|R5MAIL=/usr/bin/mail
|ECHO=echo
|
+----------------------------------------------------------------

With this master configuration file and the ssh key you can start a backup by giving r5backup some parameter. To backup the directories /etc and /opt from
newsserver.int.example.org to /var/tmp type

# r5backup.sh -a newsserver.int.example.org -t /var/tmp -d "/etc /opt"

You can give more or less parameter, but at least you have to define the address (-a) and the directories (-d). If you give additional parameter the values in /etc/r5backup/r5backup.cfg are overwritten.  Otherwise the values from /etc/r5backup/r5backup.cfg are used.

But the goal for r5backup is not to give these parameter every time you start a backup. If you want to make periodic backups you should create a profile for every client.

Preparing a new profile

Let's assumed you want to backup a Unix machine called "newsserver" r5admin have a build in mechanism to create a new profile. Type
# r5admin --setup=newsserver
to configure the new backup

# creating a new profile named "newsserver"
# using /etc/r5backup/r5backup.cfg for defaults
# creating /etc/r5backup/profiles/newsserver.cfg
# hostname or IP address: newsserver.example.org
# setting address to newsserver.example.org
#
# directories to backup: "/etc /usr/local"
# setting directories to "/etc /usr/local"
#
# use rsync verbose mode? Type yes or no or just press enter for the default value (no):
# using default value from /etc/r5backup/profiles/newsserver.cfg
#
# methed to check if host is online. Type ssh or ping or ping6 or no. Just press enter for the default value (ssh): ping6
# setting check method to ping6
#
# zip files while transfering them? Type yes or no. Just press enter for the default value (yes):
# using default value from /etc/r5backup/profiles/newsserver.cfg
#
# use verbose logging? Type yes or no. Just press enter for the default value (verbose):
# using default value from /etc/r5backup/profiles/newsserver.cfg
#
# generate protocol emails? Type an valid e-mail address or no. Just press enter for the default value (renner@vbox4php.org):
# using default value from /etc/r5backup/profiles/newsserver.cfg
#
# should /etc/r5backup/ssh-keys/id_dsa_cassiopeia be used for passwordless login? Type yes or just enter to create a new key: yes
# using default value from /etc/r5backup/profiles/newsserver.cfg

You can, if you want, use the example.cfg file as a template for an own configuration if you don't like the setup. Therefor, just copy example.cfg to newsserver.cfg. and edit it

• SCHEDULE=yes                                                -> only relevant if you use the scheduler
• VERBOSE=yes                                                 -> use verbose rsync logging
• LOGGER=verbose                                              -> r5backup log level
• ZIP=yes                                                     -> use gzip to compress the network traffic
• ADDRESS=newsserver.example.net                              -> hostname or FQDN
• DIRS="/home /root /etc /opt"                                -> directories to backup. No trailing slashes
• R5_PRERUN_COMMAND="/etc/init.d/mysql stop"                  -> command to start at the client before the backup starts
• R5_POSTRUN_COAMMAND="/etc/init.d/mysql start"               -> command to start at the client when the backup is done
• R5_PRERUN_SCRIPT=/etc/r5backup/profiles/example_prerun.sh   -> this script will be executed remote
• R5_POSTRUN_SCRIPT=/etc/r5backup/profiles/example_postrun.sh -> this script will be executed remote
• HOLD=90                                                     -> number of backups to hold 
• EMAIL=name@domain.tld                                       -> the email adress for backup reports
• ROLLBACK=no                                                 -> delete files from a failed backup

You can overwrite every setting mentioned in r5backup.cfg. So if ssh at your client machine listens to port 31522, just add the line
PORT=31522
to your configuration file newsserver.cfg Before a r5backup start to backup a file a check will be performed if the client is reachable. You can choose between a simple
test with just a ping (or ping6 if you use IPv6), or, by configuring ssh as the check method, trying to login with ssh.


If you don't use the r5admin step you have to afford a passwordless login at the client. Therefor copy the public key /etc/r5backup/ssh-keys/r5backup-key.pub to the clients
authorized_keys. Assumed it is a linux or a *BSD machine type
# cat etc/r5backup/r5backup.pub | ssh client "cat >> /root/.ssh/authorized_keys"
if root does not have a home directory (like at solaris) use this command
# cat etc/r5backup/r5backup.pub | ssh client "cat >> /.ssh/authorized_keys"
if you are unsure try the command
# cat etc/r5backup/r5backup.pub | ssh client "cat >> ~/.ssh/authorized_keys"

Now r5backup is ready to backup your client. A backup is startet by typing
# r5backup --profile=newsserver
The first backup will take a while, but the netxt backups will be much faster because the most files does not change. If you have configured verbose logging and the email report you will receive a summary report, e.g.

# will hold 80 old backups
# check for directories to delete...
# found 0 backup(s) to delete
# creating new index /space/r5backup/newsserver/r5index.db
# starting Sun May  2 17:27:22 CEST 2010
# ending Sun May  2 17:29:23 CEST 2010
#
# summary
# got 446 new file(s)
# got 36 changed file(s)
# got 3 new directorie(s)
# got 7 changed directorie(s)
# backup took 436 s
# backup done, exitting with return code 0
# mailing a report to name@example.org

pre and post commands

You can define commands or scripts to be executed before and after a backup at the client. The scripts are executed in the context of the configuration. This means you have all these variables from the r5backup.cfg and from the profiles configuration. Please check the file generic.example_postrun.sh to how to use this

virtual hosts

if you want to backup a you will into trouble: ssh will see diferent host keys for the same target machine. To avoid this, disable strict key checking!
SSH_OPTS="-o StrictHostKeyChecking=no"

directoies with white spaces

To backup a directory with white space please replace the white spaces with question marks. So if your drive C:\ is mounted under /mnt/C, but you only want to save the "Dokumente und Einstellungen" directory just set the DIRS variable in the cfg-file to something like DIRS="/mnt/C/Dokumente?und?Einstellungen /home /usr/src /usr/local /root /etc" Never mind about directories with white spaces below the directories configured in this variable. The question marks are only required in this variable.

the backup process

For every client a directory below the directory that is set with the variable TARGET is made; e.g. /space/r5backup/newsserver/ The current backup in this directory have the name "current", older backup have names accordant to the timestamp when it was made. When you start to backup a new client please check the log messages exhaustive!
If a backup fails (even if the backup for one of the configured directories fail) the whole backup is rolled back. Check the logfile for the reason.

clients with several IP addresses

Lets assume a client travel around the world. Some time it is reachable with its internal address, sometime it have a different, a VPN-address and sometime it have a IPv6 address. You can configure r5backup to try all these addresses (or multipe hostnames) by configuring more then one value to the varible ADDRESS and CHECK, e.g.

ADDRESS="192.0.2.192|example.dyndns.org|example.ipv6.example.org"
CHECK="ping|ssh|ping6"

r5scheduler

The r5scheduler is startet automaticly when ever the r5backup server comes up. It checks for clients that should be backed up at regular intervals
configured by a statement in the peers configuration:

SCHEDULE=yes

When all backups are done an email report is generated. Then the process sleeps for a while (24h) and starts again with the backups. Before it the file system for the r5backup partition is checked (fsck). While r5scheduler is sleeping it waits for connections from clients which use the r5launcher script to start the r5trigger mechanism: Let's assume you have a computer that only runs from time to time (e.g. a notebook). The chance to 'hit' the backup periode with the uptime is less. But you can trigger a backup for a single host by sending the client hostname to r5scheduler. This can be done by using the script r5launcher from the client side. You just have to configure some variables and create
some symlinks in /etc/init.d/r5launcher

R5SERVER=r5backupserver
TRIGGERPORT=100666
BOOT_SLEEP=600

In this example we assume that the notebooks hostname is identical to the profile name. At least, 100666 ist the configured port, where r5scheduler wait for a trigger.
 
r5recover
Call this script if you have to recover files. The syntax is simple:
# r5recover --profile=<profile> -f <file>
# r5recover --profile=<profile> --dir=<directory>
To be earnest: there are more options, try
# r5recover --help
to find out these. You are free to give just the filename or the filename with its full path. You can also combine full path with wildcards, but this have some pitfalls. The usage of locatedb will boost r5recover.

monitor modus

If you want to monitor what is going on you will find the monitor function in r5admin usefull. It report what the scheduler and/or r5backup just doing. Ig the partition is mounted rw or rw, how small or huge the last backup was and so on ...

+--------------------------------------------------------------------------------+
| r5backup activity monitor                      Sun May  9 21:33:58 CEST 2010   |
+--------------------------------------------------------------------------------+
| Network trigger status:                                                        |
| status: listening            since 09.05.2010 21:22                            |
| listener PID:                5652                                  |
| listener Port:               100666                                |
+--------------------------------------------------------------------------------+
| r5scheduler status:                                                            |
| /space/r5backup mounted_rw   since 09.05.2010 21:33                            |
| ongoing: backuping           processing mars                            |
+--------------------------------------------------------------------------------+
| r5backup status:                                                               |
| status: running              since 09.05.2010 21:33                            |
| remote directory: /home                                                        |
| remote host: mars                                                              |
| former backup size: 10940116 (KB)                                                    |
+--------------------------------------------------------------------------------+

+------------------------------------------------------------------------------------------------+
|\                        |  C  ||   L   |   S   |   I   |   W   |   B   |   M   |   M   |   A   |
|   \                     |  o  ||   i   |   o   |   r   |   i   |   S   |   a   |   i   |   i   |
|      \    Peer-OS       |  m  ||   n   |   l   |   i   |   n   |   D   |   c   |   n   |   x   |
|         \               |  m  ||   u   |   a   |   x   |   d   |       |   O   |   i   |       |
|            \            |  o  ||   x   |   r   |       |   o   |       |   S   |   x   |       |
|  r5backup-OS  \         |  e  ||       |   i   |       |   w   |       |       |       |       |
|                  \      |  n  ||       |   s   |       |   s   |       |   X   |       |       |
|                     \   |  t  ||       |       |       |       |       |       |       |       |
|-------------------------+-----++-------+-------+-------+-------+-------+-------+-------+-------|
| Comment                 |     ||       |       |  1)   |  2)   |       |       |  3)   |       |
|-------------------------+-----++-------+-------+-------+-------+-------+-------+-------+-------|
|-------------------------+-----++-------+-------+-------+-------+-------+-------+-------+-------|
|              ext2       |  b  ||   *   |   *   |   *   |   *   |   +   |   ?   |   -   |   +   |
|              -----------+-----++-------+-------+-------+-------+-------+-------+-------+-------|
|              ext3       |  d  ||       |       |       |       |       |       |       |       |
| Linux        -----------+-----++-------+-------+-------+-------+-------+-------+-------+-------|
|              ext4       |  d  ||       |       |       |       |       |       |       |       |
|              -----------+-----++-------+-------+-------+-------+-------+-------+-------+-------|
|              reiserFS   |  d  ||       |       |       |       |       |       |       |       |
|-------------------------+-----++-------+-------+-------+-------+-------+-------+-------|-------|
|              zfs        |  b  ||   *   |   *   |   *   |   x   |   +   |   ?   |   -   |   +   |
| Solaris      -----------+-----++-------+-------+-------+-------+-------+-------+-------+-------|
|              ufs        |  d  ||       |       |       |       |       |       |       |       |
|-------------------------+-----++-------+-------+-------+-------+-------+-------+-------+-------|
| BSD          ufs        |  d  ||       |       |       |       |       |       |       |       |
+------------------------------------------------------------------------------------------------+

Legend for server OS:
b = build in
d = difficult program increments for fsck, maybe for r5recover as well


Legend for matix:
* = working
+ = should work more or less out of the box
? = may work, I'am not familiar with it
- = does not work (yet)


Legend for peer OS:
1)
tested with irix 6.5 and rsync 2.6.5 in /usr/local/bin/. Download rsync for irix from ftp://ftp.ens-cachan.fr/mirror/RSync/binaries/IRIX/rsync-2.5.6.IRIX62.tar.gz if you have problems with passwordless login create the key under irix and copy the secret key to your r5backup server, the private key to /root/.ssh2/authorized_keys2

2)
install cygwin. See README.cygwin for instructions how to start sshd as a service

3)
does not work for unknown reason. ssh seems not to come back, even if the '-t' switch
is given.

hints

If you use the locatedb (the command locate) at your backup server make shure that the partition with the r5backup will be not includet to the locatedb. Because it would take hours. And: it would make the locatedb very large and slow! You should edit /etc/updatedb.conf and add your backup directory (e.g. /space/r5backup) to the PRUNEPATHS variable. It make more sence to configure r5backup to create an own locatedb for every client in its profile configuration.