Backups using rdiff-backup

rdiff-backup is a remote incremental backup tool which is both simple and very complete. It can be used in many situations (mirroring, local or remote backups using push or pull methods…).

For a full description, you can visit rdiff-backup official homepage and read the documentation for an overview of all the features.

I use this tool to make all my backups on a central server. The ever-running servers are backed up using the pull method, whereas the workstations, which contain the higher quantities of data, are backed up using the push method1.

rdiff-backup always uses the following syntax:

rdiff-backup [options] <source> <target>

Where both <source> and <target> can be local or remote (in the latter case, the syntax is [user@]host::/remote-dir).

Requirements

For this article, we assume that:

The aim of this document is to backup given files and directories, with the possibility to exclude either given files/directories or files/directories that match a defined pattern. Files and directories to include or exclude from the backup are listed in separate files.

Include and exclude files

In the following examples, we will use 2 files: backup.include, which lists the files and directories to be backed up, and backup.exclude which lists the files and directories to be excluded from the backup.
Both have the same very simple syntax: one file or directory per line.
If you specify a directory, all the files and subdirectories will be backed up (remember that rdiff-backup can be used as a mirror tool).
Thanks to the --{ex,in}clude-globbing-filelist options, we can even use file globbing.

Typically, you surely want to backup personal data (usually stored in /home), your configuration files (in /etc), as well as your local mail file.
Because you lack of space, you don’t want to back up your ogg or mp3 encoded audio files all stored in your ~/Music directory; you also want to exclude from the backup the several iso images that can be anywhere in your home (let’s say you can download them again later if a disaster happens!); finally, you decide to exclude from the backup a qemu hd image that you don’t use (are you one of these persons who like keeping old things just because “they can be useful”?). Your files would be as follows:

backup.include:

/home/toto/
/etc/
/var/mail/toto

backup.exclude:

/home/toto/Music
**.iso
/home/toto/qemu/qemu-hd.img

For detailed syntax of file globbing, please read carefully the man page, section “File selection”.

Local backup

For this first example, we will backup data locally, let’s say from one partition to another, or, better, from one hard disk to another. The following script will be called directly by root, and the data will be saved to a partition mounted as /mnt/backups.

local-backup.sh:

rdiff-backup --print-statistics --terminal-verbosity 2 \
    --exclude-globbing-filelist /path/to/backup.exclude \
    --include-globbing-filelist /path/to/backup.include \
    --exclude / / /mnt/backups/

Note that the options order is important: if you don’t indicate the exclude list before the include list, the script won’t work as desired (see documentation for explanations).

Remote backup using push method

In this example, we will backup the data to a remote host called backserv. As already specified, you need to have an ssh access to this host (ideally using a public key without passphrase – see the “Unattended rdiff-backup HOWTO” in external resources).

remote-push-backup.sh:

rdiff-backup --print-statistics --terminal-verbosity 2 \
    --exclude-globbing-filelist /path/to/backup.exclude \
    --include-globbing-filelist /path/to/backup.include \
    --exclude / / user@backserv::/var/backups/

Remote backup using pull method

Here, the method is different, but the result is quite the same. The idea is to pull the data from a remote host to a local directory. This method allows to set up easily a central backup server (no special configuration has to be made on the client side).

remote-pull-backup.sh:

rdiff-backup --print-statistics --terminal-verbosity 2 \
     --exclude-globbing-filelist /path/to/backup.exclude \
     --include-globbing-filelist /path/to/backup.include \
     --exclude / user@desk1::/ /var/backups/

Restoring

The restoring process is very well explained in the official documentation: http://www.nongnu.org/rdiff-backup/examples.html#restore

Just an example to restore one directory from the version backed up 10 days ago in a given directory:

rdiff-backup -r 10D user@backserv::/remote/ ./local/

Remember it is very important to test your backups!

Deleting older files

rdiff-backup’s --remove-older-than mode can be used to delete older increments.

This section assumes that rdiff-backup has been used in the past to back up to host::/remote-dir, but all commands would work locally too, if the hostname were ommitted.

The following command deletes all information concerning file versions which have not been current for 2 weeks:

rdiff-backup --remove-older-than 2W host::/remote-dir

Note that an existing file which hasn’t changed for a year will still be preserved. But a file which was deleted 15 days ago cannot be restored after this command is run.

You should run such a command on a regular basis so as to preserve your disk space.

External resources

  1. You can push and pull files for backups: i.e. have the backup server login to a client machine via ssh and pull files; or the client can push the backups to the server ↩