Speech:Rsync

rsync automated backups

This page will be dedicated to the explanation of the tools group rsync backup implementation. This is still a work in progress and will be updated when implemented.

--- rsync readme --- (unfinished) rsync backup setup readme Tom Rubino 4-21-16

Backup system using rsync and hard links

Working dir - reccomend new partition/drive/system for backups in future /mnt/main/rsyncBackups

The working dir is where backups currently reside.

Currently backed up dirs include:


 * /etc


 * /usr/lib


 * /mnt/main/bin


 * /mnt/main/local


 * /mnt/main/scripts

More system and user dirs will be added in the future.

Schedule Currenly manually run. I will update this section when schedule is determined and implemented.

Folder structure backup.0 = latest backup backup.1 = one cycle earlier than backup.0 backup.2 = two cycles earlier than backup.0 etc....

Backup folder contents This backup system takes advantage of hard linking in linux systems. Each backup folder will appear to be a full copy.

The initial backup or oldest backup will include real copies of files. All newer folders are created using hard links. These are a special type of file link that points the the same inode location and therefore the same file.

To understand how this works it is important to understand how rm (remove) file works in linux. When you rm a file Example: rm test.txt The text.txt file is not actually removed, but instead the pointer to the file is removed. This means the inode or physical location of the file remains the same, but the system thinks this space is available for use.

With hard linking you can create more than one link to a file or inode. So if your original file is text.txt and you create a hard link to the same file say text2.txt (yes you can make the name different) the file is not actually copied, but instead a second link to the same file is created using much less space (only the metadata needed to setup pointer).

As I mentioned previously if you delete the first file you are not actually deleting it, just updated the number of pointers to the file. In this case we have 2 pointers or links. So your count would change from 2 to 1 meaning the file still exists on the system.

Running a new backup. - script not yet written 2-21-16

What happens when this script runs? When a new backup is started the existing folders are first moved or renamed.

Starting with the oldest backup the folder is renamed mv backup.x to backup.x+1

example might be mv backup.17 backup.18

Each folder is renamed until backup.0 becomes backup.1 and there is no backup.0.

Backup.0 is then created by hard linking to everything in backup.1 Example: cp -al Backup.1 Backup.0

We now have Backup.0 in the latest state.

A fresh set of rsyncs are run against this dir. One for each dir being backed up. These rsyncs are set to cleanout anything not currently in the source dir.

Example: rsync -avn --delete /etc /mnt/main/rsyncBackups/Backup.0/etc

This will update Backup.0/etc to look exactly like /etc

'''This is personal log entry and an incomplete file, so ignore it... For most up to date instructions check README.txt file in rsyncBackups dir or dedicated wiki page (not yet created)'''