21.02.2009 04:26

Personal backup solution

I have at least 4 machines at home that are operational at any given time. Personal workstations, a firewall/router and a VDR/file server box. The latter has the most disk space, holds nothing of importance in it self and is perfect for storing backups. Some years ago I deleted my ~/code directory. Nothing that important, but it contained everything I wrote in the last 2 years. All that time became worthless in a second. Eventually I reconstructed most of it from scraps I had on other machines, but it was clear I need a solid backup strategy for more then just a few things I considered important by that point.

First file in my new ~/code directory became pybackup.py, a simple script that would tar some files, delete old snapshots and so on. As years passed I needed to backup more and more data, and pybackup.py didn't cut it any more. I was well aware of Amanda, Bacula, rsnapshot... and the rest. But I wanted something extremely simple yet more versatile then the previous solution.

I remembered reading a nice article by Mike Rubel about implementing rotating backup snapshots with rsync. Many people already took a crack at it, and that page lists many of those implementations, but after checking all of them not one was just right for me. Some were close though and gave me a few pointers for my own implementation. rybackup runs from a cronjob a few times a day and backups all my machines to the file server over NFS. The beauty of it is that I have multiple levels of backups that are as big(small) as just one full snapshot + changes between them. It's achieved using hard-links, while rsync automatically un-links files that have changed. This solution also has other advantages; rsync transfers only changes between files, offers compression, encryption and so on. Finally let's see a directory tree after a few months of rotating snapshots:

2008-03-14 16:04 daily.0/
2008-03-14 12:03 daily.1/
2008-03-13 08:05 daily.2/
2008-03-17 00:04 hourly.0/
2008-03-16 20:04 hourly.1/
2008-03-16 00:04 hourly.2/
2008-03-15 04:03 hourly.3/
2008-02-10 16:04 monthly.0/
2008-01-15 16:02 monthly.1/
2008-03-11 00:04 weekly.0/
2008-03-04 00:04 weekly.1/

$ du -hs ; du -hs hourly.0
1.5G    .
778M    hourly.0

Written by anrxc | Permalink | Filed under code