Backing Up via Rsync

Synopsis

The rsync remote-update protocol allows rsync to transfer just the differences between two sets of files across the network link, using an efficient checksum-search algorithm.

Some of the additional features of rsync are:

• support for copying links, devices, owners, groups and permissions
• exclude and exclude-from options similar to GNU tar
• a CVS exclude mode for ignoring the same files that CVS would ignore
• can use any transparent remote shell, including rsh or ssh
• does not require root privileges
• pipelining of file transfers to minimize latency costs
• support for anonymous or authenticated rsync servers (ideal for mirroring)

Background & Purpose

Using rsync as a means to make full backups of the Pre, both for disaster recovery and convenience. I have been using Rsync as a means to backup all my Gentoo and FreeBSD servers, and even a 4TB volume on our NetApp. I was able to restore every system simply by using rsync to push the files to a new disk/array. Using a special string of commands, rsync backups are nearly as depth-oriented as 'dd', while still being much more convenient.

Requirements

• 1 Unix-based computer (with access to the same WIFI or usbnet subnet as the Pre)
• On your Palm Pre, you need to have access to linux and have completed the next steps after accessing linux.

The Unix-based computer will be consistently referred to as the "host".

The rsync binary is part of Palm's original package, so it is possible to make a wrap-around gui-based ipk for users who do not want access to linux.

Backup Procedure

Notice

The backup method has been tested, but there are no guarantees explicitly or implicitly given for data replication and retention. These tools have worked exceptionally well for me, but they might not work for you.

Out of respect for Sprint, I will only document how to use rsync over WIFI or usbnet. I strongly disadvise against using rsync over EVDO (even through SSH). This process is bandwidth intensive and we don't want Sprint complaining to Palm.

Quick Overview

1. We will set up the rsync daemon on the Palm Pre and make it acccessible by the host.

2. The host will be set up to download the Pre's files.

3. The transfer will take place.

4. The daemon on the Pre will be killed to prevent anyone else from gaining access.

Setting up the Pre

1. Access your Pre via WIFI or usbnet.

2. Edit /etc/rsyncd.conf and make sure you fill in the host IP address (your computer)

pid file = /var/run/rsyncd.pid
uid = root
gid = root
hosts allow = FILL_YOUR_HOST_IP_HERE
hosts deny = *
use chroot = no

[root]
 path = /

3. Start the rsync daemon and allow the rsync daemon to communicate with the host. Make sure you fill in the host IP address (your computer): <source lang=bash>

export IPADDRESS=enter the IP Address of your host
rsync --daemon && /usr/sbin/iptables -A INPUT -p TCP --dport 873 -s $IPADDRESS

</source>

Setting up the host

1. Make sure you are the root user. <source lang=bash>

sudo -i

</source>

or

<source lang=bash>

su -l

</source>

2. On the host, you will need to create a directory where the Palm Pre will back up the files. My example will be /media/pre-backup <source lang=bash>

mkdir /media/pre-backup

</source>

Running the transfer

On the host

1. If you want to exclude any directories you are already backing up (or simply don't want to include), add --exclude=/path/to/your/dir after "-stats" in the command (in Step 4). You will need to do this for every directory you wish to NOT include in the transfer. Until the restore process is verified, I'm including the special filesystems /dev, /proc and /sys in the backup.

2. Fill your IP Address in the command below, along with your excludes (if you have any), and begin the transfer from the host computer. Make sure to set the correct backup directory (if you deviated from /media/pre-backup), otherwise you could overwrite data on your host machine. <source lang=bash>

export IPADDRESS=enter the IP Address of your pre
rsync -HrlptgoDPvvS --force --delete --del --stats rsync://root@$IPADDRESS/root/ /media/pre-backup/

</source>

2a. Every time you run the transfer, it will retain the exact same file-structure of the Pre, only deleting files that don't exist and only sending files that have changed.

Disabling the rsync daemon on the Pre

1. You need to disable the rsync daemon on the Pre to prevent anyone from gaining access to your files. To kill the rsync daemon, you will need to find the Process ID first, then kill the process. <source lang=bash>

ps auxw | grep rsync 
root     11567  0.0  0.0  10480   624 ?        Ss   09:18   0:00 rsync --daemon
root     11576  0.0  0.0   6124   640 pts/1    S+   09:18   0:00 grep rsync (IGNORE THIS ONE)

</source>

Then kill the process: <source lang=bash>

kill 11567

</source>

Restore Procedure

Quick Overview

1. Set up the host's rsync daemon

2. Gain access to the Pre's Linux.

3. Transfer the backed-up files to the pre and pray it works.

Setting up the host

1. Edit /etc/rsyncd.conf (or /usr/local/etc/rsyncd.conf for FreeBSD). Check your documentation. Be sure to fill in the Pre's IP Address, and set the correct directory for the Pre backup (my example will be /media/pre-backup).

pid file = /var/run/rsyncd.pid
uid = root
gid = root
hosts allow = FILL_YOUR_PALM_PRE_IP_HERE
hosts deny = *
use chroot = no

[root]
 path = /

2. Start the rsync daemon (you can use an init script, I'm using the command below just for simplicity. Check your documentation) <source lang=bash>

rsync --daemon

</source>

Setting up the Pre and restoring

A. For the scope of this guide, the requirements above still apply for the Palm Pre: you need to have access to linux.

1. Access the Pre via WIFI or usbnet.

2. Turn off the GUI while we restore all the files. <source lang=bash>

/sbin/initctl stop LunaSysMgr

</source>

3. Fill your IP Address in the command below, and start the transfer. This command will take everything from your Host system and overwrite EVERYTHING on the Pre.

export IPADDRESS=enter the IP Address of your host
cd /
mount -o remount,rw /
mount -o remount,rw /boot
mount -o remount,rw /media/internal
rsync -HrlptgoDPvvS --force --delete --del --stats rsync://root@$IPADDRESS/pre-backup/ /

3a. If you have any "Out of Space" issues, try removing the --delete --del and replacing it with --delete-before

4. Reboot immediately and pray it works. <source lang=bash>

reboot

</source>

Development/Improvement Ideas

• I need someone to test the restore process and get back to me with suggestions.
• It would be awesome if the Palm Pre supported NFS, otherwise we're stuck to setting up /etc/rsyncd.conf on the Pre (for backup) and the Host (for restore).
• A gui wrapper is possible because the Palm Pre comes shipped with rsync. Also, there needs to be an easy Windows server program to make this mainstream, only thing to use at the moment that's Windows based is cygwin. Otherwise we're stuck with Unix.
• It's possible to create a cron job to run the backup every evening over WIFI (especially if you dock you're phone overnight). The cron job would be run from the Host computer. It's also recommended to set up iptables for allowing port 873 only for the host on the interface eth0 or usb0.

Contributor(s)

• hopspitfire