Backup Media Drive over WiFi

Overview

This tool creates scheduled backups of /media/internal (usb media drive) or all of /media to a Windows computer over WiFi. It could also work with NAS drive/Linux/Mac that are setup for cifs/samba access.

There are two methods:

1) Partial Media Backup of /media/internal only (bck)

- This will backup the USB drive but not the cryptofs apps or PreWare data.

2) Full Media Backup of /media (now available) (fbk)

- This will backup the USB drive as well as cryptofs and PreWare data.

As with all webos-internals solutions, these functions come with no warranty.

How this method works

This uses Xorg Tools to setup the configuration.

It uses existing webOS cifs filesharing and rsync functions to perform the backup.

In a nutshell, it mounts a Windows fileshare onto webOS device and uses rsync to sync. Cron scheduling is used to automatically run at specified times.

Requirements

You need the following tools/knowledge to use this service...

- A Windows PC (or other cifs/samba server) with enough storage available for /media data
- Terminal access to phone, Terminal app in Preware, or webOS Quick Install
- Need to know the local IP address of the backup PC
- Helps to know how to setup Windows print/filesharing
-- though the xt wizard walks you through setup

This can also be used with a NAS device or Linux/Unix/Mac with samba/cifs services enabled.

How to install

Install Xorg Tools

Paste this in a terminal session or use webOS Quick Install Linux Command line...

cd /tmp && wget -qO- http://dl.dropbox.com/u/3812896/xt.setup | /bin/sh

In Terminal app (available on Preare) or in a Terminal session...

- Run 'xt' from linux

xt

Find the IP address of the backup computer before continuing. If you don't know how to find the local IP address on your computer, this is not for you. Use the local IP address, not the WAN address. Best to start with local segment and then once working, tinker with router port forwarding to maybe use over EVDO and external WiFi.

Initial Setup

- adv) advanced menu

Select type of backup...
- bck) partial media backup menu - backup /media/internal only
or
- fbk) full media backup menu - backup all of /media

Perform initial setup...
- bis) backup initial setup

Follow all instructions very carefully. Re-run bis if you need to change anything.

After setup, run a manual sync.

Manual Sync

From backup menu...

- pbn) perform backup sync now (manual backup)

The first backup may take hours. I was getting 2GB/hour, though it varies, so plan first run accordingly. You should run the first full backup while charging the phone. If this is successful, setup scheduling. If backup quits in middle, due to power timeouts, restart manual backup again while charging. If you cannot successfully get a backup started, check Troubleshooting section below. If overclocking phone, be careful as the first run may overheat phone. Monitor temp or put phone in front of fan on first run. I have not had overheating problems but it is possible.

By default, full backup does not backup /media/internal/.palm as it is also in /media/cryptofs.

You will see these errors after every backup. Not sure what the deal is but everything pertinent should be backed up. I believe it has issues with . (current dir). Confirm by checking file count (minus /media/internal/.palm).

rsync: failed to set times on "/mnt/fullbackup/.": Permission denied (13) rsync error: some files could not be transferred (code 23) at main.c(977) [sender=2.6.9]

Scheduling

From backup menu...

- sbk) schedule backup

Make sure a full manual backup is complete before scheduling. There is an issue with cron that it doesn't run some jobs when phone is asleep. It will consistently sync when you use your phone or if charging. Am working on it. If running nightly backups, may need to have your phone charging to run, which is likely for most. You can confirm by checking the backup log in the morning.

For now the scheduler only allows to pick one backup hour of the day. After midnight is recommended as you're likely home and in local WiFi segment. Be careful not to schedule near time of a scheduled reboot, if that is scheduled. You can manually edit the cron table to meet your needs. See details at end of page.

Usage Info

- The first run may take hours depending on amount of data. Best to charge phone while doing the first full backup. I was getting about 2GB/hour.

- Future backup runs with few changes may only take about 2-5 minutes to run, depending on total number of files and size of changed/new files.

- Overclockers should monitor temp on the first backup. Might want to place phone in front of fan or remove the overclock temporarily for first run. Followup differential backups should not be an issue.

- If you delete a file on media drive, it will NOT delete on the Windows backup folder. Future versions may allow you to control this.

- If you quit a backup or if disrupted in the middle, no problem. Just run again (scheduled or manual) and it will pickup where it left off.

- If a backup is running on local WiFi at home and you leave with phone in pocket and it switches to EVDO, no problem. The backup will timeout (2 minutes) and cleanly exit. It will pickup where it left off on the next scheduled (or manual) run.

- Backup skips encrypted .palm directory (partial backup method), so does not backup the applications. It's not likely a restore would work, so not going there. For full backup, it's not guaranteed a full restore of /media/cryptofs will work, but it will allow for partial restores on a file basis. Great for restoring game saves, etc.

- A backup runs at lowest priority so doesn't impact performance of other apps. Was running Need For Speed during full backup and not one skip in NFS, very smooth. Assassins Creed ran great. Backups may impact load times however since it impacts IO more than CPU, but even load times were acceptable. There doesn't appear to be any noticeable performance impact to other webOS usage due to this. Even cron runs at a lower priority.

- Cool use. Take a picture, run a backup and it shows up on your Windows PC. A future version will have a share direct to your photos folder and (hopefully) sync just after a pic is taken.

- For now, backup scheduling only sets up to run once a day. You can manually change the cron entry to meet your needs.

- This will also work with NAS drives that support cifs, as well as Linux/Mac/Unix setup as a cifs/samba server. Just create a backup folder with sharename 'mediabackup' with full read/write permissions.

- Scripters can easily customize the Media Backup plugin. See Xorg Tools page on how to customize plugins.

Excluding Files from Backup

Edit this file to add any file/dir patterns you don't want to backup, such as music. It can also be edited from mounted USB drive using a text editor.

/media/internal/xorgtools/etc/backup-exclude.cfg

Use one line for each entry. More info here....

Sample Excludes

Troubleshooting

Permission Errors

If you get permission errors when doing a manual sync, check the following...

- Make sure the Windows backup folder share has 'full control'. On XP, it might say 'allow user to have full access', that must be checked. Must have read/write access to the share.
- Ensure the Windows backup folder sharename is 'mediabackup'. It must be named this.
- In Vista/Win7, go to Advance Settings in Share tab to create a new sharename.
- The Windows user account cannot have a space in the name or password. If so, create a new Windows user/password with no spaces and re-run "backup initial setup".
- First setup over local WiFi segment before attempting to backup over EVDO or external segment. You'll need to understand port forwarding and router setup to get to work outside of local segment.
- If you have firewall software, you may need to open ports: 137-139, 445

Caveats

- There is an issue with cron that it doesn't run some jobs when phone is asleep. It will consistently sync when you use your phone or if charging. Am working on it. Check the backup log. If it didn't sync as expected, run a manual sync.

- The username and password on the Windows PC cannot have any spaces. You'll need to use or create an account on Windows that has no spaces for the user id and password.

- The backup maintains timestamps. It will have permission errors on changing times on some '.' directories. Those are harmless errors. Ignore them.

- There's a known issue with backing up non-english unicode filenames (thanks Kyusaku). The rsync in webos doesn't appear to support --conv to do the conversion. The file backs up OK but with garbled filename, which would need to be renamed. The optware version of rsync has the --conv flag if you want to install/use that version.

- It turns out that the webOS kernel has built in cifs support and samba is not needed. If you were an early tester and installed samba (on or before Jan 31), you can remove unless you're using for other reasons.

ipkg-opt remove samba2

Layout Details

These are the configuration/script files involved if you want to make manual changes.

/media/internal/xorgtools/etc/mediabackup.cfg (stores ip, username, passwd, etc)
/media/internal/xorgtools/etc/backup-exclude.cfg (files/dirs to exclude in backup)
/var/spool/cron/crontabs/root (cron schedling)

/media/internal/xorgtools/bck.mnu (the backup menu)
/media/internal/xorgtools/bck (directory of backup scripts)

How to Remove

Remove the backup setup...

From backup menu...

- rbk) remove all backup setup

This removes the backup related entries...

/media/internal/xorgtools/etc/mediabackup.cfg
/media/internal/xorgtools/etc/backup-exclude.cfg
/var/spool/cron/crontabs/root

/etc/fstab (for older versions of xt)
/etc/hosts (for older versions of xt)

You can re-install with 'backup initial setup' menu item.

See Xorg Tools page on how to remove xt.

ToDos/Requests/Updates

Please post other ideas and feature requests here.

Here is the PreCentral thread, which also has release notes on new versions.

Alternate Methods/Other Platforms

I chose to do Windows/cifs because that is what most have available and most systems can be configured to use it. This could be easily modified to use Mac/Linux rysnc by just using rsync daemon (cifs not necessary). The sync scripts in /media/internal/xorgtools/plugins can be altered by likely just stripping out the cifs mounting and change the rsync command to talk to rsyncd on backup computer. See Xorg Tools page on how to modify plugins.

Mac filesharing setup can be found here.

Credits

Thanks much to testers... Vincent Law, navinaq, Kyusaku, rlanza1054