rsync backup over home network via ssh

[njtuneguy]

I would like to do incremental backups over the home network, the source is a mac, the destination would be my hda's backup share location: /var/hda/files/Backups

I found a page in the Ubuntu Community Documentation https://help.ubuntu.com/community/rsync ... e%20Backup that helped me with some of the required syntax. I'll be working from mac's Terminal app.

From the Ubuntu page (and some of mac's rsync man page), the basic syntax of the command would be as follows (somewhat changed from Ubuntu page) :

Code: Select all

rsync -avx --delete --progress -h -e ssh /Users/njtuneguy/Music/iTunes/ remoteuser@remotehost.remotedomain:/var/hda/files/Backups/iTunes/iTunes

Options are as follows:
-a (Archive Mode, same as -rlptgoD)
-v (Verbose)
-x (Don't cross file system boundaries)
--delete (Delete extraneous files from destination directories)
--progress (Show progress during transfer)
-h (Human-readable output format)
After that I get a bit hazy:
-e (Specify the remote shell to use, which is obviously "ssh"... I'm just hazy because I just started using ssh, not very confident)

Question 1) Will this work? lol... I mean with the options suggested will there be any conflict between machines/filesystems, or will the "-x" option take care of that? Are there any other options that should be included that anyone suggests?

Question 2) For my hda 'remoteuser' will be <HDAusername>, 'remotehost' will be <HDAcomputername>... what is the 'remotedomain'? Will it be the hda's ip address, or something else?

I haven't seen any options on the Amahi website or the wiki for rsync between the hda and macs so I hope this is a viable option. I've used rsync to backup files from my mac to an external drive, and it's freaking GREAT!! Thanks in advance for any help.

[njtuneguy]

Well I went ahead and gave it a shot... the syntax above didn't work. But this did:

Code: Select all

rsync -avx --delete --progress -h -e ssh /Users/njtuneguy/Music/iTunes/ 192.168.1.10:/var/hda/files/Backups/iTunes/iTunes

It turns out the username/hostname aren't necessary on a comp connected to the hda's network, just the ip addy. It just asked me for my user (njtuneguy) password, and then got to work... its syncing right now. Will report back if I find any problems, but I have my fingers crossed!!

[moredruid]

the .remotedomain is just replaceable with your local domain AFAIK

from my course last year:
note that the trailing slash indicates that only the files in that directory will be sent, not the directory itself.
e.g.:

Code: Select all

rsync -av -n /my/directory/ user@backuphost:/my/backupdir

will copy everything in /my/directory to /my/backupdir on backuphost (result: /my/backupdir/<files from directory>)

Code: Select all

rsync -av -n /my/directory user@backuphost:/my/backupdir

will copy "directory" from /my/ to /my/backupdir on backuphost (result: /my/backupdir/directory/files)

the --delete option tells rsync to delete files in the destination target if they don't exist anymore in the source directory. Sometimes you would want this, sometimes not.

bleh I hope that makes sense

anyway, you don't have to specify ssh as shell transport, this is the default. You can however give it some extra parameters (suppress X-forwarding, use compression).

I should really test this some time and write it up on the wiki

[njtuneguy]

Yeah, I definitely think alot of people would find rsync useful, mainly due to the fact that it's so much more efficient due to only syncing the files in a directory that have changed; which saves a ton of time/bandwidth. AND AFAIK it's available for all platforms.
I guess to be safe full (directory or harddisk) backups should/will be made to remedy potential file corruption or other transfer errors, which won't be any quicker with rsync... but I live on the EDGE and incremental backups are my THRILL.

[t0bbe]

I have been spending many hours trying to figure out how to do exactly what njtuneguy wanted to do, backup the iPhoto images from a Mac to the HDA.
Too bad I didn't find this thread sooner!

I am completely new to rsync and the terminal in Mac, have to learn it just because the girlfriend refuses to use PC and doesn't think she needs a backup...

njtuneguy, is this what you use still?

Well I went ahead and gave it a shot... the syntax above didn't work. But this did:

Code: Select all

rsync -avx --delete --progress -h -e ssh /Users/njtuneguy/Music/iTunes/ 192.168.1.10:/var/hda/files/Backups/iTunes/iTunes

It turns out the username/hostname aren't necessary on a comp connected to the hda's network, just the ip addy. It just asked me for my user (njtuneguy) password, and then got to work... its syncing right now. Will report back if I find any problems, but I have my fingers crossed!!

Two questions:
1) Do you need the -x option?
2) Is there a way to NOT have to manually enter the password every time? I would like to schedule it to run once every day without interaction.

Anyway, I think you just saved me a few more hours searching the net...

[moredruid]

you can use ssh-key based authentication instead of password authentication.
not sure how that works on Windows tho :-/

[t0bbe]

After quite a few more hours of searching the net and reading man pages I finally got it to work! *so happy*

Here's what I used in the end:

Code: Select all

rsync -trC --delete -e ssh --exclude '.DS_Store' /Users/username/Pictures/iPhoto\ Library/Originals/ username@192.168.0.10:/var/hda/files/pictures/iPhoto

I also got the ssh-key setup to work by running the following on the Mac:

Code: Select all

ssh-keygen -t dsa scp id_dsa.pub username@192.168.0.10:.ssh

Then on the server:

Code: Select all

cat id_dsa.pub >> authorized_keys rm id_dsa.pub

I should probably explain what it is exactly I want to do. I do all the editing and organization of the photos on the Mac, but want to be able to view the photos on any computer on the network therefore I do a one way sync to the HDA. If a photo is deletet on the Mac it is also deleted on the HDA, but not the other way around. I don't use the -a option as I don't need all the file permissions etc preserved.

I know that with the ssh-key setup I have now, anyone getting access to the key-file on the Mac can login as "username" to the HDA without knowing the password, and that's a security weakness. I've seen people talking about how to get around this, but I don't have the time to try and set that up.

Now all I have to do is to setup cron or something similar on the Mac to run the script once every day and it should be fully automatic.