A friend asked me about rsync, and I remembered i hadn’t posted my success story; in short, it works, you don’t need to install all of cygwin, but there are weaknesses in the filesystem exposed occasionally.
in AutoFS Mounted FTP via FUSE, I discussed setting up an FTP client as a filesystem to do a client-side pull of FTP content. That turned out quite vulnerable to our IT guy truncating a connection randomly through the firewall: the data stream didn’t seem to handle timeouts, and any process waiting on a data buffer tends to hang. Pulling data from a filesystem is a kernel-level thing, so a process cannot really abort a hanging FUSE request. Bad news.
I later found the simplified route, and installed it on the FTP server:
- download the ZIP file (cygwin-rsyncd-2.6.8_0.zip)
- unpack the zip file
- use rsync.exe, which uses a cygwin.dll
- that’s it
The rsync.exe is a cygwin binary, and acts very similar to its UNIX-y cousins; instead of installing a huge Cygwin stack, the developer has bundled just the necessary parts of Cygwin into the smaller single DLL, and provides it for rsync’s dependency. There’s a few other files in there, but as an rsync client, I didn’t need them.
The server has a fairly standard rsync server config, with an rsyncd.conf identifying shared directories as filesystems, and such. In general, if you have a working rsyncd, you can connect to it from this rsync.exe
Finally, my command line, as an example (my rsync.exe and the DLL are in C:\Program Files\rsync\, hand-installed), is as follows. Note that I’ve broken the line up with back-slashes to show functional groups, but when you run it, you should have it all on one line, without backslashes except the rsync.exe path.
“C:\Program Files\rsync\rsync.exe” \
–exclude=some-skipped-file’ –exclude=*a_wildcard_pattern* –exclude=backup*z \
–exclude=backup*.zip –exclude=*.mov –exclude=*.wmv –exclude=.* \
–delete –delete-excluded –chmod=ugo=rX ‘ \
-avr /cygdrive/f/path/to/users/library/ server.example.com::library/
The server has a rsyncd.conf config that says:
[ftp-services]
path = /shared/docs/library
uid = libraryowner
gid = docs-ro
comment = Library to share to all remote staff
write only = true
read only = false
list = false
exclude = some-skipped-file’ *a_wildcard_pattern* backup*z
incoming chmod = u=rwX
You’ll notice a few things that are unusual here:
- I use “-avr” in my rsync command. ”r” should be redundant with “-a”. try it without
- the source and destination paths end in a slash. I would recommend the same convention. Be consistent
- I used to have /library/* as my source, but on a push, deleted directories as direct children of /library/ are gone, no longer found by the “*” wildcard, so there deletion does not sync
- I have a bunch of –exclude options, some of which are duplicate on the rsyncd.conf entry. Try removing some once you have yours working.
- ditto for the chmod — they should act similarly, and setting it on the server sets a consistency and avoids users forgetting
- my uploads are write-only — I use a different share label in rsyncd.conf for uploads than I do downloads so that those who have the keys to read can’t butcher my stuff.
- for those crazy A:\, C:\ drive letters, you can use the cygwin special path /cygdrive/a/ or /cygdrive/c/ to map to them without playing the game of escape slashes: “was that 21 backslashes for 10 laters of scripting, or 22?”
If your transfer isn’t working (i.e. transfers nothing, or transfers it all every time):
- try adding “–stats” to get a list of what rsync is doing
- try adding “–max-size=20k” to avoid pushing HUGE files while diagnosing issues
- if the datestamping is seriously butchered on your windows boxes, try “-c” option to use checksums, understanding it’ll take (much) longer to sync
- if you habitually rename files, and rsync faithfully re-pushes the file every time, “-y” or “–fuzzy” tells rsync’s receiver to check around for similar files to use for missing files
Good Luck!