[svlug] Rsync across SSH & alternatives

[Andrew Farley]

Hey Skip,

To reduce your overhead and speed it up I'd recommend going for something like netcat (with tar).  Netcat is just a really simple transport mechanism and you can just push files through it, with no actual intelligence or overhead to it (like rsync).

If you want a persistent connection/listener run netcat via xinetd or make a bash script to keep running a netcat listener with the proper parameters.

If you're worried about security and want a more persistent connection, run this this via a ssh tunnel or through a vpn (like openvpn).

Cheers,

Andrew Farley
CouchSurfing Systems Administrator
http://couchsurfing.org/farley
"making the world a smaller place..."




On May 12, 2010, at 9:00 PM, Skip Evans wrote:

> Hey all,
> 
> I'm doing some PHP work for a small company that monitors 
> international currency trading data. So they have 15 files on 
> a server that represent data on 15 international currencies, 
> and these files are updated in round robin fashion each 
> minute, so that basically they have 15 file updates per 60 
> seconds.
> 
> They want these files sync'd up to another server in a remote 
> location (east coast to west cost) in real time.
> 
> Initially I thought of using rsync across SSH for security 
> would work, until they told me how often they need to be 
> updated, and given the time each rsnyc execution might take to 
> make a connection, log in, move the file and log out, that 
> might not be quick enough.
> 
> Can I get some feedback on this scenario and what some 
> alternatives might be? Would it be possible to keep an SSH 
> connection open to run the rsync's across, remote mounting, etc?
> 
> They already have a lag in their updates on the initial 
> server, so they want the updates from the first source server 
> to the destination to be as fast as possible.
> 
> Thanks,
> Skip
> -- 
> ====================================
> Skip Evans
> PenguinSites.com, LLC
> 503 S Baldwin St, #1
> Madison WI 53703
> 608.250.2720
> http://penguinsites.com
> ------------------------------------
> Those of you who believe in
> telekinesis, raise my hand.
>  -- Kurt Vonnegut
> 
> _______________________________________________
> svlug mailing list
> svlug at lists.svlug.org
> http://lists.svlug.org/lists/listinfo/svlug