Synchronizing large volume over WAN

[Minnesnowtan]

I have a 2TB mirror here and another one across the Atlantic. I need to keep them synchronized and am considering using Rsync, but my command line experience is limited.

I am willing to learn if someone is willing to teach.

I am hoping for a script where a file gets saved to the other mirror whenever it is modified. I think that this would save on overhead. It has to go in the proper location of course. The files are from a design studio if that information helps.

[bschappel]

I tried something very similar using rsync and had very poor results. Rsync would spend so much time trying to figure out what had changed it never really got around to performing the sync.Rsync worked for a while but when the number of files grew it stopped being efficient.

I eventually went looking for commercial products and came upon FileReplicationPro. When setup properly this worked pretty well. It uses file system events and will only sync changed files. It works in real time as well. If one file changes it will start replicating immediately. You can try FRP for 30 days. I’d give it a shot.

[cloudmac]

Hello,

The issue is going to be the latency between the servers.  Rsync will get the job done, but I would suggest running multiple instances at once to get decent bandwidth.

Also check out Aspera Sync. Aspera works very well over high latency connections and it could be scripted / automated.

[Author]

Posts