[TriLUG] copying files

[Alan Porter]

> I'm uploading through a home dsl line where 30kBps up is about it and 
> it's already busy doing other things. This is about the bandwidth I'll 
> get when it's in production.

Ah, so if you're not tired of hearing about rsync yet...

One thing you'll notice about a large OUTBOUND file transfer from your 
home is that the rest of your internet usage is crap while the file 
transfer is going on. The problem is not with your inbound connection... 
you have plenty of incoming bandwidth. The problem is that you are 
saturating your outbound connection.

So while you're backing up your data at full-throttle, your wife comes 
in to check her email and it takes FOREVER to retrieve even the tiniest 
file. She can't even play Words With Friends... it keeps timing out. 
Then you catch flack for killing the network.

Her email is slow because her TCP connection needs to send a handful of 
ACK packets to tell the remote end to keep sending. Without the ACK 
packets, the flow stops. But her ACK packets are stuck behind 400 
tractor-trailers on the information superhighway. You need to open up an 
HOV lane.

Rsync allows you to cap the bandwidth at a fixed level. Say you want to 
stay just under your 30kBps limit. You simply do this:

rsync -avz --bwlimit=28 /home/joe/pr0n/ joe at trilug.org:pr0n-from-home/

That keeps the goods flowing out, but it also leaves just a small amount 
of outgoing bandwidth to allow other normal traffic on the home network. 
In fact, you could probably even watch a Netflix movie while your files 
are transferring... as long as you don't saturate that uplink.

And here's the cool thing... you can tweak that bwlimit (KBYTES/sec) 
number by CTRL-C and trying again with a new bwlimit. Rsync is smart 
enough to pick up where it left off (OK, it's dumb, it recalculates the 
differences each time, but the end result is the same). So you never end 
up re-copying files that don't need to be re-copied.

-- 
# ɹǝʇɹoԀ uɐl∀