[TriLUG] copying files

Joseph Mack NA3T jmack at wm7d.net
Thu Jun 21 21:24:53 EDT 2012


On Thu, 21 Jun 2012, Steve Litt wrote:

> Hey Joe,
>
> Check out this badboy:
>
> http://www.psc.edu/index.php/hpn-ssh

interesting.

He also talks about simpler encryption methods being 
faster. I don't need encryption of my data. I only need the 
ssh so the passwd doesn't go over the wire in the clear. 
After that I don't care who sees the data. I wonder if 
there's an ssh option to send the data in the clear.

There's a link to a long paper on network stack tuning

http://www.psc.edu/index.php/networking/641-tcp-tune

fortunately for us, linux is pretty well optimised.

> https://launchpad.net/~w-rouesnel/+archive/openssh-hpn

these are the patches for ubuntu

> Up to 20x faster transfer than normal SSH. This would rock 
> whether you're doing rsync over ssh, or ssh mount, or scp, 
> or pretty much anything involving SSH. I haven't tried it 
> -- I just heard about it a few minutes ago.
>
> And of course, if your bottleneck turns out to be disk 
> access at either end, or CPU at either end, or a slow 
> wire, this won't help you.


I'm uploading through a home dsl line where 30kBps up is 
about it and it's already busy doing other things. This is 
about the bandwidth I'll get when it's in production.

I have 25k files (370M) and rsync takes 57 mins to sync an
already sync'ed set of files.

> But if you're transferring between two robust machines 
> connected by Gigabit Ethernet, who knows how fast it would 
> go!

not for this project. If I'd had this setup, I wouldn't have 
needed to optimise my throughput in the first place. I could 
have just done `cp -auv` and never given it any more 
thought.

> I'm probably going to try this in the next few days.

I don't know enough about cryptography to know whether 
changing buffer sizes exposes you to intrusion.

Let us know how it goes

Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!



More information about the TriLUG mailing list