[TriLUG] Ethernet Question

[Aaron S. Joyner]

Steve Kuekes wrote:

> No collisions or errors on the TX or RX and this machine has been up
> >100 days.  The adapter that is giving the slowdown is a 1000/100/10
> speed adapter as well.  Its running the sk98lin driver.  Maybe I need
> a different driver for the adapter (Its on the motherboard).

If the card works, it's probably the right driver.  There's not usually
a lot of duplication of effort among driver authors in the open source
world, because of the natural collaborative nature (exceptions aren't
too hard to come by between proprietary / open drivers for graphics
cards, or occasionally things like the h.323 channel drivers for
asterisk, but generally my statement is true, and even more so with in
kernel space).

> I tried running the mii-tool package and I get an error on the eth0
> adapter

Try ethtool instead of mii-tool?

So considering the high level problem, I'd suggest that you look at the
system while you're stressing it in this way.  Start up your copy on one
console, switch to another and run `iostat -d -x /dev/hdX 1`, then see
if the io you see in iostat lines up with the amount of io you're seeing
from the remote system.  If everything seems to line up, try throwing
some more IO at the system and see what you get.  `cat /dev/zero >
/tmp/zerofile` and see if those numbers go up (don't forget to delete
that file later, and try not to fill up the disk).  If your iostat
numbers go up you're not disk bound.  Do note that either way your
transfer speed is likely to go down, as you're at least causing more
swapping, so don't be too surprised by that.

If you're not disk bound, try reducing the complexity of your network. 
Replace the switch with a cross over cable (if lengths allow).  This
takes all of your original cables, as well as the switch, out as
potential buggy parts.  If you're still flaky, see what kind of through
put it's possible to receive at all.  You can't do this with sftp, but
use ftp or http or netcat to serve up /dev/zero on one machine and sink
it into /dev/null on the other.  A simple variation would be...
on the server: nc -u -l -p 3000 < /dev/zero
on the client: nc -u -p 3000 client > /dev/null

If you want to get even crazier, tack an extra set of redirects to send
/dev/zero to /dev/null the other way also.  That'll put some crazy
stress on your network.  Once you have this ridiculousness in play, look
at the amount of traffic going through the interfaces, and see if it's
your limiting factor.  There are a million and one ways to do this,
perhaps the simplest is to start off by zeroing the iptables counters
like this: `iptables -Z`, then run your test for a fixed amount of time
(say, 1 minute or 100 seconds), then use `iptables -L -v` to see how
much data went through the system, and divide by the time to get
units/second of throughput.

Also, don't forget to account for potential caching issues.  This
doesn't really seem to fit your scenario though, since writing to the
server is faster than reading (if I understood your original post
correctly).

If you gather all this data and still can't figure out what's going on,
post back and we'll see what else we can come up with.

Aaron S. Joyner

As another note, all this cat'ing /dev/zero to /dev/null business will
throw the cpu through the rough, so be sure to keep an eye on that in
case your cpu bound as well.  Of course, it's easier to send /dev/zero
through netcat than it is to read data from the hdd, encrypt the data,
and send it out via tcp, so if you're cpu bound for this operation
you've definitely got a problem for your real scenario.