[TriLUG] Rate-limiting TCP - using "tc"

Wed Jun 8 01:37:54 EDT 2005

Okay, I finally got around to looking into throttling Apache more. I ended 
up using bw_mod <http://www.ivn.cl/apache/> which I must say is extremely 
easy to set up and extremely easy to use. Everything worked correctly and 
perfectly the first time around (I don't think that's ever happened before). 
I'm not necessarily the greatest linux admin either, so the fact that 
everything worked so well really impressed me....

NO MORE HAVING TO STOP HTTPD BECAUSE SOMEONE IS DOWNLOADING A LARGE FILE 
FROM MY SERVER AND I WANT TO USE GOOGLE.

Very much worth installing and using. Thank you Jon Carnes for the 
suggestion. I am very happy with this. After the excellent instructions for 
compiling (don't be scared, this was so easy my Mom could do it), 
installing, and configuring at http://www.ivn.cl/apache/bw_mod-0.6.txt , I 
added the following to my httpd.conf:

begin snippet

# from the LoadModules section -- added automatically during compile/install
LoadModule bw_module modules/bw_mod-0.6.so <http://0.6.so>

####Setup BandWidth limitations per bw_mod####
# Configuration info: http://www.ivn.cl/apache/bw_mod-0.6.txt
BandWidthModule On
BandWidthDebug On
ForceBandWidthModule On

BandWidth 192.168.1.0/24 <http://192.168.1.0/24> 0
BandWidth all 300000

end snippet

I am using Fedora Core 2, but the author claims that it has been developed 
and tested for FC3. I have found no issues thus far for compiling, 
installing, and configuring for FC2.

Owen Berry, to answer your question "Crawls" meant something like 6 Kb/sec 
or something like that. It was very, very slow with the way I had things 
before with trying to get tc to do this.

Thanks again to Jon Carnes for his help!
Kipp

On 30 May 2005 09:12:23 -0400, Jon Carnes <jonc at nc.rr.com> wrote:
> 
> First off, Apache has it's own rate-limiting modules that are pretty
> slick. Since it's Apache you want to rate-limit that seems to be the
> best choice.
> http://www.ivn.cl/apache/
> http://www.linux-mag.com/2003-02/lamp_01.html
> http://www.steve.org.uk/Software/mod_curb/
> 
> 
> That being said, what you are trying to do will also kind of work, but
> will only affect traffic coming in to your server - not traffic going
> out... which is what I *think* you really want. In any case, you can
> rate limit incoming http traffic requests by changing the filter to
> match port 80. The third line of the script sets the filtering buffer's
> criteria. Since Voice uses udp I set my filter to match all tcp traffic:
> ...match ip protocol 6 0xff flowid 1:1
> 
> You want to set the match filter for port 80 traffic:
> ...match ip dport 80 0xffff flowid 1:1
> 
> This will match any type of traffic with a destination port of 80.
> 
> Good Luck - Jon Carnes
> 
> On Sun, 2005-05-29 at 09:18, Kipp Spanbauer wrote:
> > Jon,
> >
> > I've seen a couple of your emails regarding the linux tc command. I've
> > copied the script you thrown out here, and I'm still having some 
> trouble.
> > The issue that I face is that my linux box is also connected by samba to 
> my
> > two windows machines. If I try to copy a file from the linux box via 
> samba
> > internally, it crawls. I'm not trying to do VoIP or anything like that. 
> I
> > just want a way to throttle Apache as well as a couple of other web 
> servers
> > that I run.... Here's the script that I am using:
> >
> > # Commands to add rate limiting for TCP in Linux
> > # These commands must be run as root
> > #
> > # Create a Class based queue
> > # Set normal interface speed (10Mb) for use in calculations
> > # Note: use "100mbit" if your eth0 connection is 100Mb
> > tc qdisc add dev eth0 root handle 1: cbq avpkt 1000 bandwidth 10mbit
> >
> > # Create a 300Kb class - Beware the line wrap
> > tc class add dev eth0 parent 1: classid 1:1 cbq rate 300kbit allot 1500 
> prio
> > 5 bounded isolated
> >
> > # Tell which traffic should use the shaped class
> > # Protocol 6 = TCP - Beware the line wrap
> > tc filter add dev eth0 parent 1: protocol ip prio 16 u32 match ip 
> protocol 6
> > 0xff flowid 1:1
> >
> > # ... to match a single ip address
> > # ... match ip dst 192.196.12.9 <http://192.196.12.9> <
> http://192.196.12.9> flowid 1:1
> >
> > # If it breaks everything, back out by using:
> > # tc qdisc del dev eth0 root
> >
> > Like I said, very much the same thing that you sent out in two different
> > emails (one in October and one in March). I am on Roadrunner residential
> > class. Do I need to increase the "10mbit" limit in the first uncommented
> > line of the script to speed up internal data transfers?
> >
> > I have tried Googling tc on the web, but it seems difficult to find
> > easy-to-understand information. Any help you can provide would be 
> greatly
> > appreciated.
> >
> > Thank you,
> > Kipp Spanbauer
> >
> >
> >
> >
> > On 05 Oct 2004 21:36:34 -0400, Jon Carnes <jonc at nc.rr.com> wrote:
> > >
> > > "tc" is a very powerful Linux tool! I'm using it to setup some simple
> > > rate limits (and to setup some priority queuing based on destination
> > > IPs).
> > >
> > > We need to have a seminar on using tc!
> > >
> > > One of the interesting things about VoIP is that it uses UDP for 
> Voice.
> > > This means that you can rate-limit TCP traffic on a firewall and 
> reserve
> > > some of the precious upload Bandwidth for Voice.
> > >
> > > As an example, my cable connection gives me a 2.5Mb download but only
> > > 347Kb upload. If I want to send large emails and talk on the phone at
> > > the same time, I need to rate-limit my workstations upload speeds.
> > >
> > > This three line script works on the RedHat servers/workstations that
> > > I've tested. It limits the TCP upload to 300kb, reserving over 40kb 
> for
> > > my voice use (and since I use the G7.29 codec I only use 8kb of that
> > > bandwidth)
> > >
> > > ===
> > > # Commands to add rate limiting for TCP in Linux
> > > # These commands must be run as root
> > > #
> > > # Create a Class based queue
> > > # Set normal interface speed (10Mb) for use in calculations
> > > # Note: use "100mbit" if your eth0 connection is 100Mb
> > > tc qdisc add dev eth0 root handle 1: cbq avpkt 1000 bandwidth 10mbit
> > >
> > > # Create a 300Kb class - Beware the line wrap
> > > tc class add dev eth0 parent 1: classid 1:1 cbq rate 300kbit
> > > allot 1500 prio 5 bounded isolated
> > >
> > > # Tell which traffic should use the shaped class
> > > # Protocol 6 = TCP - Beware the line wrap
> > > tc filter add dev eth0 parent 1: protocol ip prio 16 u32
> > > match ip protocol 6 0xff flowid 1:1
> > >
> > > # ... to match a single ip address
> > > # ... match ip dst 192.196.12.9 <http://192.196.12.9> <
> http://192.196.12.9> flowid 1:1
> > >
> > > # If it breaks everything, back out by using:
> > > # tc qdisc del dev eth0 root
> > >
> > > ======
> > > References:
> > >
> > > 
> http://www.linuxforum.com/linux-advanced-routing/lartc.ratelimit.single.html
> > >
> > > 
> http://www.linuxforum.com/linux-advanced-routing/lartc.qdisc.filters.html#LARTC.FILTERING.SIMPLE
> > >
> > > ===
> > >
> > > For clients at Soho sites I go through some bandwidth testing to find
> > > the Choke Point -- the point at which packets begin to queue-up on the
> > > outbound routers. I run flood pings while slowly increasing the 
> outbound
> > > bandwidth. The pings report a steady latency until the out-bound
> > > bandwidth reaches a certain point, and then the latencies begin to 
> rise.
> > >
> > > The rise is caused by packets being queued up on one of the network
> > > routers. If outbound bandwidth exceeds this Choke Point then traffic
> > > will be queued up on the router (and I'm talking about your ISP's 
> router
> > > - not yours). It makes no sense to send data faster than this out of
> > > your facilities, as it will only be queued-up on your ISP's network.
> > > This will slow down *all* packets, including your time sensitive VoIP
> > > packets.
> > >
> > > The best course of action is to manually throttle your connection so
> > > that it stays below this choke point.
> > >
> > > Linksys WRT45G-S routers running the latest 2.09.1 firmware can do 
> this
> > > for you as well. However, some of my clients have Linux firewalls, so 
> I
> > > was forced to figure out how to do this simple bandwidth shaping on
> > > Linux too.
> > >
> > > As always, I thought I would share.
> > >
> > > Jon
> > > http://www.featuretel.com
> > >
> > > --
> > > TriLUG mailing list : http://www.trilug.org/mailman/listinfo/trilug
> > > TriLUG Organizational FAQ : http://trilug.org/faq/
> > > TriLUG Member Services FAQ : http://members.trilug.org/services_faq/
> > > TriLUG PGP Keyring : http://trilug.org/~chrish/trilug.asc
> > >
> 
> --
> TriLUG mailing list : http://www.trilug.org/mailman/listinfo/trilug
> TriLUG Organizational FAQ : http://trilug.org/faq/
> TriLUG Member Services FAQ : http://members.trilug.org/services_faq/
> TriLUG PGP Keyring : http://trilug.org/~chrish/trilug.asc
>