[TriLUG] why is it slow?

[Ryan Leathers]

Heheh... Aaron, I try to always read your posts, even when I'm not all
that interested in the topic.  Today was the sweet payoff.  What solemn
pride is mine in the knowledge that I have, in some small way,
contributed to your appreciation for doing things the "network guy"
way :)



On Thu, 2006-09-14 at 23:22 -0700, Aaron S. Joyner wrote:
> Rick DeNatale wrote:
> 
> > On 9/13/06, Aaron S. Joyner <aaron at joyner.ws> wrote:
> >
> >> So this post isn't entirely useless - my gut instinct is that the
> >> problem is related to the "u-turn" problem as described, but I'm at a
> >> loss to explain precisely the internals of why.  Assuming the NAT
> >> implementation is anything close to *sane* on the embedded router, this
> >> really shouldn't be a problem.  Then again, don't trust the Chinese or
> >> Korean guy who wrote the firmware to have done a sensible job on his
> >> first programming project.
> >
> >
> > I've had appliance routers (SMS I think) which refused to recognize
> > "U-turn" addressing.
> >
> > Unlike Brian, there are routers that do this, the Netgear I'm using
> > now has no problem with it.
> >
> > Just out of curiosity how would you set up NAT routing in Linux to do
> > this, with port forwarding?  For example say:
> >
> > wan-interface gets it's ip address from isp via dhcp
> > lan-interface 192.168.1.1
> > lan devices use 192.168.1.1 as gateway.
> >
> > for requests coming from either outside or inside:
> >     http connections to wan ip address get forwarded to 192.168.1.2
> >     wan ssh connections to wan ip address get forwarded to 192.168.1.3
> >
> Okay, so I started writing this like 30 mins ago, and discovered along 
> the way that this doesn't work for this case easily at all.  In fact, I 
> tried to cook up a way with just iptables, and failed miserably.  Allow 
> me to summarize what I was originally thinking, and then explain why 
> this case is messy, and provide a completely wacky solution or two.
> 
> In my mind, when you have a simple home network, and a Linux box, you 
> run the services on the gateway.  I don't typically have additional 
> servers behind my gateway which I push ports through to, via some 
> mechanism ala DNAT.  If you do run services on the inside, and DNAT into 
> them, you don't loop through the router for numerous reasons, including 
> a) inefficient use of network links, b) weird problems like we're about 
> to discuss, c) it just feels wrong.  :)  The typical way to handle this 
> problem is at the name space level.  If you're addressing service Foo as 
> foo.joyner.ws, then the external DNS view will return the external IP 
> address, and the internal DNS view will return the internal IP address.  
> In the days before views, you either ran two DNS servers with different 
> zone files, or you used two different domain names (foo.int.joyner.ws, 
> for example - where int is short for internal).  This way, you use the 
> names, life is happy, and traffic flows logically.  If you need to use 
> IPs to address the service, you better understand what IP to use from 
> where.  :)
> 
> Anyway, on with why this is messy in Linux.  If you originate traffic 
> internally from 192.168.1.5, and address your traffic to the external IP 
> address, and have the appropriate DNAT rule, your packet gets passed 
> back inside to 192.168.1.2 just fine.  The problem is that you can only 
> change the destination address, or the source address.  You can do 
> either one with the DNAT or SNAT targets in iptables, respectively.  
> Unfortunately, both of these targets terminate rule processing and 
> immediately deliver your packet on it's merry way, out the interface.  
> Choosing the DNAT rule is the logical choice, as that will at least get 
> your packet to where it needs to go.  The problem is, the source address 
> is still 192.168.1.5, so when 192.168.1.2 writes the response packet, it 
> delivers it directly to 192.168.1.5.  It does not send it back through 
> the gateway (as you would sort of hope it would).  Consequentially, when 
> that packet arrives, 192.168.1.5 doesn't recognize it as part of the 
> stream it originated to the external IP address (naturally), so it drops 
> it on the floor, and you get no where.  I googled, I looked through 
> Usenet archives, I searched for quite some time for a way to change both 
> the source and destination address of a packet with iptables.  Some 
> people seem to be suggesting that you can just use an additional SNAT to 
> fix the problem (and believe me, it seemed logical before reading the 
> iptables man page, and I did try - oh did I try), but my testing proves 
> out that this simply does not work.  Once the packet matches the DNAT 
> rule, you get no more opportunity to match any appropriate SNAT rules, 
> and vice versa.  If someone else out there knows of a way to make this 
> work via iptables alone, I'd be mighty interested in it, because I can't 
> come up with a way, and it sure feels like there should be one.
> 
> On to the truly wacky ways to actually make this work.  So, a few things 
> came to mind immediately, all of which are horribly ugly, and if you 
> implement them... well... I'm not to blame.  First off, make sure your 
> DNAT rule is sufficiently specific that it doesn't capture traffic from 
> the internal network (a simple -s ! 192.168.1.0/24 will suffice).  Then, 
> start tacking on rules like -j REDIRECT -port 8080, to capture that port 
> 80 traffic and redirect it to a local port 8080.  Then, run a 
> port-redirector on port 8080 which will capture the traffic, and 
> originate new traffic on to the destination box.  This will get the 
> right source IP address into the packet, and cause 192.168.1.2 to 
> respond to the gateway, which can then pass the packets back through the 
> port redirector, back to the REDIRECT session / rule, and then 
> eventually back to 192.168.1.5.  Don't blame me, I said it was ugly.  I 
> also promised to provide more than one ugly solution.
> 
> The other way to skin this cat, is to do weird things on the backend 
> servers.  You could do something akin to port based policy routing, such 
> that you add an iproute2 rule to match traffic originating from port 80 
> on the web server, traveling to any host on the local network, and shunt 
> it to a different route table, which only has an explicit nexthop set 
> for all hosts on the network to the default gateway.  The amusing thing 
> here is that you can't match packets with just iproute2 based on port, 
> you only get src and dst ip address.  You can match on fwmark, from 
> iptables, though.  So you mark the packets with iptables via the -j MARK 
> target, then apply the route to them with the `ip rule` command.  This 
> solution may or may not actually work, depending on how the gateway 
> handles that incoming traffic destined for another host on the same 
> network.  It's late, and I don't want to setup the above absurd solution 
> to see it if will work or not.  :)  If anyone's feeling bored, let me 
> know -- I think it'll work, but I'm not quite sure how you'd have to 
> specify the route(s) in the alt routing table, ie. can you do a simple 
> /24 with the nexthop set to the gateway, or do you need individual /32s 
> for everything but the machine itself and the gateway, or perhaps some 
> odd mix to make it easier to specify, with the same effect?
> 
> Let it be said that I'm a died in the wool Linux fan, and I don't think 
> many people on the list will question that.  But, I was thinking while 
> writing that last paragraph, that this is dramatically easier to do on 
> an honest-to-god router.  Not just part of it, but all of it.  Granted, 
> you have to think like a network guy, and the terminology is almost all 
> different, conceptually half of it is different, and the configuration 
> of things is 100% different, but once you beat the learning curve, man 
> is it easier.  On the flip side, all the things that seem easy and you 
> take for granted in the Linux world, become their own challenge in that 
> world.  *chuckle*  I'll stick with the things I know inside and out, and 
> jump through hurdles like above as required every once in a while.  In 
> this world, at least I know how to overcome the bizarre edge cases 
> better.  :)  Yes Ryan, this paragraph was for you.  I hope you made it 
> this far in.
> 
> It's late, I'm tired, enough typing.
> Aaron S. Joyner
>