[TriLUG] LVS persistence and NAT

[Jon Carnes]

On Tue, 2004-01-20 at 11:17, Ryan Leathers wrote:
> I want my cake and eat it too.  The more I use and read about LVS the
> less optimistic I am about cake eating.  Don't get me wrong - I think
> LVS is great.  I just want it to handle persistence and distribute load
> at the same time.  Let me explain...
> 
> I have set up an LVS-NAT instance in my lab with three real servers
> fielding http requests.  The real servers run an application server
> where state is important.  
> 
> Prior to turning on persistence I observed that the load was being
> distributed accross all three servers, but the application was unusable.
> With persistence turned on, the application state is kept but the load
> is no longer distributed.  That is to say, all connections made from all
> hosts behind a NAT router wind up going to the same real server due to
> the persistence rule.

So you've got persistence turned on for LVS and it matches your
persistence value for your web-servers and that works.

The problem is that your LVS cluster is being accessed by multiple
people all behind the same firewall - and all their requests go to the
same webserver in your LVS cluster...

That's interesting.  I setup a similar LVS cluster four years ago and
the persistence table at that time used the socket address (IP Address +
Source Port Number).  Nat-ted hits were treated as separate connections
since they came from different source ports and different folks coming
from behind the same firewall were distributed.

I know this, because I tested the setup from behind a secondary firewall
that I setup, and then measured the hits on each LVS server as they were
coming in (of course I was coming at the LVS from behind a OpenBSD
firewall).

Check to see if your LVS persistence table can be setup to use the whole
socket rather than just the IP address of the source.
> 
> I understand that persistence is dependant solely upon the source IP
> address and the protocol in use.  I also see that a mask may be
> specified to account for multiple / changing source addresses.  This
> seems fine if there are not too many requests from the same host /
> network.
> 
> Suppose I have a number of hosts connecting to my application servers. 
> Is there a way to maintain state while also distributing the load?  Can
> I have my cake and eat it too?  I originally thought firewall marks were
> the ticket but I am coming to understand that marking will only
> associate multiple protocols which will do nothing to distribute the
> load when persistence is required.
> 
> I suppose I could move to a more complex clustering model on the back
> end, but it would be the bees knees if LVS could be configured to
> acheive both goals.