[TriLUG] Has DNS on RR died?

[Ben Pitzer]

On Sat, 2002-06-01 at 11:40, Andrew C. Oliver wrote:
> I gotta say dude, between the mail server (which finally seems to be 
> working) and the DNS, its kinda laughable.
> I mean I had a DNS server running on Linux for a few years with no 
> problem, granted RR gets more volume, but
> then again they can probably afford something bigger than an old box of 
> parts put together.

Actually, we have very solid hardware, it's the software that's the
problem.  Yes, we do get a ton of volume, as our servers are handling
more customers than most enterprise solutions can handle (> 300K
customers).  Look, nobody knows better than us what we're dealing with. 
Most of us there were not working for RR when all this got built.  We're
looking for ways out of the current situations, and we're looking for
ways to build scalable solutions that will continue to scale even if we
can double our customer base.  However, there are precious little
solutions out there that can do this, and most of them cost money. 
Plus, in a massive user environment such as ours, it is an absolute MUST
to have some kind of support from the manufacturer of any product that
we use, because class action law suits kill companies, and we don't want
to die.  Pardon the melodrama, but it's happened.

> The mail server obviously suffered from the Windows virus of the day 
> syndrome and was constantly "out of memory", but
> there seemed to be a strong correllation between the viruses I heard 
> about on CNN or watched trying to attack my webserver
> and the RR mailserver being down.

Actually, to my knowledge, our mail servers have never been hacked or
infected with a virus.  What kills mail servers that can also correlate
to tons of viruses showing up in your mailbox, or attacking your web
server?  Spam.  We've put a solution in place to reduce the amount of
spam hitting our mail servers going in either direction.  It's been
working so far, but as folks who have worked in an evironment such as
this one know, anything can happen.
 
> While I understand the requisite amount of Coporate BS, I'm a big fan of 
> listing the problems, showing a solution and putting it
> in front of the right person.  You win a lot more often than you might 
> think (as long as your not a contractor ;-) ).

As to this, the short answer is:  we have done exactly this.  Look, just
listing the problems, and putting them in front of the right person does
not mean that the problem will always get fixed expeditiously.  We've
put the issue in front of those people.  And they've listened.  And
they've said "Tell us what you need."  And we've told them.  And they've
said "Excellent.  I'll have engineering build that solution and get it
tested right away."  And we've waited.  We've been expecting a new DNS
solution for a while now.  And it should be here soon.  I promise.  I
could explain it all to you here, but you don't want to hear it, and I
don't want to explain it.  Just suffice it to say that in any corporate
environment (as in larger than you and your 5 coworkers in your
home-based business) things just aren't always as easy as we the geeks
seem to think that they should be.  Black and white issues to us are
very gray to the business types who sign our paychecks.  And until I can
sign my own paycheck, I play it their way, and try to make it better
however I can.  I don't bullshit, I don't play politics, I just tell it
straight, and become a pain in the ass till it gets done.  'Nuff said.
 
> I  mean "we're working on it" would mean more if it hadn't been 
> degrading for a whole year.  While I hate corporate welfare
> or giving charity money to poor disadvantage multi-billion dollar 
> companies, I think this affect enough of us...Maybe this should be a trilug
> project ;-) hehe.

Yeah, well I'm just going to hope that I didn't violate my NDA, and go
back to making sure that y'all have DNS available now.  Sorry to get
snippy, but trust me, this is a sore spot for me, too, and you folks
aren't the only ones who've noticed and complained.  I'd love to tell
you more, but what I've said, which incidentally all boils down to
"we're working on it" is about as much as I can say.

Regards,
Ben Pitzer