[TriLUG] need consulting for LAMP server

[Phil Smith via TriLUG]

It's been a long time since I used Debian, but back then they had a 'server' and 'desktop' kernel package (I built my own from source, there are thousands of configuration options).  Looking through the kernel packages section, there seems to be only one Debian amd64 kernel now.  There are some options in building the kernel that tune it for server usage where all tasks receive predictable time slices under heavy load, whereas a desktop kernel would try to make the foreground task such as playing a game more responsive at the expense of background tasks.  Since hardware is so fast today, a badly configured kernel can still appear fast under light load.  


Even on the desktop, the defaults for tickless timers, etc. need some tuning, for example, a couple of symptoms I'm still working on are: at the (non graphical) login prompt, the password is echoed as Gentoo, like most distros, is trying to run many things in parallel during init to win the 'fastest boot' contest.  Another issue is when the palemoon browser (and firefox from which is descended) are having trouble loading a page, they sort of lock up and this affects the rest of the desktop, so the kernel is allowing excessive time slices to tasks doing heavy socket of file i/o (kernel level vs. userspace preemption).  These symptoms are likelier to show up when you have older hardware than what the kernel developers use and test on.

I've no experience with LAMP where P=Perl vs. P=PHP, but I've seen a server farm of ~1000 physical blade servers run 'slow' with JAVA apps, and apparently Python has an incurable 'big lock' problem with multithreading so you might want to search what interpretive Perl's limitations are.


If writing a socket server app in a language like 'C', you accept connections quickly but then only queue the request, and pull requests from the queue with a reasonable number of threads that do not bog down the system, so possibly Apache is being allowed to dispatch too many tasks/threads at once.

A simple way to lock up almost any system is with node.js, which likes to do everything asynchronously, in parallel.  A few lines of code that act on every file in a directory (at the same time) may cause you to have to power off the system to regain control, and will take much longer to complete than if the tasks were done one file at a time.

Since you mentioned 'peak daytime hours' one cause of that is 'daisy chained switches' -- go in the server room during your slowdown and watch the blinking lights.