[TriLUG] Runaway Java Process

[Robert Dale]

On 4/11/07, Steve Hoffman <srhoffman at gmail.com> wrote:
> I have a runaway java process that happens every 3 or 4 months or so that we
> can't seem to identify the cause of.  top shows that this one process is
> running @300%CPU and 24%RAM (full 2GB).  In the past we've just pulled this
> server from the load balancer, restarted jboss (using java 1.5.0_10) and
> gone on with our lives, but it makes me feel incredibly dumb to not know how
> to isolate the issue.
>
> What tools can anyone recommend to identify the cause of a runaway thread in
> a multi threaded application and even better, once identified is there a way
> to kill or renice a single thread as opposed to the whole process?

This is a question for TriJUG ;)

1. Enable remote JMX monitoring and attach with JConsole w/JTop
plugin.  JConsole will give you a system overview (memory, threads,
etc).  Go to the JTop tab and see which thread is eating cpu.  Go to
the threads tab and do a stacktrace dump.

2. Use the cli tools - jhat, jstat, jstack, jmap

3. Turn on JPDA and do remote debugging.  I don't know what, if any,
overhead this has when one is not connected.  But if the first two
options don't help you solve it, then this would of course give you
full access to the running system.

4. Turn on JVM logging options.  This might generate too much data
over the course of months and might not produce any more useful
information than the above.

HTH

-- 
Robert Dale