[TriLUG] Outage report: dargo, 4 Sep

Kevin Otte nivex at nivex.net
Mon Sep 4 13:30:46 EDT 2006


Impact:
dargo.trilug.org, login shell, most users

Duration of outage: 1210-1315

Synopsis:
dargo became unresponsive on most services, leaving users unable to log
in.  Access to other hosts in the cluster were somewhat affected as user
home directories are NFS mounted from dargo.

At the end of the outage, the machine simply became available again.

Analysis:
After review of the logs during and after the event, it would appear
that this was in part the result of a portscan/DoS attack.  The
firewalling rules on dargo appear to be logging most (all?) packets
dropped by iptables.  The rate of incoming packets appeared to exceed
the rate at which they could be written to disk and the whole system
became I/O bound.  In fact, even after the system became responsive,
entries were still being written to the syslog.

Recommendation:
- Review the firewalling rules in place.
  - What do we REALLY need to log?
  - DROP rules for repeat offenders (denyhosts?)

Further discussion to occur on sys@

Respectfully submitted,

Kevin Otte
System Administration committee
Triangle Linux Users Group



More information about the TriLUG mailing list