[TriLUG] Web Usage Tracking...

[Christopher Knowles]

OK, I thought that was what was happening.  While analyzing the packet logs 
is possible, it's pretty hard to do right.

Doing an hourly netstat doesn't show you everything... or are you mainly 
interested in people being web whores?  I think I've got the masq tables set 
to 15 minutes, so a quarter hourly would show me everything... yikes.

In any case... thanks.  I've not used the cut function before.  (Learn 
something... sometimes several somethings... everyday)

CJK

On Friday 28 September 2001 02:58 am, you wrote:
> > On Thursday 27 September 2001 06:24 pm, you wrote:
> > > > You could set ipchains to log all outgoing packets destined for ports
> > > > 80, 443, and 8008.  This would give you basic traffic analysis.  Not
> > > > perfect or elegant, but it's quick and easy.
> > >
> > > Heck, go ahead and log everything, but make sure you put in a scsi disk
> > > sub-system for the firewall or your performance is going to take a hit.
> > >
> > > I have a couple of firewalls based on RH6.2 and they work great for
>
> logging
>
> > > and I can even tell in real time what is going on:
> > >   netstat -Mn |grep 80
> > > tells me who is using the web and for what.  I have cron jobs pull the
>
> top
>
> > > 10's out of my logs nightly and then flush the logs after seven days.
> >
> > <SNIP>
> >
> > OK, can you give me an example of the cron job you use for that?  (I
>
> assume
>
> > the logs to which you refer are the packet logs...)
>
> Actually for logs, I pump netstat -Mnc into hourly log files that end with
> .txt.  To process them I use the following script:
> ===
> echo Top Ten http users
> cat *.txt |grep " 80 "|cut -c15-32 |sort |uniq -c |sort -g |tail -10
>
> echo Top Ten http destinations
> cat *.txt |grep " 80 "|cut -c36-55 |sort |uniq -c |sort -g |tail -10
>
> ===
> If you don't like IP's then you can pipe the IP's through:
> ... |nslookup |grep "Name"
>
> I also like to look at just the records from midnight to 6am.  This gives
> me a good idea of the general Background traffic on the network.  This
> traffic is mostly from folks who have left automated processes running on
> their boxes.  It is amazing how many connections we have that go to
> weatherbug...
>
> Most folks just don't realize the cumulative consequences of their actions.
> I had one lady here that had subscribed to over 16 automated services.  80%
> of my night-time traffic was being generated from her box.  Oddly enough,
> she also complained about how slow her box was compared to everyone else...
> Well daaa. (sorry, its late.)
>
> Jon
>
> _______________________________________________
> TriLUG mailing list
> http://www.trilug.org/mailman/listinfo/trilug