[TriLUG] MSN bot is pounding my website...

[David McDowell]

I've seen a lot of MSN bot traffic... with this new search they are
trying to provide to people, they are trying to build up their
resources.  That MSNbot has hit me constantly for over an hour once...
hitting every single thing I own from regular web pages to images to
thumbnails, etc.  I've never seen Google hit me that often or that
much, and Google has me so well indexed, you can surf my photo gallery
visa-vi the Google Image search... LOL


On Thu, 9 Dec 2004 15:05:19 -0500 (GMT-05:00),
gregbrown at mindspring.com <gregbrown at mindspring.com> wrote:
> I know it's from MSN because it leaves the following in my log:
> "msnbot/0.3 (+http://search.msn.com/msnbot.htm)"
> 
> I assume over at MSN they are trying to scrape the Internet to build up their own web search engine.  I am curious if others are seeing this same activity.
>