How odd, I was looking at the statistics for this web site and I see that I get absolutely hammered by the search engines:
|Top Total User Agents
||Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)
||Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
||Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html)
||Mozilla/4.0 (compatible; Vagabondo/4.0; webcrawler at wise-guys dot nl; http://webagent.wise-guys.nl/; http://www.wise-guys.
||Yandex/1.01.001 (compatible; Win16; I)
So, this means that 101075 hits are from search engines out of the total 126256 hits or that search engines make up exactly 80% of the traffic to this site. Nice that Yahoo and Google love me so much but the 1GB of content they downloaded over 1GB of content. Would be nice if they could tone it down a bit. I've already adjusted my crawl rate on the engines but I seem to have missed Yahoo and now have made some efforts there to correct that.