inktomisearch.com

Posted: Sat Oct 01, 2005 5:10 pm

I was wondering if anyone knows who runs these search bots:

inktomisearch.com

I get many of them at a time on the site.

Posted: Sat Oct 01, 2005 6:00 pm

Thats Yahoo.If you check your logfiles you´ll find something like this:

GET /robots.txt HTTP/1.0" 200 5235 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"

Slurp is sometimes a problem. But you can use also the crawl delay for slurp.

If you wish to get in the index of msn and some other search engines you should not ban slurp. Smile

Just read the FAQ for Webmaster at yahoo.

Posted: Sat Oct 01, 2005 6:06 pm

thanx.. Its just annoying. I have been hit by almost a 1000 since this site has opened.. 300 yahoo, 300 msn, 300 google. and 100 others.

my diabetesbytes site will have 20-30 google bots at once on the site..

very annoying

Posted: Sat Oct 01, 2005 6:22 pm

20-30 google bots is normal. But some people hate slurp, because of his behavior.I have no problems with slurp. There are other bots very rarely bots and bots who ignore the robots.txt. and these bots are a problem.