Here's a fairly complete description of Robots, Spiders, Web Crawlers, Worms, and Ants and why "/robots.txt" files can keep appearing as a "file not found" on your web stats review.


RETURN TO Widgets