robots.txt

Not many web master take the time to use a robots.txt file for their website. For search engine spiders that use the robots.txt to see what directories to search...