# Robots.txt file created by http://www.garbagecansstore.com # For domain: http://www.garbagecansstore.com/ Sitemap: http://www.garbagecansstore.com/sitemap.xml # All robots will spider the domain User-agent: * User-agent: Googlebot User-agent: InfoNaviRobot User-agent: TV33_Mercator User-agent: AVSearch User-agent: Mercator User-agent: Scooter User-agent: Slurp User-agent: SearchengineLicenceSheep User-agent: shadow User-agent: MultiText User-agent: FAST-WebCrawler User-agent: Lycos_Spider User-agent: Atomz User-agent: htdig User-agent: spider00.logika.net User-agent: NetMechanic User-agent: libwww-perl User-agent: Teleport Pro Disallow: /.dbinfo/ Disallow: /_notes/ Disallow: /cgi-bin/ Disallow: /stats/ Disallow: /logs/ Disallow: /mail/ Disallow: /.eletters/ To Disallow pages that are NOT NEEDED User-agent: Googlebot Disallow: /*.pdf$ Disallow: /msds/*.pdf$ Disallow: /store/cart.php$ User-agent: * Disallow: /images/ Disallow: /stats/ User-agent: Googlebot-Image Disallow: /