# Robots.txt file from http://www.searchengineworld.com # # All robots will spider the domain User-agent: * Disallow: /cgi-bin/ Disallow: /admin/ Disallow: /dbroot/ Disallow: /old web site/ Disallow: /remove flash into/ Disallow: /aspnet_client/ Disallow: /media/ Disallow: /images/ Disallow: /_private/ Disallow: /_vti_cnf/ Disallow: /_vti_pvt/ Disallow: /_vti_script/ Disallow: /_vti_txt/ Disallow: /lib/