###################### # *** Robots.txt *** # ###################### # Rules for certain robots ########################## # Teoma, Inktomi User-Agent: Ask Jeeves/Teoma User-Agent: Slurp Crawl-delay: 60 Disallow: /mailman/ Disallow: /cgi-bin/ Disallow: /images/ # Googlebot· User-Agent: Googlebot Crawl-delay: 60 Disallow: /mailman/ Disallow: /cgi-bin/ Disallow: /images/ # Rules for all Clients ####################### User-Agent: * Crawl-delay: 120 Disallow: /mailman/ Disallow: /cgi-bin/ Disallow: /images/ # Fasterfox-Extension User-Agent: Fasterfox Disallow: /