#User-agent: * sets the rule for ALL search engine spiders User-agent: * #Disallow: /cgi-bin/ this stops the search engine spider indexing all items in this folder Disallow: /cgi-bin/ Disallow: /artw/ Disallow: /developmentartw/ Disallow: /docs/ Disallow: /developmentfiles/ Disallow: /pics/ Disallow: /buttons/ #Created by Adrian Turner 14.07.08