# Robot exclusion file. For format see "A standard for robot exclusion" # at http://info.webcrawler.com/mak/projects/robots/norobots.html # # Simon Williams (webmaster@cf.ac.uk) 22nd August 2003 # # Don't distinguish between robots User-agent: * # # Exclude all dynamic pages produced by Voyager # Disallow: /cgi-bin/