view wiki/htdocs/robots.txt @ 726:04671b858641

add a crawl-delay for all robots
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Sat, 27 May 2006 21:54:51 +0200
parents 77665d8e2254
children 005d8cee8f32
line wrap: on
line source
User-agent: *
Crawl-delay: 20
Disallow: