# robots.txt for http://www.whitehouse.gov/ User-agent: * crawl-delay: 20 Disallow: /fnf.cfm Disallow: /error.cfm Disallow: /fnf.htm Disallow: /test/