# robots.txt for http://www.ntsb.gov/ User-agent: Roverbot Disallow: / # EmailSiphon extracts email addresses for general use User-agent: EmailSiphon Disallow: / User-agent: * Disallow: /NTSB/ Disallow: /special/ Disallow: /Error/ Disallow: /search/ Disallow: /alj/ishelp/ Disallow: /pubmail Disallow: /registration Disallow: /tools Disallow: /bb/ Disallow: /academyregistration Disallow: /hotelresv Disallow: /speeches/engleman/cc/ Disallow: /speeches/rosenker/cc/ Disallow: /academy/rental/ Disallow: /Dockets/