# Robots.txt file for http://www.bohemiancommerce.com # User-agent: * Disallow: /cgi-bin/ Disallow: /global/var/ Disallow: /global/update/ Disallow: /global/support/ Disallow: /global/share/ Disallow: /global/settings/ Disallow: /global/schemas/ Disallow: /global/packages/ Disallow: /global/lib/ Disallow: /global/kernel/ Disallow: /global/extension/ Disallow: /global/doc/ Disallow: /global/design/ Disallow: /global/cronjobs/ Disallow: /global/bin/ Disallow: /global/autoload/ Sitemap: http://www.bohemiancommerce.com/sitemap.xml # but allow htdig to index our doc-tree User-agent: susedig # disallow stress test User-agent: stress-agent Disallow: /