User-agent: * Disallow: /ajax/ Disallow: /client/ Disallow: /email/ Disallow: /ipages/ Disallow: /payment/ Disallow: /search/ Disallow: /subscribensave/ Disallow: /subscription/ Disallow: /supplier/ Disallow: /user/ Disallow: /z_fabric/ Disallow: /_default/admin/ Disallow: /_default/secure/ Disallow: /_default/ajax/ Disallow: /_default/ws/ Disallow: /contact/ajax/ Disallow: /oms/secure/ Disallow: /product/secure/ Disallow: /product/ajax/ Allow: / # # disallow adsense bot, as we no longer do adsense. # User-agent: Mediapartners-Google Disallow: / # # Yahoo bot is not very nice # User-agent: Slurp Disallow: / # # Yahoo Pipes is for feeds not web pages # User-agent: Yahoo Pipes 1.0 Disallow: / # # Baiduspider - Chinese Search Engine Bot - bit of a bandwidth hog - suspected of calling image scripts which create huge tmp files for us # User-agent: Baiduspider Disallow: / # # for some reason sitemaps should be ABSOLUTE, but we can't do that so here is a relative one anyway # Sitemap: https://lewissurveyors.com/_default/xml/sitemap.xml