# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: User-agent: DataForSeoBot Disallow: / User-agent: * Disallow: /fabric_items/ Disallow: /designs/ Disallow: /product_information/ Allow: /assets/ Allow: /intl/ Allow: /*.css Crawl-delay: 1 Sitemap:https://www.spoonflower.com/sitemap.xml User-agent: Pinterestbot Crawl-delay: 0.01 # Amazon User-agent: Aranhabot Crawl-delay: 0.1