User-agent: Scrapy Allow: / # This space intentionally left blank # If you want to learn about why our robots.txt looks like this, read this post: https://yoa.st/robots-txt User-agent: * Disallow: Sitemap: https://www.itccacentro.it/sitemap.xml