Draftly Sitemap and Robots.txt: Control What Google Crawls
A sitemap and robots.txt file are standard infrastructure for any professional website. Draftly generates both automatically — but you can customise them.
Sitemap
What it is: An XML file listing all your pages, their last modified date, and relative priority.
Where it lives: `yourdomain.com/sitemap.xml`
What Draftly does: Automatically generates and updates the sitemap whenever you publish changes. Every new page is added; deleted pages are removed.
What you need to do: Submit the sitemap URL in Google Search Console → Index → Sitemaps.
Sitemap Customisation
In Draftly Settings → SEO:
- Exclude specific pages from the sitemap (thank-you pages, internal tools)
- Set page priority (0.1 to 1.0) — useful for emphasising key pages
Robots.txt
What it is: A plain text file telling search engine crawlers which parts of your site not to visit.
Where it lives: `yourdomain.com/robots.txt`
Draftly default:
```
User-agent: *
Allow: /
Sitemap: https://yourdomain.com/sitemap.xml
```
This allows all crawlers to access all pages — correct for most sites.
When to customise:
- Block staging or development pages: `Disallow: /staging/`
- Block internal search results: `Disallow: /search`
- Block admin pages
Common Mistake
Don't add `Disallow: /` to your robots.txt accidentally — this blocks all crawlers from the entire site. Easy to do, catastrophic for SEO.
Verification
After any robots.txt change, use Google's robots.txt tester in Search Console to verify your important pages are still accessible.
