Safeguard
Developer Tools•

Sitemaps for docs sites: what they do, limits, and IndexNow

A sitemap helps discovery but does not guarantee indexing. Limits, which tags matter, when a small docs site can skip one, and what IndexNow adds.

Safeguard Security Team
3 min read

A sitemap lists the URLs you want crawled and can carry a last modified date. It helps discovery but does not guarantee crawling or indexing. Small, well linked docs sites may not need one, while large or new ones usually benefit. IndexNow is a separate way to tell participating engines about changes.

What Google says

Google describes a sitemap as a file that lists pages and other files and how they relate, helping crawlers work more efficiently. You may need one if your site is large, new with few external links, or heavy on video or images. You may not need one if you have roughly 500 pages or fewer that you want in results and strong internal linking.

Google adds that a sitemap helps discovery but does not guarantee every listed URL is crawled or indexed.

Protocol limits

The sitemaps.org protocol sets these limits:

  • 50,000 URLs per sitemap.
  • 50 MB uncompressed per file. Gzip is allowed, but the uncompressed size must still fit.
  • A sitemap index can list up to 50,000 sitemaps.
  • Each loc value must be under 2,048 characters, include the protocol, and all URLs must come from one host.
  • The file must be UTF-8.

Only urlset, url and loc are required.

Which tags to bother with

lastmod is the date the page itself last changed, not the date you generated the sitemap. A plain YYYY-MM-DD works. Keep it truthful, since a lastmod that changes on every build teaches crawlers to ignore it.

changefreq is a hint that crawlers may not follow. priority is a relative value among your own URLs, and the protocol notes it is unlikely to change result positions. I would leave both out.

Keep it honest

Only list canonical URLs you want indexed. Google treats sitemap URLs as canonical candidates, a weak signal, so do not list redirected or duplicate pages. See canonical URLs for versioned docs.

Generate it from your build or CMS so it matches your pages. Reference it in robots.txt and submit it in Search Console.

IndexNow in brief

IndexNow is a protocol for notifying participating search engines when URLs are added, updated or deleted. You submit one URL with a request, or up to 10,000 in a JSON POST, and prove ownership by hosting a key file. Participating engines share submitted URLs with each other.

A 200 response only means the engine received the URL, not that it will index it. Avoid submitting too often. The documentation I read did not name the participating engines, so check which ones matter to you before relying on it. I did not confirm whether any particular assistant uses IndexNow.

Quick check

Does a sitemap improve my ranking?

Google describes it as a discovery aid, with no guarantee of indexing. It is not a ranking lever.

Can one sitemap cover subdomains?

The protocol says all URLs in a sitemap must come from a single host.

Sources

Last reviewed: 19 October 2026.

Never miss an update

Weekly insights on software supply chain security, delivered to your inbox.

Self-healing security runs on Safeguard.

Your first fix PR is minutes away.

No sales call required, even your agent can complete the purchase over MCP.