Sitemap generator or framework sitemap: how to decide

A sitemap generator is usually the first tool people reach for when Search Console asks for a sitemap: paste a domain, let it crawl, download an XML file, upload it. For some sites that is exactly right. For most sites running on a CMS or a modern framework, it is the wrong tool for the permanent sitemap and a very useful tool for something else. This article sets out how to decide, using the criteria that actually affect how search engines use the file: accuracy of dates, coverage, scale, languages and maintenance.

First, the two options. A crawler-based sitemap generator discovers URLs by following links from your homepage, then writes what it found into the sitemaps.org XML format. A framework or CMS sitemap is produced by the application itself from its own database of pages: WordPress, Wix and many other platforms do this, and in frameworks such as Next.js you write a small function that returns the list of URLs, which the framework serves as sitemap.xml.

Criterion 1: can the file tell the truth about lastmod?

Google's documentation says it uses the lastmod value when it is consistently and verifiably accurate, and that priority and changefreq are ignored. The sitemaps.org protocol adds that lastmod should be the date the page last changed, not the date the sitemap was generated.

This is where crawler-based generators are structurally weak. A crawler sees the page as it is today; unless the server sends a reliable Last-Modified header, it cannot know when the content changed, so many tools either omit lastmod or stamp every URL with the crawl date. A framework sitemap can read the real updated-at field from the CMS. If your site changes regularly, this criterion alone usually decides the question in favour of the framework.

Criterion 2: coverage, and why a crawl finds different URLs

A crawler can only find pages that are linked. Orphan pages, which exist but are not linked from anywhere, will not appear in a crawled sitemap. A database-driven sitemap lists them all, including pages you may not want indexed, such as thin tag archives or test pages, unless you filter them out.

This difference is useful. Running a sitemap generator or site crawler against your site and comparing its URL list to your framework sitemap is one of the quickest ways to find orphan pages and pages that should not be in the sitemap at all. We use exactly this comparison in technical SEO audits: the permanent file comes from the application, the crawler is the check.

Criterion 3: scale and sitemap index files

A single sitemap is limited to 50,000 URLs or 50 MB uncompressed. Larger sites split the list into several files and reference them from a sitemap index, which must be on the same site as the files it lists (unless cross-site submission is set up in Search Console) and in the same directory as those files or higher. Frameworks handle this in code: Next.js, for example, documents a generateSitemaps function that produces numbered sitemap files from a database query. With a crawler you would have to re-crawl the whole site every time and upload several files by hand, which is manageable for a few hundred URLs and fragile beyond that.

Criterion 4: languages and hreflang alternates

Multilingual sites can declare language alternates inside the sitemap using xhtml:link elements; Google documents this as one of its supported ways to tell it about localized versions of a page. A framework sitemap can generate these from the same data that builds the pages, so the page, its translations and the sitemap agree. Some online generators can output hreflang sitemaps too, but they depend on reading each page's tags correctly and have no way to know about a translation that is not yet linked. If you run more than one language, read our article on multilingual sitemap architecture before choosing.

Criterion 5: sitemap generator free versus maintained

A sitemap generator free of charge is fine for a one-off file, but check the tool's own page for how many URLs it will crawl and whether it respects robots.txt and canonical tags. The real cost is maintenance: a generated file is a snapshot, and every new page means regenerating and re-uploading it. A framework sitemap costs some developer time once and then stays current by itself. Google's guidance says that a sitemap with more than a few dozen URLs will need to be generated rather than written by hand, and that a CMS such as WordPress, Wix or Blogger has likely made one available already; for that reason we recommend letting the application maintain the file.

A decision checklist

• Use your CMS or framework sitemap if the site has a database of pages, publishes regularly, has more than a few dozen URLs or runs in more than one language.
• Use a crawler-based sitemap generator if the site is a small static site with no CMS, a temporary microsite, or you need a file today while the framework version is being built.
• Use a crawler in either case as an audit tool: compare its URL list against the sitemap to find orphans, redirects, noindexed pages and non-canonical URLs that should not be listed.
• Whatever produces the file, submit it in Search Console and watch the Sitemaps report for "Couldn't fetch" or "Has errors", and compare discovered pages with what you expected.

Common mistakes we see

The recurring problems in our practice are: sitemaps that list redirected or noindexed URLs, lastmod values that change on every deploy although content did not, a sitemap on the www host submitted to the non-www property, and a crawled file that was uploaded once years ago and never updated. Each of these teaches search engines that the file is not worth trusting.

If you want to check your own setup, our tools page is a starting point, and our technical SEO team can review how your framework builds the file. You can also contact us with the URL of your sitemap and we will tell you what we would change first.

Sources

• Build and submit a sitemap — Google Search Central
• Manage large sitemaps with sitemap index files — Google Search Central
• Sitemaps XML protocol — sitemaps.org
• sitemap.xml file convention — Next.js documentation
• Sitemaps report — Search Console Help
• Tell Google about localized versions of your page — Google Search Central