Skip to content

← All writing

How-to

Your sitemap lastmod is probably lying

Thirty-seven per cent of sites stamp every URL with the deploy timestamp. It tells a crawler nothing except when you last shipped, and it was true of this site until we checked.

·2 min read

Figures as of 9 September 2026, taken over 350 scanned sites. They move as scans arrive — the live version is here.

Thirty-seven per cent of the sites we scan have a sitemap where lastmod is not the date the page changed. Usually every URL carries the same timestamp, and that timestamp is the build.

This site did it too. We found it by running our own check against ourselves, which is a good argument for running checks against yourself.

Why it happens

Almost every static-site generator and framework offers a sitemap helper, and the easy implementation is new Date() at build time. It is correct in the sense that the file was generated then. It is useless in the sense that a crawler is asking a different question.

// The version nearly everyone ships
export default function sitemap() {
  return PAGES.map((page) => ({
    url: page.url,
    lastModified: new Date(),   // ← the deploy, not the page
  }));
}

Why it costs you

A crawler with a budget uses lastmod to decide what to re-fetch. If everything changed today, nothing is prioritised, and the signal is discarded — a sitemap where every date is identical is treated as a sitemap with no dates, which is what you have.

For a reading agent it is worse than neutral. It is a small, checkable claim about your site that turns out not to hold, on a file whose only purpose is to be trusted.

What to do instead

Get the date from something that actually knows when the content changed.

  • A CMS — you already have an updated-at column. Use it.
  • Markdown or MDX in the repogit log -1 --format=%cI -- path/to/file gives the last commit that touched it. Note the file's mtime is the checkout time in CI and is not this.
  • Hand-maintained pages — keep the date beside the page in one place and let the sitemap read it. Ours is a list in lib/content.ts with the date and the files that are the evidence for it.
  • Genuinely unknown — leave lastmod out. An absent field is honest. A wrong one is a claim.

The same mistake in three other places

Once you have a real date, the same one should appear everywhere the page is dated, and usually does not:

  1. The Last-Modified header. Most frameworks send none for a server-rendered page, so every revisit is a full download of a page that has not changed since August. A conditional request is impossible without it.
  2. dateModified in your Article or WebPage JSON-LD, which is frequently a third different date.
  3. The visible "updated" line on the page, which is often the only honest one of the four.

Four dates for one page is three opportunities to disagree. One source, read by all four, is the fix. That is how this site does it, and the check and its weight are published like everything else.

Read this on your own site

A scan requests your page as five clients, one second apart, and shows you what each one got back. Free, no account, nothing blurred.

Scan a site

  • Does llms.txt do anything?

    We generate llms.txt files and charge for them, so read this with that in mind. The honest answer is that no major assistant is documented as fetching it, and it is still worth twenty minutes.

  • JavaScript is probably not your problem

    Six per cent of the sites we scanned fail on client-side rendering. Nine per cent are refused at the door before rendering is even a question. The advice everyone gives is aimed at the smaller number.

Your sitemap lastmod is probably lying — botready.dev