When Google stops recrawling your blog: the sitemap date that gave it away

If you edit a blog post and Google never seems to notice, check what your sitemap says about when that post last changed. On a builder's client site our sitemap generator gave every blog post a last-modified date equal to the day it was first published, so posts we edited in September still told Google they had not changed since May, June or July. The posts were live, indexed and linked from other pages, and they still stalled. Fixing the date was the part that mattered.

What we saw

In early September we noticed that the blog section of the site had dropped from 1,212 impressions across 21 pages before 4 August to 306 across 12 pages. The site as a whole was roughly flat, so this was one section. Three posts that had each earned around 200 impressions went to zero.

All three still returned a normal 200 response and were listed as "Submitted and indexed". Search Console showed each had last been crawled between 22 and 28 July. Resubmitting them on 24 August did not bring Google back.

The first fix, which was necessary but not enough

Our first explanation was orphaning. We searched the site for links to each of the three posts. Two were linked only from other blog posts, and the third from exactly one page outside the blog. A post that is linked only by other posts in a section Google is not visiting is, in practice, hard to reach. We added seven links from service and landing pages that Google was crawling.

That was the right fix for reachability, and six days later nothing had changed. All three were still last crawled in late July. By 21 September they still had not been recrawled.

The cause: the date field

We then read the sitemap generator, which we should have done first. It had three faults:

1. Blog posts were given a last-modified date equal to their published date. Posts we had edited on 7 and 16 September still advertised May, June or July. The article's modified date in its structured data repeated the same stale date.
2. Every other page was stamped with today's date on each build. About 35 addresses claimed to have changed on every deploy.
3. The suburb pages came from a second, hardcoded list that had drifted from the real one. It listed 13 of 28, so 15 of 28 suburb pages were missing from the sitemap entirely.

Put together, the sitemap told Google that nothing in the blog had changed, and that pages without real changes changed daily. A date field that is both frozen on some pages and constantly refreshed on others is noise, and Google is free to ignore it.

Reachability gets a page crawled once. The date is what encourages another visit after you edit it. We had fixed the first and not the second.

What Google says about the field

Google's guidance on sitemaps is that it uses the last-modified date when the value is consistently and verifiably accurate, and the other fields are ignored. That is a conditional. A sitemap whose dates are wrong in both directions fails the test, and a site that fails it gets no benefit from the field at all. This is the reason a missing date is better than a false one: you lose nothing, and you do not teach the crawler to distrust the file.

What the fix looked like

The date for each page is now taken in order from a deliberate "updated" date in the post, then the date of the file's last commit in version control, then the published date, and today only as a last resort. The suburb list is read from the same place the site itself uses, so the two cannot disagree. Static pages are tied to their own route files.

When we wrote the fix up, a recrawl of the three posts had not yet been confirmed. That needs a check in Search Console's URL inspection after the next deploy, so we are not claiming the posts have recovered.

Is your own site exposed?

We checked the other sites built from the same template on 21 September. Our own pagecog.com.au and six other client sites all had the fallback that stamped today's date. One of them had drifted the other way, with the sitemap listing 18 suburb pages against 16 on the site, which left orphan addresses.

To test yours:

1. Open /sitemap.xml and pick a post you edited recently.
2. Compare its lastmod with the date you actually changed it.
3. Pick two pages you have not touched in months. If both say today, the field is being stamped on build.
4. Count the suburb or service addresses on your site and in the sitemap. They should match.

If the dates are wrong, the cheapest fix is often to remove the field from the sitemap until it can be accurate. A missing date is neutral. A false one is not.

Why this belongs in ongoing care

A stale date does not show on launch day. It shows months later, as a section that quietly stops earning impressions, and the cause is in a script nobody opens. It is the sort of fault a regular look at Search Console catches, and a reason we keep watching sites after they go live. Our SEO add-on is built around that kind of monitoring, and the base plan keeps the site itself maintained. For the related case where a site is not appearing at all, see why your website isn't showing up on Google.