POST: Mission Reports
ETA: 11 min
PUBLISHED: 18/11/2026

Technical SEO audit checklist 2026: crawl, index, render, verify

A field-tested technical SEO audit for 2026: Page indexing triage, canonicals and hreflang, sitemaps, JavaScript rendering, log files, bot verification and CDN traps.

// SHARE_THIS_REPORT
01 / Context

Why the 2026 audit is not the 2022 audit

Most technical SEO checklists still in circulation were written for a simpler crawler. Since late 2025 Google has changed the ground under them, and an audit that ignores those changes will miss real problems.

  • New documentation home. On 20 November 2025 Google moved its crawling documentation from Search Central to a dedicated crawling site, and it has been editing it steadily since. Its crawling changelog is now required reading for any auditor.
  • Hard fetch limits. In a Search Central blog post on 31 March 2026, Gary Illyes spelled out that Googlebot fetches up to 2 MB per URL for HTML and other non-PDF files, 64 MB for PDFs, with a 15 MB default for crawlers that do not state their own limit. Bytes past the cutoff are not rendered or indexed.
  • More user agents. Google documented Google-Agent for AI agents acting on a user's behalf in March 2026, added experimental Web Bot Auth request signing in May, and on 17 November 2026 published guidance on allowing Google-Agent through.

None of this replaces the fundamentals. It does change the order of work. We audit in the sequence Google processes a site: can it be fetched, can it be rendered, which URL is canonical, and is it worth indexing. Each layer depends on the one before it, so fixing titles on pages Googlebot cannot reach is wasted effort.

The checklist below is the one we use on engineering-led audits. It assumes access to Google Search Console, raw server or CDN logs, and a crawler that can render JavaScript. If logs are out of reach, say so in the report; an audit without them is an estimate, not a diagnosis.

02 / Crawl and index

Start with the Page indexing report, not a crawler

A third-party crawl shows what could go wrong. Search Console's Page indexing report shows what Google actually decided. Start there, export every reason, and sample URLs from each bucket in URL Inspection before you form an opinion.

Reasons worth separating early

  • Discovered – currently not indexed: Google knows the URL but has not crawled it. On large sites this usually points to crawl demand or server capacity, not page quality.
  • Crawled – currently not indexed: fetched, then left out. Typical causes are thin or near-duplicate templates, faceted URLs and weak internal signals.
  • Duplicate without user-selected canonical and Duplicate, Google chose different canonical than user: your canonical signals disagree with each other or with what Google sees.
  • Soft 404: a 200 response that looks empty or like an error. Check out-of-stock, empty search and filter pages.
  • Excluded by 'noindex' tag, Blocked by robots.txt, Page with redirect, Not found (404): often correct, but confirm each one is intentional.

The report is a sample with a delay, so do not treat a single day's change as a trend. What matters is the ratio of valuable URLs in each bucket. A few thousand excluded parameter URLs is healthy; a few hundred excluded product pages is a revenue problem.

Map every bucket back to a template. Indexing problems are almost always template problems, and the fix belongs in code, not in a spreadsheet of individual URLs.

03 / Canonicals and hreflang

Make canonical and language signals agree

Google treats rel=canonical as a strong hint, not an order. It weighs the tag against redirects, internal links, sitemap entries and content similarity. When those signals disagree, Google picks its own canonical, and you see it in the report above.

Canonical checks

  • Every indexable page carries one absolute, self-referencing canonical in the server-rendered HTML.
  • Exactly one canonical exists after rendering. Google's JavaScript documentation supports injecting it with JavaScript but does not recommend it, and warns against conflicting tags.
  • Internal links, sitemaps and hreflang point at the canonical URL, not at a redirect or parameter variant.
  • Paginated series canonicalise to themselves, not to page one.

Hreflang checks

  • Each language version lists all alternates, including itself, and the links are reciprocal. One-way annotations are ignored.
  • Codes use ISO 639-1 language codes with optional ISO 3166-1 region codes. Ukrainian is uk; ua is a country code and a common mistake.
  • An x-default covers users who match no version, typically a language selector or the main market.
  • Hreflang targets return 200 and are canonical. Pointing hreflang at a redirected or noindexed URL breaks the cluster.

Multilingual sites fail here more than anywhere else, usually because translation and engineering teams own different halves. We cover the workflow side in localisation without friction.

04 / Sitemaps and redirects

Sitemaps, redirects and the host you actually serve

XML sitemaps are a discovery aid, not an indexing guarantee. Google said in 2023, when it retired the sitemap ping endpoint, that it does not use changefreq or priority, and that it uses lastmod only when the value is consistently accurate. A build that stamps today's date on every URL teaches Google to ignore the field.

Sitemap checks

  • Only canonical, indexable URLs that return 200. No redirects, no noindex, no parameter duplicates.
  • lastmod changes only when the main content changes.
  • Sitemaps are split by template (products, categories, articles) so the Page indexing report can be filtered per sitemap.
  • The sitemap index is declared in robots.txt and submitted in Search Console.

Redirect and host checks

  • One preferred host. http://, http://www. and the non-preferred https:// variant each reach the final URL in a single 301 or 308 hop.
  • Trailing-slash and letter-case policy is consistent and enforced at the edge, not partly in the app and partly in the CDN.
  • No redirect chains over two hops, and no loops. Re-crawl the old URL list after every migration.
  • Removed pages return 404 or 410. Redirecting everything to the homepage tends to be reported as soft 404.

Host and redirect logic often lives in three places: the app, the web server and the CDN. Audit all three. When we rebuild platforms in our web development work, this rule set is the first thing we move into one place.

05 / Rendering

JavaScript rendering and the 2 MB ceiling

Google renders JavaScript, but rendering is a separate stage with its own rules. In December 2025 Google revised its JavaScript SEO basics page on three points that belong on every audit.

  • noindex in the original HTML wins. Google may skip rendering when it sees noindex, so JavaScript that removes the tag later may never run. If a page should be indexed, do not ship noindex in the initial HTML.
  • Non-200 pages may not be rendered. Pages returning 200 go to rendering; pages with other status codes might not. Error pages that rely on JavaScript to show content or signals are fragile.
  • Canonicals via JavaScript are supported but not recommended. Keep them in the server response.

The byte budget

The 2 MB fetch limit sounds generous until you meet a framework that inlines its full state as JSON, or a page builder that embeds SVG icons hundreds of times. Measure the uncompressed HTML size of your heaviest templates. If critical content, links or structured data sit near the end of a large document, move them up or slim the document. Google says the limit may change, but content past the cutoff currently does not exist for Googlebot.

Rendering checks

  • Compare raw HTML with rendered HTML in URL Inspection for each key template.
  • Primary content, internal links and product data are present without user interaction.
  • Links are real <a href> elements, not click handlers.
  • JavaScript and CSS needed for rendering are not blocked in robots.txt.

Heavy rendering also hurts users. If templates are slow as well as large, read our notes on Core Web Vitals for e-commerce.

06 / Logs and bots

Log files and bot verification

Logs are the only place you see what crawlers did, rather than what a report summarises. Pull at least 30 days of raw access logs from the origin and the CDN; the edge often absorbs requests the origin never sees.

What to extract

  • Googlebot hits by template and status code. Large shares of 3xx, 4xx or 5xx point to wasted crawling.
  • Crawl frequency on money pages against parameter, filter and internal search URLs.
  • Response time for crawler requests. A slow origin reduces how much Google is willing to crawl.
  • Valuable URLs that receive no crawler hits at all: these are usually orphaned or buried.

Verify before you trust a user agent

Anyone can send a Googlebot user agent string. Verify with a reverse DNS lookup that resolves to googlebot.com, google.com or googleusercontent.com, followed by a forward lookup that returns the same IP, or match the IP against Google's published JSON range files. Those files moved to developers.google.com/crawling/ipranges/ in 2026; the old path still works for now, so update any firewall automation that fetches it.

Treat AI and agent traffic separately. Google-Agent is user-triggered, so it does not follow the same rules as Googlebot and should be judged on its own merits. Web Bot Auth, which lets crawlers sign requests cryptographically, is still experimental at Google.

Unverified bots that claim to be Googlebot are a security finding as much as an SEO one. If your logs show scraping or credential stuffing behind fake crawler strings, that belongs in a security audit, with WAF rules that let verified crawlers through.

Page indexing reasons: triage table

Use this to turn Search Console buckets into engineering tickets.

GSC reasonWhat it usually meansFirst checkTypical fix
Discovered – currently not indexedGoogle has not crawled the URL yetServer response times, crawl hits in logs, internal links to the URLFaster origin, stronger internal linking, prune low-value URLs
Crawled – currently not indexedFetched but judged not worth indexingTemplate uniqueness, near-duplicates, rendered contentConsolidate or enrich thin templates; noindex true duplicates
Google chose different canonical than userYour canonical conflicts with other signalsRedirects, internal links, sitemap and hreflang targetsAlign every signal with one URL
Soft 404200 response that looks empty or like an errorOut-of-stock, empty filters, mass homepage redirectsReturn 404/410, or add real content
Blocked by robots.txtCrawling disallowedWhether the rule is intended; blocked JS/CSSFix the rule, or use noindex if the aim is deindexing

Reason names as shown in Google Search Console's Page indexing report. Always sample URLs in URL Inspection before acting.

The one-page technical SEO checklist

Run these in order; each step depends on the one before it.

  • [+] Export every Page indexing reason and map each bucket to a template.
  • [+] Confirm robots.txt allows the CSS and JavaScript needed for rendering.
  • [+] Check one self-referencing canonical per indexable page, in server HTML.
  • [+] Validate hreflang: reciprocal links, valid codes (uk, not ua), x-default, 200 targets.
  • [+] Strip redirects, noindex and parameter URLs from sitemaps; keep lastmod honest.
  • [+] Enforce one host and one-hop redirects for http, www and trailing-slash variants.
  • [+] Compare raw and rendered HTML; keep noindex and canonical out of JavaScript.
  • [+] Measure HTML size on heavy templates against Googlebot's 2 MB fetch limit.
  • [+] Analyse 30 days of origin and CDN logs by template and status code.
  • [+] Verify Googlebot by reverse DNS or Google's IP range files at the new /crawling/ipranges/ path.
  • [+] Audit CDN features: email obfuscation, bot challenges, cached redirects, HTML rewriting.
  • [+] Find orphan pages and check titles and descriptions for duplicates by template.

Technical SEO audit: common questions

[01]

How often should we run a technical SEO audit?

A full audit once a year, or before and after any migration, redesign or platform change. Between audits, monitor the Page indexing report and server logs monthly, and run an automated crawl on each major release. Most serious technical problems arrive with deployments, so tying checks to your release process catches them sooner than a calendar does.

[02]

Does Googlebot's 2 MB limit affect most websites?

Most HTML pages are far below 2 MB, so for typical sites it is not an issue. It matters for templates that inline large JSON state, embed many SVGs or base64 images, or render very long listings. Measure the uncompressed HTML of your heaviest templates. Google says content beyond the cutoff is not fetched, rendered or indexed.

[03]

Should we fix every URL in 'Crawled – currently not indexed'?

No. Many URLs in that bucket do not deserve to be indexed: parameters, filters, thin tag pages. Sample the bucket, separate valuable URLs from noise, and act at template level. For valuable pages, improve uniqueness and internal links. For noise, stop generating or linking to it rather than asking Google to index it.

[04]

Is it safe to block fake Googlebot traffic?

Yes, provided you verify first. Use reverse and forward DNS, or Google's published IP ranges, and block only requests that fail. Never block on user agent alone, and never block by guessing IP ranges. Test WAF rules with URL Inspection's live test so that real Googlebot can still fetch your pages after the change.

[05]

Can Cloudflare settings really cause SEO errors?

They can create noise and, occasionally, real problems. Email Address Obfuscation generates /cdn-cgi/l/email-protection links that crawlers may report as 404 or soft 404. Bot protection can challenge crawlers, and cached redirects can outlive a migration. None of these is a reason to drop a CDN; they are reasons to include edge configuration in every audit.

Need an audit that ends in tickets, not slides?

We audit crawl, rendering, logs and edge configuration together and hand over fixes your engineers can ship. Tell us about your stack.

// EDITORIAL_ROUTE

Technical SEO Audit Checklist 2026: Step-by-Step | Git-Force