Prepare a checklist that can actually be completed
Create one row for each combination of technical behaviour and meaningful URL class. A single “canonicals checked” box hides differences between products, categories, pagination and regional pages. Record the intended state, sample selection, test method, observation date, result, evidence location and owner. Use explicit results such as passed, failed, not applicable and untested.
Before crawling, agree the authorised hosts and a safe request rate. Identify login boundaries, preview environments, language variants, parameter routes and publishing schedules. Ask for recent release notes and known incidents. A failed request against a deliberately protected staging host should not become a public-site access finding.
Keep a fixed verification sample as well as investigative samples. Include successful pages and deliberate exceptions so changes can be tested in both directions. The audit methodology provides the reasoning behind this sequence. The checklist supplies coverage discipline, while the investigation decides which observations deserve action and which need a more suitable test.
Check discovery against independent inventories
Begin with a crawl from normal public entry points, then compare its URL inventory with sitemaps and a current publishing export. Include Search Console page data where available, preserving its date range and property. Mark the discovery source of each URL. A page found in a sitemap but absent from navigation is a candidate for investigation, not automatically an orphan.
For representative important pages, inspect the actual incoming links. Confirm that the source page is accessible, that the destination exists in a normal anchor href, and that the link is present under the relevant rendering conditions. Record where the link appears and whether users can follow it without a session or a form submission.
Pass criteria depend on purpose: public landing pages should have an intentional discovery route, while account or transactional URLs may not belong in search navigation at all. Check crawl configuration before declaring a gap. Folder restrictions, parameter exclusions and rendering settings can produce apparent missing pages. Use orphan-page investigation when independent inventories reveal an unresolved discovery difference.
Test HTTP responses and redirect destinations
Request valid pages, retired pages, malformed routes and representative parameter variants. Record the first response and every redirect hop rather than retaining only the final destination. Verify that a moved page reaches the intended replacement and that internal links point directly to current destinations where practical. Do not redirect unrelated missing content to a generic page merely to remove a warning.
Inspect both response status and visible content. An error message delivered with a successful status is a different failure from a genuine 404. A 200 response containing an authentication challenge is not the expected public document. Repeat suspicious cases at a lower request rate to separate persistent behaviour from crawler-induced throttling.
The evidence checklist here is concrete: input URL, method, timestamp, response chain, destination, headers, representative body and expected routing rule. Pass valid routes only when their content matches their identity. For invalid routes, test the site’s intended error behaviour. Document temporary service failures separately from permanent URL decisions so availability work does not accidentally change content lifecycle rules.
Check indexing controls and canonical consistency
Inspect robots.txt, robots meta directives and X-Robots-Tag headers as different controls. Blocking crawling does not reliably remove a URL from search. A noindex directive needs to be accessible to Google to be processed. Test both the public pages that should remain eligible and the routes intentionally excluded; never assume that one template rule suits every state.
Compare canonical declarations with response destinations, sitemap entries and internal links. Where duplicate pages intentionally consolidate, inspect the target and the reason for that relationship. A canonical pointing to an unavailable, blocked or unrelated page needs investigation. Canonical signals express a preference; a local declaration does not prove that Google selected the same URL.
Record declared and Google-selected canonicals separately when inspection data is available. Preserve original URL strings before creating comparison keys. Over-aggressive normalisation can merge meaningful filter or language variants and make a harmful pattern disappear. For deeper cases, follow canonical diagnosis rather than changing every non-self-referencing canonical just to satisfy a crawler filter.
Verify rendered content, metadata and enhancements
Compare the initial response and rendered document on representative templates. Check that the main subject, useful content and critical links remain available when a visitor lands directly on the URL. Investigate missing output through resource failures, script errors, consent conditions and data dependencies. A screenshot alone cannot establish which text or links exist in the document.
Review titles and descriptions for relevance and template mistakes, not an imagined universal character limit. Test empty fields, long names, special characters and translated records. For structured data, check syntax, correspondence with visible content and any applicable feature requirements independently. Schema validity is not equivalent to Google feature eligibility. FAQ rich results were retired in May 2026; useful FAQs do not need a search-feature promise.
For performance, record the measurement context and coverage. Good field thresholds at the 75th percentile are LCP at or below 2.5 seconds, INP at or below 200 milliseconds and CLS at or below 0.1. Use laboratory diagnostics to investigate slow behaviour, and mark missing field measurements as unavailable rather than green.
Illustrative example: resolve conflicting checklist results
Illustrative example: a fictional catalogue passes the crawler’s status-code check but fails a rendered-content check on some product pages. The pages return 200, display navigation and contain a title, yet the description and purchase details are absent. Three plausible hypotheses are a failed product API, an extractor selecting the wrong container and a consent-dependent rendering path.
Choose two affected pages and one unaffected control. Open fresh sessions, capture the response and rendered HTML, and compare network failures. If the content is visible in the rendered document but your selector returns nothing, fix the test. If the data request fails only for a particular record state, investigate that state before proposing a platform-wide rendering rewrite.
The checklist records the status check as passed, content availability as failed for the confirmed state, and broader scope as under investigation. This prevents a single overall pass from hiding a real defect. It also prevents an extractor fault from creating hundreds of false engineering tickets. These are invented observations used to demonstrate interpretation, not measured site results.
Turn failed checks into acceptance criteria
A failed row needs a repair statement that describes what should change. Include the observed output, intended output, affected route conditions and a reproducible test. Group rows sharing one root cause, while retaining examples and exceptions. Give the development owner enough information to reproduce the problem without installing the original audit tool.
For the product example, acceptance criteria might require the expected product details on direct entry for every supported record state, correct error handling for invalid records, no accidental change to public indexing directives and working category-to-product links. Specify the tested consent and cache conditions. If the proposed repair changes data fetching, also check that old records and unavailable products retain their intended behaviour.
Separate engineering completion from later search observation. The developer can demonstrate correct responses and content immediately, while fresh Google crawl or indexing evidence arrives separately. Use issue prioritisation to schedule confirmed failures. Uncertain rows should become bounded investigations, with a question and a next test, rather than low-quality implementation tasks.
Retest the scope and record rollback exposure
Close each checklist item against the evidence that originally failed it. Repeating an entire audit is unnecessary when a targeted test can establish the repair, but shared code changes justify checking neighbouring templates. Retain the configuration, sample and expected results so a later run can distinguish changed behaviour from a changed test.
Define rollback exposure before deploying broad fixes. Removing a robots directive globally can expose routes that should remain excluded. Rewriting canonical logic can merge distinct landing pages. A cache purge may briefly increase origin load. Name the rollback control and the exceptions that must survive reversal, particularly when application, CDN and CMS settings are managed separately.
Finally, summarise coverage honestly: which URL classes passed, which failed, which were not applicable and which remain untested. Include the date and known inventory limits. An audit with a few well-described gaps is more actionable than a perfect-looking score based on incomplete checks. Schedule rechecks around releases and known risks rather than mechanically rerunning every row at the same frequency.
Frequently asked questions
What is the minimum evidence for a failed checklist item?
Keep the exact tested URL, observation time, request or rendering conditions, observed output, expected behaviour and a reproducible route. Add enough scope information to identify the affected URL class. A tool screenshot can support the evidence, but it should not replace the response or document detail that explains why the behaviour is wrong.
How many URLs should each checklist check cover?
There is no universal sample size. Cover each distinct template, routing rule and exceptional state, then expand where results vary. A small stable site may justify checking every public URL. A large dynamic catalogue needs explicit sampling and independent inventory comparisons. Record coverage and uncertainty rather than suggesting that a convenient sample proves universal correctness.
Should missing meta descriptions always fail the audit?
Treat the observation in context. Check whether it reflects an empty publishing field, a broken template or an intentional choice, and whether a useful description would clarify the page’s purpose. It should not automatically outrank an access failure or missing main content. The checklist records the finding; priority follows from consequence, evidence and implementation cost.
Can a passed checklist certify future releases?
No. A pass describes tested behaviour at a recorded time under recorded conditions. Future content states, dependency changes or deployments can alter the outcome. Preserve useful checks as regression tests and monitor high-risk public routes. Keep the test sample current so new templates do not silently sit outside an apparently complete checklist.