Choose the audit that fits the decision
Start by writing the decision the audit must support. A sudden loss of product visibility needs a different investigation from a planned platform migration or an annual architecture review. Record the affected commercial journeys, countries, devices and dates. Ask which changes preceded the concern, which pages should attract search visits, and which areas deliberately remain private or excluded.
Choose a broad baseline when the site's technical state is unknown. Choose a focused diagnostic when there is a reproducible symptom. Choose release assurance when a specific implementation is about to change. Mixing these without an explicit scope produces an expensive inventory with no clear finish line. The audit methodology explains how to turn an uncertain symptom into a testable investigation.
Set boundaries before collecting data: hosts, environments, templates, URL parameters, access permissions and safe request rates. Name the person who can explain routing and publishing rules. Agree what constitutes a useful delivery: usually a prioritised set of findings with evidence and acceptance criteria, accompanied by a record of unresolved questions.
Build an inventory with an intended state
A crawl is a map of what one configured crawler discovered. It is not a complete list of everything a business publishes or everything Google knows. Combine internal discovery with XML sitemaps, CMS exports and suitable search data. Preserve the source of every URL so that a page found only in an old export remains distinguishable from one linked in current navigation.
Assign an intended state to each meaningful URL class. A public service page may be expected to return a successful response and remain eligible for indexing. A retired offer may need an appropriate redirect or a genuine missing-page response. An account area needs proper access control. Without this expectation, an exclusion report cannot tell you whether anything is wrong.
Sample by behaviour as well as template: new and old pages, empty and populated categories, parameterised URLs, redirects and errors. Keep a fixed sample for later comparison and add investigative samples as evidence develops. Report the coverage gap when a source is unavailable; do not quietly substitute crawler totals for the missing inventory.
Collect evidence at the right layer
Keep four questions separate. Can a crawler request the URL? What response does the server send? What content and links exist after rendering? What does Google report about its recorded version? These are related observations with different timestamps. A successful live request does not establish that Google indexed the same output.
The evidence checklist for each finding should include the exact URL, observation time, request conditions, response status and headers, relevant HTML, expected behaviour, affected sample and a reproduction route. Add Search Console evidence or verified crawler requests where they resolve uncertainty. Preserve raw exports alongside your interpretation so another person can challenge the conclusion without repeating the entire audit.
Use crawl and Search Console analysis when visibility and current technical state appear inconsistent. Use server log analysis when the unanswered question concerns actual requests. Avoid collecting logs merely because they sound thorough: they are most useful when a clear hypothesis depends on who requested what and when.
Challenge the first explanation
Suppose a group of pages has no recorded impressions. Plausible explanations include limited demand, a recent publication date, a different selected canonical, incomplete reporting, inaccessible content or a genuine exclusion. A missing title warning cannot choose between them. Write down the alternatives and select the cheapest observation that separates them.
Investigate shared behaviour before issuing page-by-page tickets. If many URLs inherit the same robots header, the repair probably belongs in a response middleware or publishing rule. If only a few pages differ, compare their content records, routing and deployment history. A useful finding identifies the mechanism and its boundary, not just the number of rows in an export.
Negative evidence needs particular care. No URL in a crawl can mean that the page is orphaned, excluded by your configuration, outside the start folder or simply beyond a stopped crawl. No request in an origin log can mean the edge served a cached response. State what your instrument could observe before drawing conclusions from what it did not find.
Review technical systems without chasing scores
Review discovery, response handling, indexing controls, canonical signals, rendered content, internal linking and template metadata as connected systems. Follow a representative journey from a category to a detail page and an error route. Check whether the URL, response, content and links tell a consistent story. Use the audit checklist to track coverage without treating every failed check as equally important.
Performance deserves a separate evidence trail. Good Core Web Vitals thresholds are LCP at or below 2.5 seconds, INP at or below 200 milliseconds and CLS at or below 0.1, assessed at the 75th percentile of field experience. Laboratory traces help explain mechanisms; a single laboratory score does not establish the field result. Missing field coverage is an unknown, not a pass.
For structured data, distinguish parse validity, faithful representation of visible content and eligibility for a supported search feature. Valid markup does not guarantee eligibility or display. Google retired FAQ rich results in May 2026, so retain useful questions for readers without presenting FAQ markup as a visibility opportunity.
Illustrative example: audit a disappearing service group
Illustrative example: a fictional service site reports that regional pages stopped receiving search visits after a publishing update. Begin with the release date and the intended public regional inventory. Compare affected and unaffected templates. Current requests return 200 responses, but a subset carries a noindex response header inherited from a preview configuration.
Do not stop there. Check whether the header is present at both origin and edge, whether it varies by host or cache state, and whether the page HTML adds contradictory directives. Inspect a small sample in Search Console, recording its crawl date separately from your current request. A recorded exclusion predating the update would weaken the proposed explanation for that URL.
The resulting ticket names the preview-header rule, identifies the public route conditions that must change and lists exceptions that must remain protected. Expected results concern served directives and accessible content, not a promised traffic recovery. After deployment, repeat the same request matrix and monitor subsequent Google evidence. These invented observations demonstrate the workflow; they are not client results or a recovery benchmark.
Deliver findings developers can accept
Each repair needs a concise problem statement, affected scope, evidence, expected state, proposed owner and testable completion rule. Separate confirmed defects from investigations and optional improvements. Keep URL-level evidence attached to a system-level ticket where the underlying cause is shared. Explain why fixing that mechanism matters to a search journey.
Developer acceptance criteria should describe observable behaviour. For a public template, specify successful responses for valid routes, appropriate errors for invalid routes, intended robots directives in headers and HTML, canonical consistency, and critical content and links under agreed rendering conditions. Include a negative control such as a private preview route that must remain protected. Avoid criteria such as “all SEO errors removed”, which are neither precise nor stable.
Prioritise by plausible consequence, affected useful pages, confidence, dependencies and implementation risk. A small confirmed access failure on an important route may deserve attention before thousands of low-consequence warnings. Keep ownership and review dates visible. The report becomes useful when it supports engineering decisions rather than requiring developers to interpret an unexplained spreadsheet.
Verify repairs and preserve a recovery route
Retest the original reproduction before expanding checks. If the observed defect remains, a better headline metric does not close the ticket. Compare the deployed response with the expected state, then test neighbouring routes that share the changed code. Record the release identifier and evidence time so a later investigator can distinguish a regression from an unfinished repair.
Plan rollback around the affected configuration, cache and content state. Reverting application code may not revert a CDN header rule or a CMS publishing flag. Broad robots or redirect changes can damage unaffected routes, so start with a bounded deployment where the platform allows it. Define the failure that triggers reversal and who can perform it.
Close the implementation once its observable criteria pass; keep search observation separate until there is fresh relevant evidence. Google may recrawl and process changes on a different timetable, and neither a technically valid page nor a completed audit guarantees indexing or rankings. Convert confirmed failure modes into monitoring so the next audit starts with a useful history rather than another unexplained total.
Frequently asked questions
How is a technical audit different from a crawler report?
A crawler report records observations under one configuration. An audit establishes the intended behaviour, checks whether observations represent a real problem, investigates causes and gives an owner a testable repair. The report is an input. The audit also records coverage gaps, alternative explanations and deployment risk, making the conclusions useful beyond the tool that produced them.
Should every website have a complete crawl before investigation?
No. A complete crawl can be useful for baseline coverage, but a reproducible outage or harmful directive should be investigated immediately. Start with a safe representative sample and preserve its evidence. Expand discovery when it helps establish scope. A large uncontrolled crawl can overload infrastructure or become trapped in unnecessary URL combinations without improving the decision.
Can an audit guarantee that excluded pages will be indexed?
No. The audit can identify and verify removal of technical obstacles, but Google makes its own indexing decisions. Some exclusions are intentional or appropriate. Define success first through correct responses, directives, content and discovery, then observe fresh Search Console evidence. Do not make a development ticket depend on a guaranteed indexing date.
What should be handed over when access is incomplete?
Deliver confirmed findings with their supporting evidence and explicitly mark the missing sources. Explain which questions remain unresolved and the smallest additional access or export needed to resolve them. For example, missing edge logs limits conclusions about requests; it does not invalidate a directly observed incorrect header. Keep confidence specific to each conclusion.