HTTP status codes for SEO

Treat the response as a contract about the resource

Decide what the requested URL represents before choosing its response. A published article, a moved product, a removed document and a temporarily unavailable application are different states. Mapping them all to a successful homepage hides useful information from visitors, crawlers and your own monitoring.

Test the exact public GET request, including redirects and the final body. A browser address bar can hide intermediate responses, while a HEAD request may follow a different application path. Record request time, final address and relevant headers. If behaviour depends on authentication, region or cookies, document that dependency rather than calling the page universally available.

Use the crawling and indexing framework to separate delivery from eventual inclusion. A correct response supports processing but does not guarantee indexing. The goal of a status audit is to make resource states accurate and stable. It should not turn every non-success response into a defect or every successful one into a pass without looking at the content.

Check whether successful responses contain useful content

A 200 response normally indicates that the request succeeded, but the body still needs inspection. An empty application shell, a consent wall or a product-not-found message can arrive with that code. Google can classify apparent error content returned successfully as a soft 404 rather than treating it as a normal useful page.

A 204 response contains no content, so it is not a substitute for delivering a searchable article. A 304 belongs to conditional request handling: it indicates the stored representation has not changed under the applicable validator. It is not a page-body response that a normal first request should use to serve content.

Sample the main heading, substantive text and expected record information, not merely content length. A large navigation shell can conceal a missing article. Compare raw and rendered responses when the site depends on JavaScript. If the main content is missing despite successful delivery, investigate rendered-content failures rather than changing the status code to make a report look better.

Use redirects that match the intended move

Use a permanent redirect when a resource has genuinely moved and the new destination replaces it. Use a temporary redirect for a genuinely temporary relocation. Google treats permanent and temporary redirect classes differently as canonicalisation signals, so choose the meaning rather than relying on whichever default a framework generates.

Inspect each hop and final destination. A redirect can appear correct in a mapping spreadsheet while a second infrastructure rule sends the destination elsewhere. Chains add failure points; loops prevent reaching content. Aim for a direct relevant destination and update internal links to avoid unnecessary intermediary requests.

Keep method semantics in mind when redirecting application actions, especially forms. SEO routing changes should not accidentally change transactional behaviour. Coordinate with developers rather than applying a path-wide redirect to every request method. Read redirect-chain diagnosis when routes interact across the application, CDN and hostname rules. Do not depend on a crawler's maximum hop tolerance as an acceptable design target for normal site navigation.

Return truthful removal and access responses

A 404 means the requested resource was not found. A 410 explicitly describes a gone resource. Choose according to the actual state and application design; do not invent a precise removal-speed advantage as the basis for a site-wide change. Both can be appropriate for genuinely removed material.

A helpful error page can retain navigation, search and contact options while returning a real error status. Do not redirect every missing product to a category that does not replace it. If a discontinued product still has useful specifications or manuals, retaining a substantive page with clear availability may be a better user decision.

401 and 403 concern access, not ordinary publication failure. If an intended public page receives them, investigate authentication, authorisation or bot protection. A CDN challenge can also return other statuses or misleading successful bodies. Treat accidental access restrictions as a delivery issue. Use single-page indexing checks to connect the response with the page's intended search role and distinguish deliberate protection from a public-page regression.

Handle temporary failure without disguising it

Server errors and throttling need infrastructure diagnosis. Capture 5xx and 429 responses with timestamps, route families and request volume. Identify whether failures originate at the edge, load balancer, application or dependency. A generic branded error page can conceal which system actually failed.

Google's crawlers can reduce crawling in response to server failures or overload signals. Persistent unavailability can also affect retained search content. Do not use temporary error codes indefinitely as a way to exclude URLs. Restore reliable service or make a deliberate lasting publication decision.

For planned temporary maintenance, coordinate the correct unavailable response and any appropriate retry information with the operations team. Avoid switching all URLs to a successful maintenance notice, which misrepresents their content. Test robots.txt delivery separately because failure of that control endpoint has its own crawler implications. Keep the incident window and recovery checks explicit. The acceptance condition is restored public service with truthful responses, not an immediate return to an earlier crawl rate or search visibility level.

Illustrative example: deleted products returning successful pages

Illustrative example: a retailer removes obsolete products, but the application returns a 200 page saying no product was found. The template includes navigation and recommendations, so a simple response-size check passes. Search Console reports soft 404s for some of these URLs.

Two explanations need testing: genuine missing records and a failed product API affecting valid records. Comparing database state and dependency responses separates them. Deleted records should receive the agreed removal response, while valid records experiencing an upstream failure should not be permanently classified as gone.

The proposed fix distinguishes record absence from service failure in the routing layer. It retains a helpful error design without changing the real state conveyed by the response. A small set of discontinued products keeps substantive support content and remains available by explicit business decision.

Test all three groups through the public edge. The illustrative counts are irrelevant; the important distinction is the state model. Rollback must not restore a blanket success response simply to suppress errors in monitoring, because that would hide the underlying classification problem again.

Build a response matrix developers can verify

Create a matrix of known live, moved, removed, private, invalid and temporarily failing examples. Specify the expected status, Location destination where relevant, final content and permitted request methods. Include a healthy control page for every major template. Test both valid and non-existent record identifiers.

Verify the public delivery stack after deployment. A framework may generate the intended response while a CDN rewrites it or serves a stale version. Check cache headers, cached error behaviour and invalidation where those affect correctness. Test both direct requests and navigation from existing links.

Define rollback triggers around changed state classification, broken transactions and widespread failure on valid pages. Preserve the route configuration and redirect map. A safe rollback may need to disable only the new matching rule rather than revert unrelated application changes. If caches retain incorrect redirects, restoring code alone will not repair visitor journeys. Include a post-rollback request matrix and named owners for edge and application configuration. Automated monitoring should alert on unexpected transitions, not just the presence of any error code.

Interpret errors in context and monitor the right set

Not every 404 in logs is a broken site feature. Crawlers can request old addresses, external typos or arbitrary probes. Prioritise errors reached through current internal links, important external references or the intended sitemap inventory. A rising count needs classification before a large redirect project.

A temporary browser success does not disprove an earlier server error. Compare log times with Search Console's crawl observations and release history. Conversely, a stored error does not prove the page still fails now. Keep current response evidence and historical processing evidence separate.

Monitor status distribution by valuable route family, origin of failure and relevant release. Track soft-404 candidates with content checks rather than counting only codes. For migrations, compare expected source-to-target pairs rather than celebrating fewer errors after everything redirects to the homepage. The useful outcome is accurate treatment of each resource class. That makes later migration monitoring and indexing diagnosis more reliable without pretending that a status code determines rankings or guarantees inclusion.