Skip to content
Kuraib Ali

How does Google handle 404, 410, 5xx and 429 status codes?

Google's crawlers act on the status code a page returns. Here is what each common code tells Googlebot, according to Google's own documentation.

By Kuraib AliPublished Updated 10 min read

A yellow Dead End road sign on a sunny street, representing a 404 or 410 status code that tells Googlebot a page is gone
Photo: Unsplash

HTTP status codes tell Googlebot what happened to each request it makes. Google's documentation sorts them into groups. 2xx responses are considered for indexing and 3xx responses are followed as redirects. 4xx responses, except 429, mean the page doesn't exist. 5xx and 429 responses mean the server is struggling. A wrong code can remove pages from Google Search or slow crawling across a whole site.

Where Google documents status codes

Google's reference is a page titled "How HTTP status codes affect Google's crawlers", last updated 4 February 2026. It now sits in Google's crawling infrastructure documentation. It says it covers "the top 20 status codes that Google encounters on the web."

The same page sets the expectation for success codes: "For Google Search, an HTTP 2xx (success) status code doesn't guarantee indexing." A 200 means Google passes what it received on to the next processing step. Whether that content is indexed is a separate decision, covered in the crawling vs indexing guide.

The table below summarises how the page describes each group.

Status code groupWhat Google's crawlers doEffect on Google Search
2xx (success)Pass the content on for processingThe content may be indexed, with no guarantee
301 and 308Follow the redirectStrong signal that the target should be processed
302, 303 and 307Follow the redirectWeak signal that the target should be processed
304 (not modified)Report that the content is unchangedNo effect on indexing beyond recalculating signals
4xx except 429Treat the content as missingIndexed URLs are removed and crawling slows over time
429 and 5xxTreat the server as overloaded or failingCrawling slows, and indexed URLs are kept but eventually dropped

The sections below take each group in turn, starting with the codes that tell Google a page is gone.

Is a 410 different from a 404 for Google?

Google's documentation gives no separate treatment for 410. The page lists 400, 401, 403, 404, 410 and 411 together and states: "All 4xx errors, except 429, are treated the same."

It then describes what that treatment is. Google's crawlers tell the next system that the content doesn't exist. For Google Search, "the indexing pipeline removes the URL from the index if it was previously indexed." The page adds: "Newly encountered 404 pages aren't processed. The crawling frequency gradually decreases."

My reading is that the choice between 404 and 410 is about describing the situation accurately. A 410 says a page was removed on purpose, which other clients may use. For Google, both lead to the same documented outcome. The redirect question is separate, and the 301 redirects guide covers when a removed page should point somewhere else instead.

What is a soft 404?

A soft 404 happens when a page says "not found" but returns a 200 success code. Google's Search Console help defines it as a page that "returns a user-friendly 'not found' message but not a 404 HTTP response code."

Google's status code page explains how this gets flagged. If a 2xx response contains "an empty page or an error message", Search Console reports it as a soft 404. The help page gives a direct recommendation: "We recommend returning a 404 response code for truly 'not found' pages."

Common causes I would check first include:

  • out-of-stock product templates that keep a 200 code with an empty product area
  • search or filter pages that return zero results
  • deleted content where the content management system (CMS) shows a friendly message on a normal page
  • JavaScript apps that render an error message after the server has already sent a 200

To check a flagged page, Google suggests running a live URL inspection and clicking "View tested page" to see how Google renders it. If the page has useful content, add more of it. If it is empty, return a 404 (or a 410).

How Google handles 5xx server errors

Server errors work differently from 4xx codes. Google's documentation says "5xx and 429 server errors prompt Google's crawlers to temporarily slow down with crawling." For Google Search, "already indexed URLs are preserved in the index, but eventually dropped."

The slowdown scales with the problem. The page says "The decrease in crawl rate is proportionate to the number of individual URLs that are returning a server error." When the server recovers and returns 2xx again, Google "gradually increases the crawl rate for the site."

The page also sets a limit on that patience. "Google's indexing pipeline removes from the index URLs that persistently return a server error." Google does not give a fixed number of days in that sentence. Its crawl rate page, covered below, gives the most specific guidance.

My reading is that a short outage is tolerated, while a long one costs indexed pages. The crawl budget and log files guide shows how to spot 5xx spikes in Search Console's Crawl Stats report.

Why 429 counts as a server error

The 429 code (too many requests) sits in the 4xx range, but Google treats it like a 5xx. The documentation says: "Google's crawlers treat the 429 status code as a signal that the server is overloaded, and it's considered a server error."

This makes 429 the one 4xx code that slows crawling without being read as "this page doesn't exist." That matters for rate limiting, where Google reported a rise in misuse in 2023.

An illuminated red no-entry signal glowing against a dark background, representing a 403 or 404 used to block Googlebot

Don't use 403 or 404 to slow Googlebot

Google published a Search Central blog post on 17 February 2023 titled "Don't use 403s or 404s for rate limiting." It said Google had noticed "an uptick in website owners and some content delivery networks (CDNs)" using 4xx errors to slow Googlebot. The post's short version was: "please don't do that."

The reason is the 4xx rule above. The post states: "All 4xx HTTP status codes (again, except 429) will cause your content to be removed from Google Search." Google's status code page repeats the point: "Don't use 401 and 403 status codes for limiting the crawl rate. The 4xx status codes, except 429, have no effect on crawl rate."

My reading is that a bot protection rule that answers Googlebot with 403 fails twice. By Google's account it does not slow crawling, and it can remove the blocked URLs from Search. The AI crawlers guide covers how firewall rules can send 403 or 429 responses to bots without anyone noticing.

How to reduce crawl rate safely

Google's page on reducing crawl rate (last updated 6 October 2026) gives the supported route. For an urgent, short reduction, it says to "return a 500, 503, or 429 HTTP response status code instead of 200 to the crawl requests."

It also states the cost. Google reduces the crawl rate "across the whole hostname." It advises against doing this "for a long period of time (meaning, longer than 1-2 days)." In Search, "if Googlebot observes these status codes on the same URL for multiple days, the URL may be dropped from Google's index."

Pausing a site with a 503

Google has a separate page on temporarily pausing a website, last updated 10 December 2025. Its first advice is to limit functionality, such as disabling the cart, instead of taking the whole site offline. If you must disable the site for 1 to 2 days, it says to return a 503 error page. For a longer closure, it suggests an indexable home page that returns 200.

For the 503 route, the page lists these practices:

  • keep allowing crawling through robots.txt
  • don't return a 503 for the robots.txt file itself, because "this blocks all crawling"
  • send a Retry-After header with an estimated date or delay
  • keep the 503 error page light, using static HTML and inline CSS

It also lists what not to do. Google says not to block the site "by returning 403, 404, 410 HTTP status codes, or with a noindex robots meta tag or x-robots-tag HTTP header." Its FAQ warns that "completely closing a site even for just a few weeks can have negative consequences on Google's indexing of your site."

A blue Sorry We're Closed sign hanging inside a glass shop door, representing a 503 response that tells Googlebot a site is temporarily unavailable

What happens when robots.txt returns an error?

The robots.txt file has its own rules, set out in Google's robots.txt specification (last updated 23 October 2026). The status code of that one file can change how Google crawls the whole site.

robots.txt responseWhat Google does, according to its specification
4xx except 429Treats the file as if it didn't exist and assumes no crawl restrictions
5xx, first 12 hoursStops crawling the site and keeps trying to fetch robots.txt
5xx, next 30 daysUses the last good version of the file while still trying to fetch a new one
5xx, no cached versionAssumes there are no crawl restrictions
5xx after 30 daysBehaves as if there is no robots.txt if the site is generally available. Stops crawling if it has general availability problems

The 4xx row has a side effect that Google's 2023 blog post points out. If robots.txt returns a 4xx, any rule you had there is gone. In my view, the robots.txt file needs its own uptime check. The robots.txt and AI crawlers guide covers the rules themselves.

A status code check for your site

These checks use tools most sites already have. Run them after any migration, CDN change or new firewall rule.

  1. Test removed pages. Request a few deleted URLs and confirm they return 404 or 410, or a 301 to a close replacement.
  2. Test empty states. Request an out-of-stock product, a zero-result search and a deleted article. Any that return 200 with an error message are soft 404 candidates.
  3. Read the Page indexing report. Look for soft 404 and server error (5xx) rows, and inspect a sample with URL Inspection.
  4. Read Crawl Stats by response. A rise in 4xx or 5xx responses after a change points to the cause.
  5. Check your bot rules. Confirm that rate limits and firewall rules send 429 or 503 to Googlebot, never 403 or 404.
  6. Check robots.txt. Confirm it returns 200 and loads fast, including during maintenance.
  7. Plan maintenance windows. Use 503 with Retry-After, keep robots.txt at 200 and keep the outage under one to two days.

None of these checks guarantees rankings. They make sure Googlebot reads each page's status the way you intend, which is a precondition for everything else in SEO.

Related: crawling vs indexing, crawl budget and log files, 301 redirects and link equity, robots.txt and AI crawlers.

Frequently asked questions

Is a 410 better than a 404 for removing a page from Google?

Google's status code documentation lists 404 and 410 in the same group. It says every 4xx error other than 429 gets the same treatment. The URL is removed from the index if it was indexed, and crawling of it slows over time. Use 410 if it describes the situation better, but Google documents no separate effect.

What is a soft 404?

A soft 404 is a page that shows a "not found" message to visitors but returns a 200 success code. Google's Search Console help recommends returning a 404 response code for pages that are truly not found.

How long can a site return 503 before Google drops pages?

Google's crawl rate documentation advises against serving 500, 503 or 429 for longer than one to two days. It warns that a URL returning these codes for several days in a row may drop out of the index.

Can I use 403 or 404 to slow Googlebot down?

No. Google says 401 and 403 should not be used to limit crawl rate. It adds that 4xx codes other than 429 have no effect on crawl rate. They can instead lead to the URLs being removed from Google Search. To slow crawling, Google points to 500, 503 or 429.

What happens if my robots.txt file returns an error?

Google's robots.txt specification says a 4xx response, except 429, is treated as if no robots.txt file exists, so Google assumes no crawl restrictions. A 5xx response makes Google stop crawling the site for the first 12 hours while it keeps trying to fetch the file.