You submitted a sitemap, and Search Console answered with a sentence that reads like a fault.

Sitemap could not be read. Or the shorter version on the list: Couldn't fetch. They are the same message on two different screens, and neither one says your file is broken. They say Google did not get it.

My sitemap was fine the whole time this happened to me. A valid XML file, correct content type, one request, a 200 back. I could open it in a browser and I could fetch it from a terminal. Google still said it could not be read.

The thing stopping it was not in the sitemap at all, and finding that out took me two days of looking in the wrong place. So this is the order to check things in, taken from Google's own page, with the one cause that took me longest to see written down in full.

The three statuses, in Google's own words

Google's sitemaps report page lists three values for the status of its last request. Learn these three and most of the panic goes away.

"Success: The sitemap was loaded and processed successfully with no errors. All URLs will be queued for crawling."

"Has errors: The sitemap could be fetched, but has one or more errors. Any URLs that could be parsed without errors from the sitemap will be queued for crawling."

"Couldn't fetch: The sitemap could not be fetched for some reason. See how to troubleshoot fetch errors."

Notice what the middle one says. If your file has a mistake in it, Google can still fetch it, and you get Has errors with a list of the specific problems. That is what a broken file looks like.

Couldn't fetch is not that. It is the status for a file Google never got a look at.

There is a fourth value that is easy to miss, on the Type column rather than the status:

"Unknown: The submitted file is not a known sitemap type or the sitemap hasn't been processed yet."

If your report says Type Unknown, Google is telling you it has not looked yet. Nothing is wrong with the file, because nothing has read the file.

Why you see two different sentences for one problem

This is the part that sends people in circles, and it is just two screens.

the Sitemaps list       "Couldn't fetch"
the details page        "Sitemap could not be read"

Google's page describes the second one as the explanation of the first:

"If the details page says Sitemap could not be read, then the fetch failed for one of these reasons, as shown in the details page. Expand the section below the error to see more details."

So there is no second problem to fix. Click into the sitemap, expand the error, and you get the reason. If you have been treating them as two separate faults, you have been looking for something that does not exist.

The five reasons Google lists, and what each one means in practice

This is Google's list of what could not be fetched, in Google's order. In practice, the first two explain most of it.

1. Your robots.txt is blocking the sitemap itself.

"The sitemap is blocked by your robots.txt file. Google respects robots.txt when fetching sitemaps. You must remove the rule that prevents Google from fetching your sitemap."

This one catches people because a robots.txt rule is usually aimed at pages, and sitemaps get caught in it by accident. If your file says Disallow: / for every crawler, that includes /sitemap.xml. Google will not fetch a file you have told it not to fetch, and it will not make an exception because the file is called sitemap.

2. The site has a manual action.

"Your site has a manual action. Sitemaps are not read when a site has an unresolved manual action. Check the manual actions report."

"Sitemaps are not read when a site has an unresolved manual action." That is a sentence worth knowing exists, because the sitemap report will not explain it for you. If your site has a manual action, no amount of sitemap work matters. Check the manual actions report first, and clear it.

3. The URL you submitted is wrong.

"The URL provided for the sitemap is wrong (HTTP 404 error: sitemap not found at the address provided). Try visiting the URL in your browser to see if it exists."

Two versions of this happen constantly. One is a typo. The other is the property mix-up Google warns about separately: a sitemap belongs to the property you submitted it in, and http is not https and www is not the bare domain. Google says it plainly on the same page:

"If you're not seeing a sitemap that you expect to, make sure that you're not confusing http/https, or www/non-www properties."

4. A general error, including your server refusing Google.

"Some other general error: Sometimes other errors can prevent Google from retrieving a sitemap from your site, such as a server unavailability. Some of these errors can be transient: wait a bit and see if Google continues to encounter this error in later crawl attempts."

This is the one that got me, and it does not say what it covers. "Server unavailability" is the phrase, and it is broad enough to include a 401, a 403, a firewall rule, a login page, a country block, or a host that serves your file to you and refuses everyone else.

If your file is perfect and everything else on this list is clean, this is almost certainly your bucket, and the next section is how you catch it.

5. There is low crawl demand for the sitemap.

"There is low crawl demand for the sitemap. The higher the quality of the site's content, the higher the crawl demand."

That is Google saying, politely, that a thin site's sitemap goes to the back of the queue. Nothing to fix and nothing to check. What counts as quality is a separate question, and it is the one I picked apart in what Google means by low value content. It resolves with the site, not with the file.

The check that found it for me

Google publishes a procedure for this, and it takes about two minutes once you know where to look:

"Click the sitemap in the list to see details about the last fetch request." "Click the error name in the details page to see more details about the error." "Confirm the error in the URL Inspection tool. Copy the sitemap URL from the Sitemaps report details page. Paste the URL into the URL Inspection tool and press Enter. Click Live test in the URL Inspection tool. Expand the Page availability section to see why Google isn't able to fetch the sitemap. You can find information about all the fields here, but basically you want to see Crawl allowed? = "Yes", Page fetch = "Successful.""

Crawl allowed? and Page fetch. Those two fields in the live test answer the question, and they are on a screen most people never open for a sitemap, because they think URL Inspection is only for pages. It works for any URL, including your sitemap.

That is where my answer was. The page was fetching successfully for me and failing for Google, which is the whole signature of a gate rather than a broken file.

Opening the URL in your browser proves less than it feels like it proves

This is the trap my own reasoning fell into for two days.

Your browser is you. It arrives from your network, with your cookies, your session, and whatever you set up for yourself. When you type your sitemap URL and see a perfectly formatted list of URLs, you have proved that the file exists and that you can read it. That is all.

Google arrives as a stranger, from an address you have never seen, with no cookie, no login, and no exemption.

When those two disagree, the file is not the interesting one. Something between the internet and your file is treating strangers differently from you.

The things that do this, in the order I would check:

a password or basic auth on the whole site
a firewall rule or bot protection that dislikes an unknown visitor
a host holding the file behind a login during a build or migration
a maintenance mode that answers everyone except you
a country or IP rule
a CDN rule that serves your file and blocks the path for others

I wrote about the other half of that mistake, where a robots.txt rule hides a page it was never able to hide, in robots.txt is the wrong tool for that.

And the one that catches people last: your own robots tag or header. A noindex on the sitemap does not stop the fetch, but it does tell Google not to use it, which produces a different and equally confusing report.

What my own report said

I can show this one, because it happened to me and I still have the screens. The domain in the block is not the interesting part, so it is not here.

The report, before:

Sitemap          https://your-site.com/sitemap.xml
Type             Unknown
Submitted        the day it was submitted
Last read        (empty)
Status           Couldn't fetch
Discovered pages 0

Type Unknown. Last read empty. Zero discovered pages. Read that with the definitions above and it says one thing clearly: Google had not read the file. Not that it had read it and rejected it.

The file, at the same time:

GET https://your-site.com/sitemap.xml   200
Content-Type                            application/xml
valid XML, urlset, all URLs in it       yes
same result for a Googlebot user agent  yes

Every URL in it returning 200, all of them allowed by robots.txt. Nothing was wrong with the file, and that is not a contradiction. Both things were true at once, which is the whole lesson of this article.

What was actually wrong. For the whole time I was building the site, every page answered with a 401 and a password prompt. That is the right thing to do with an unfinished site, and it also means that when Google came looking, it was turned away at the door. The sitemap was inside the same gate. Google asked politely, got a login prompt, and wrote down that it could not fetch it.

Then I opened the site, resubmitted, and:

Status           Success
Type             Sitemap
Last read        the same day
Discovered pages 13

Thirteen discovered pages inside a day, from a file that had not changed by a single byte.

The file never moved. The report did. Those two facts sitting together are the most useful thing on this page.

What a sitemap does not do

Worth saying, because the status report makes it feel more powerful than it is.

It does not request indexing. Google's own description of the Success status is "All URLs will be queued for crawling", and queued is the right word. What happens to a page after it lands in that queue is its own subject, and I went through every status that queue produces.

It does not guarantee a crawl. From the documentation:

"Keep in mind that submitting a sitemap is merely a hint: it doesn't guarantee that Google will download the sitemap or use the sitemap for crawling URLs on the site."

And the number it shows you is not indexed pages. Google's report page is explicit about what Discovered pages counts:

"The number of page URLs that were parsed from the sitemap. If this is a sitemap index, the number is the count of all URLs in all child sitemaps. Duplicate URLs are counted only once. There is no guarantee that a page URL discovered in a sitemap has been or will be crawled or indexed by Google."

So 20 discovered pages means 20 URLs were read out of the file. Not 20 pages indexed. I have two numbers that disagree with each other on purpose, and neither of them is a fault.

One more, if you run more than one site. The report only lists sitemaps you submitted through it:

"This report shows only sitemaps that were submitted using this report or the API. It does not show any sitemaps discovered through a robots.txt reference or other discovery methods."

If you added your sitemap to robots.txt and never submitted it here, an empty report is correct and your sitemap may be working fine.

What to do, in order

1  Open the sitemap in the report and expand the error. Read the specific reason
   instead of the two-word status.
2  Check the manual actions report. If something is there, nothing else on this
   list matters until it is cleared.
3  Open your robots.txt and read it as if you were a stranger. Is /sitemap.xml
   inside a Disallow that was meant for something else?
4  Check the property. Same host, same scheme, www or not, exactly as submitted.
5  Run the live test on the sitemap URL, not on a page. Read Crawl allowed? and
   Page fetch.
6  If page fetch succeeds for you and fails for Google, stop looking at the file.
   Look at what surrounds it: the gate, the firewall, the host, the CDN.
7  Fix that, then resubmit. Nothing else needs to change, and the file you already
   have will start working.
8  Then leave it alone. "If a sitemap cannot be fetched after several attempts,
   Google will eventually stop trying to read that sitemap", which is a reason to
   fix and resubmit once, not to resubmit daily.

The one thing I would say to someone stuck on this

The label is about the fetch, not the file. Everything you do to the XML while the status says Couldn't fetch is work on a file that nobody has read.

There is a version of this story where the file really is broken. You get Has errors, a list, and a fix. When your status is Couldn't fetch with an empty Last read date, the file is not the suspect, and the fastest way to find the real cause is to make Google the stranger at your door and watch what happens to it.

Sources

  • Search Console Help, Sitemaps report
  • Google Search documentation, Build and submit a sitemap
  • Google Search documentation, Manage your sitemaps with a sitemap index file
  • A first-hand Search Console Sitemaps report, read alongside the matching server access logs