Usually it is a discovery problem, not a quality problem. Search engines find pages by following links. A page with no internal links pointing at it can sit unseen for months. An XML sitemap hands crawlers a plain list of every URL you want found, so discovery stops depending on luck.
We see this on almost every site we take over. The blog is fine. The main service pages are fine. But a handful of pages sit in a corner with nothing linking to them, and nobody at the company knows why those pages never show up in Google.
A sitemap will not fix thin content or a slow site. It fixes the much smaller problem of search engines not knowing a page exists. That problem is common, cheap to solve, and surprisingly easy to get wrong.
An XML sitemap is a file that lists the URLs on your site you want search engines to know about. It follows a shared format called the Sitemap protocol, published at sitemaps.org. Each entry holds one address and, if you want, the date that page last changed.
The format has been stable for years. Sitemaps.org lists the current protocol version as 0.9, with the specification last updated in November 2016. That is good news for you. A sitemap built correctly today will still be valid years from now.
Google supports four formats: XML, RSS or Atom feeds, plain text files, and sitemap index files. Google Search Central calls XML "the most versatile of the sitemaps formats". We use XML on every project because it is the only one that carries the extra data cleanly.
The file normally sits at the root of your domain, so yoursite.com/sitemap.xml. That is a convention rather than a rule. What actually matters is that search engines can reach the file and that you tell them where it lives.
Not every site does. Google Search Central says you may not need one if your site is small, if it is linked well internally, and if you have no media or news content. Google defines small in plain numbers: "about 500 pages or fewer."
We still add one to nearly every build. The cost is close to zero, and the value shows up the moment a site grows past what its own navigation can reach. Sites almost never shrink, so we would rather have the file in place early.
Google is honest about the ceiling, too. The documentation states that "a sitemap helps search engines discover URLs on your site, but it doesn't guarantee that all the items in your sitemap will be crawled and indexed." Anyone selling a sitemap as an indexing guarantee is overselling it.
The sites that gain most are large catalogs, brand new sites with few backlinks, and sites where important pages sit deep in the structure. If your product pages are five clicks from the homepage, a sitemap is doing real work. We looked at how crawlers spend their time in our guide to whether crawl budget actually matters, and a clean sitemap is one of the few levers that reliably helps.
Three tags are required by the Sitemap protocol. The urlset tag wraps the whole file, the url tag wraps each entry, and the loc tag holds the actual address. Sitemaps.org sets one hard limit there: a URL inside a loc tag must stay under 2,048 characters.
Then come three optional tags, and this is where most sites go wrong. The lastmod tag records when the page changed. The changefreq tag suggests how often it changes. The priority tag rates a page from 0.0 to 1.0, with 0.5 as the default value.
Sitemaps.org is blunt about how much those hints count for. On changefreq it says "the value of this tag is considered a hint and not a command." On priority it warns that "assigning a high priority to all of the URLs on your site is not likely to help you."
Our rule is simple. Always include loc. Include lastmod only when it is accurate. Skip changefreq and priority. Sitemaps.org states that the lastmod date "must be set to the date the linked page was last modified, not when the sitemap is generated," and plenty of plugins get that exactly backwards.
Fifty thousand. Google Search Central states that "all formats limit a single sitemap to 50MB (uncompressed) or 50,000 URLs." Whichever limit you reach first is the one that stops you. Most sites hit the URL count long before they get near the file size.
Once you pass either limit, you split the file and create a sitemap index. An index is just a wrapper that lists your other sitemaps. Google lets you submit that single index file instead of every child file, which keeps the setup tidy as a site grows.
We split sitemaps by content type even on small sites, so pages, blog posts, and case studies each get their own file. It costs nothing and it makes diagnosis much faster. When a report says 40 URLs are not indexed, you want to see straight away whether they are all blog posts.
Two ways, and you should use both. Add a Sitemap line to your robots.txt file pointing at the file URL. Then submit the same URL inside Google Search Console and Bing Webmaster Tools. Submitting is a signal to crawlers, not an order they must obey.
Google says so directly. Its documentation notes that "submitting a sitemap is merely a hint: it doesn't guarantee that Google will download the sitemap or use the sitemap for crawling URLs on the site." Think of submission as opening a door, not as pushing crawlers through it.
One old habit needs to go. Google's Search Central blog published a post in June 2023 titled "Sitemaps ping endpoint is going away". If your deploy script still pings that endpoint after every publish, it is burning a request for nothing. We check for this on every site we inherit.
For faster notice on other engines, look at IndexNow. Its FAQ lists Bing, Yandex, Naver, Seznam.cz, Amazon, and Yep as participating engines, and it allows up to 10,000 URLs per POST request. Google does not participate, so IndexNow adds to your sitemap rather than replacing it. We walk through the reports worth watching in our guide to using Google Search Console to improve your rankings.
The most common failure is a sitemap that disagrees with the rest of your site. It lists a URL that redirects somewhere else, or a page marked noindex, or a page whose canonical tag points elsewhere. Search engines then receive two conflicting instructions from the same website.
Fix the conflict, not the symptom. A sitemap should list only final, indexable URLs that point at themselves as canonical. If a page redirects, list the destination instead. If a page is set to noindex, keep it out of the file entirely. We explain that tag in our piece on what canonical URLs do about duplicate content.
The second common failure is a stale file. A sitemap generated once by hand and saved into a repository slowly drifts away from reality. Ours regenerates on every build, which is the only setup we trust on a site that publishes new work often.
The third is scale creep. A site crosses the 50,000 URL limit and the generator quietly truncates the list instead of splitting it. Watch your entry count as the site grows, especially on ecommerce catalogs and pages built from a database.
Indirectly, yes. Tools like ChatGPT, Perplexity, and Google AI Overviews mostly draw on pages that are already discovered and indexed. A page nothing has crawled cannot be quoted back to anyone. A sitemap does not earn you the citation, but it removes the first reason you might miss it.
There is a second benefit people overlook. An accurate lastmod date signals freshness, and answer engines lean toward recent sources on fast moving topics. A file that honestly reports when each page changed is quietly doing useful work for you.
What a sitemap cannot do is make your content readable. If your article text only appears after JavaScript runs, a crawler that does not execute scripts sees an empty shell no matter how perfect your sitemap is. That is why we prerender page content into static HTML on the sites we build.
Webflow builds the file for you. According to Webflow's Help Center, you turn on the auto-generated sitemap in Project settings under the SEO tab, and Webflow then regenerates sitemap.xml every time you publish the site. No plugin and no build step required.
By default that file includes every live page on the site. You remove a page from it using the sitemap indexing toggle in that page's settings, which is the correct way to keep thank you pages and internal utility pages out of search results.
Webflow's Help Center also notes that on sites using Localize, the auto-generated sitemap includes hreflang tags for static and dynamic pages, so search engines can see the language and region versions of your content. That saves real hours on a multilingual build.
The one trade-off is control. If you need a custom structure, split files, or entries the Designer will not produce, you switch to a custom sitemap and take on the job of keeping it accurate forever. We only make that switch when a site genuinely needs it.
If you have never opened your own sitemap, yes. Load your domain followed by /sitemap.xml, read the list, and ask one question: would you happily send a customer to every URL in here? Plenty of sites fail that test in under a minute.
It is a small job with a clean payoff. You are not chasing a ranking trick. You are making sure the pages you paid to build are actually visible to the systems that send you customers, whether that is Google, Bing, or an AI assistant answering a question about your industry.
If you want a second pair of eyes on your sitemap, your indexing reports, or the rest of your technical setup, we are happy to walk through it with you. Let's talk. Come find us at phoenix.studio and tell us what you are seeing.
Tell us where you want to go. We'll tell you how we'd get you there.