A sitemap is only useful if search engines can trust it. A clean file lists the right pages, in the right form, with honest dates. Here is what to include, what to leave out, and the mistakes that show up most often in Search Console.

Think of your sitemap as a list of recommendations. Every URL in it says "this page is worth indexing". If a third of the list turns out to be redirects, errors or duplicates, search engines learn that your recommendations are unreliable. A clean sitemap contains:
rel="canonical" tag, list that other address, not this one.noindex tag has no place in the sitemap.?sort=price or ?sessionid=, which show the same content as the main page.Google has said that it uses lastmod when the dates are consistently accurate, and ignores changefreq and priority. Change the date only when the main content changes: a new paragraph, an updated price, a corrected fact. A new copyright year in the footer or a changed sidebar does not count. If you cannot produce reliable dates, leave the tag out rather than filling it with today's date.
An old sitemap slowly fills up with deleted pages and misses the new ones. Generate it again whenever you publish a batch of pages, delete a section, or change your URL structure. On a site that changes every day, the sitemap should be produced automatically by the publishing system. On a site that changes a few times a month, regenerating it with a crawler and uploading it again is enough.
| Point to check | Expected value |
|---|---|
| Encoding | UTF-8, with special characters escaped (&) |
| URLs | Absolute, same protocol and same host as the site |
| Size | 50,000 URLs and 50 MB maximum per file |
| HTTP response of the file | 200, readable without a login, not blocked by robots.txt |
| Location | At the root, declared in robots.txt |
| Compression | Optional, gzip accepted (.xml.gz) |
A sitemap helps search engines find pages, but it does not tell them a page is important. Internal links do that. A page that appears in the sitemap but receives no link from the rest of the site will often be crawled rarely and ranked poorly. Use the sitemap as a safety net, and make sure your menus and your content link to the pages that matter.
sitemapauto.com applies most of these rules for you: it follows your internal links, stays on your own domain, and keeps out every URL that answers with an error. You also get a report of the broken links it found along the way.
No. Only the pages you want to see in search results: canonical, indexable and working pages.
No. Search engines do not give more weight to the first URLs of the file.
Not as separate URLs. Images can be described with the image sitemap extension, inside the entry of the page that displays them.
It will not cause a penalty, but a sitemap full of errors wastes crawl time and makes search engines trust your file less.
Another free tool we built
Bye404.org automatically scans a website and reports every dead link and 404 page it finds, broken images included. Broken links frustrate your visitors and waste crawl budget, so they are worth fixing before search engines run into them.