Google indexing guide for Magento
A powerful e-commerce platform (Adobe Commerce) for large catalogs. Layered navigation filters create many parametrised URLs.
Contents
Where are the settings?
Stores > Configuration > Catalog > Catalog > Search Engine Optimization and Marketing > SEO & Search.
Settings that affect indexing
- noindex and visibility: Under Stores > Configuration > General > Design > Search Engine Robots the default should be INDEX,FOLLOW. A NOINDEX,NOFOLLOW value left over from development is a common live mistake.
- robots.txt: Edit robots.txt on the same screen. Disallowing filter parameters, search results and cart URLs protects crawl budget.
- canonical and duplicate URLs: In Catalog > SEO turn on "Use Canonical Link Meta Tag" for categories and products. Dropping the category path from product URLs reduces duplicates.
- redirects and 404: Add 301s under Marketing > URL Rewrites. When a product URL key changes, keep the "create permanent redirect" box ticked.
- sitemap: Create the sitemap under Marketing > SEO & Search > Site Map and add it to robots.txt. Tie generation to cron to keep it current.
- speed and crawling: Enable full page cache and Varnish and turn on JS and CSS merge/minify. Slow catalog pages reduce crawl frequency.
Common indexing problems on Magento
Open the matching page for steps specific to each status:
- Crawled - currently not indexed
- Discovered - currently not indexed
- Duplicate without user-selected canonical
- Duplicate, Google chose different canonical than user
- Alternate page with proper canonical tag
- Page with redirect
- Soft 404
- Not found (404)
- Blocked by robots.txt
- Excluded by 'noindex' tag
- Server error (5xx)
- Redirect error
- Indexed, though blocked by robots.txt
- Blocked due to unauthorized request (401) / access forbidden (403)
- URL is unknown to Google
- Indexed, not submitted in sitemap
Next steps
Before publishing, run through Google indexing checklist: 15 items before you publish, scan the address with the free Free indexability checker tool and add new pages to the sitemap: How to build a sitemap and submit it to Search Console. For many pages, Google index panel: send, track and resend from one screen makes bulk submission easier; indexing is never guaranteed.
Frequently asked questions
Why is my Magento site not showing on Google?
Usually because of a noindex setting, a robots.txt block, a missing sitemap or weak content. Check the settings above and the matching status pages in order.
What is the sitemap address for Magento?
Create the sitemap under Marketing > SEO & Search > Site Map and add it to robots.txt. Tie generation to cron to keep it current.