Fixes

How to fix “Blocked by robots.txt” on WordPress

If Search Console shows “Blocked by robots.txt” on your WordPress site, you are in the right place. robots.txt stops Googlebot from fetching the URL. This page covers the causes and a step-by-step fix for WordPress. The most widely used open-source CMS. Most indexing controls live in an SEO plugin such as Yoast SEO, Rank Math or All in One SEO.

Updated: 08.11.2026 · 2 min read

Contents
  1. What does this status mean?
  2. Likely causes on WordPress
  3. Step-by-step fix on WordPress
  4. Extra checks for WordPress
  5. After fixing
  6. More information

What does this status mean?

A Disallow rule in your robots.txt forbids crawling this URL. Google cannot read the content; if other sites link to it, it can still appear in the index without content.

Likely causes on WordPress

  • A "Disallow: /" rule added during development was forgotten live.
  • A broad pattern (for example /*?) also catches pages you want.
  • The CMS or a plugin rewrote robots.txt on its own.
  • Sitemap URLs sit in folders closed by robots.txt.
  • WordPress note: WordPress serves a virtual robots.txt; edit it in your SEO plugin or replace it with a real file in the web root. Review any Disallow lines beyond /wp-admin.

Step-by-step fix on WordPress

  1. In your WordPress admin go to: Settings > Reading and the post or page box of your SEO plugin.
  2. WordPress serves a virtual robots.txt; edit it in your SEO plugin or replace it with a real file in the web root. Review any Disallow lines beyond /wp-admin.
  3. Use the Search Console robots.txt report or a robots tester to find which rule closes the URL.
  4. Narrow or delete the rule; close only folders that really should not be crawled.
  5. Verify that sitemap URLs do not conflict with robots.txt.
  6. After fixing, give Google a day to refresh its robots.txt cache or request a recrawl in the report.
  7. Send the URL for indexing and follow progress with URL inspection.

Extra checks for WordPress

  • speed and crawling: A caching plugin, image compression and decent hosting raise crawl speed. Remove unused plugins; a slow server makes Googlebot crawl less.
  • noindex and visibility: Make sure "Discourage search engines from indexing this site" under Settings > Reading is off; for a single post check the Advanced tab of your SEO plugin.

After fixing

In Search Console start "Validate fix" for this status and resubmit the page with URL inspection. First confirm the blocker is gone with the free Free indexability checker tool; for many URLs follow Bulk URL indexing: how to send thousands of pages to Google. Google makes the indexing decision itself and timing ranges from days to weeks.

More information

Frequently asked questions

Does robots.txt remove a page from the index?

No. It blocks crawling, it does not remove from the index. To remove, allow crawling and use noindex.

How do I test robots.txt?

Enter the file and a URL in our free robots.txt tester to see the Googlebot allow status.

Where are these settings on WordPress?

Settings > Reading and the post or page box of your SEO plugin.

Related pages