Welcome to the Treehouse Community

The Treehouse Community is a meeting place for developers, designers, and programmers of all backgrounds and skill levels to get support. Collaborate here on code errors or bugs that you need feedback on, or asking for an extra set of eyes on your latest project. Join thousands of Treehouse students and alumni in the community today. (Note: Only Treehouse students can comment or ask questions, but non-students are welcome to browse our conversations.)

Looking to learn something new?

Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and a supportive community. Start your free trial today.

WordPress

Ricardo Sala
Ricardo Sala
13,390 Points

Cannot change robots.txt - My site is "searchers blocked"! :(

Good morning tree team! Hope you have a wonderfull weekend!

I just launch my wordpress website ( https://aceitesyesencias.com ) but wanted to submit it to Google Search Console.

The website has been a while in "developing mode" and so I was blocking access to searchs with the Wordpress option: "Prevent Search Engines from Indexing..." (something like that) in the settings menu and BEFORE A REAL ROBOTS.TXT WAS ACTUALLY CREATED (I think that is important for some reason...).

The think is that today I unchecked the option to prevent search engines, and created a Robots.txt file with Yoast, which creates the basic lines:

"User-agent: * Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php"

I then went to Google to submit the site and...

...oops! Seems that some "other" robots.txt file is blocking my site!

In fact, if I go to https://aceitesyesencias.com/robots.txt the file that is there is NOT my actual file...it says:

"User-agent: * Disallow: /"

I red that Wordpress uses something like a virtual robots.txt (which I guess it is what has been working in my site until today), and somehow the "real one" is not overriding the virtual one, and so the virtual one keeps blocking my site...

Coud you please help me with this?

How could I say wordpress something like... "Hey, serve the real file not the virtual one"..?

I would really appreciate your help :)

1 Answer

Hello,

Maybe you having the option "Search Engine Visibility" active. You can change that on Settings > Reading and uncheck the option.