Welcome to the Treehouse Community
Looking to learn something new?
Ricardo Sala13,390 Points
Cannot change robots.txt - My site is "searchers blocked"! :(
Good morning tree team! Hope you have a wonderfull weekend!
I just launch my wordpress website ( https://aceitesyesencias.com ) but wanted to submit it to Google Search Console.
The website has been a while in "developing mode" and so I was blocking access to searchs with the Wordpress option: "Prevent Search Engines from Indexing..." (something like that) in the settings menu and BEFORE A REAL ROBOTS.TXT WAS ACTUALLY CREATED (I think that is important for some reason...).
The think is that today I unchecked the option to prevent search engines, and created a Robots.txt file with Yoast, which creates the basic lines:
"User-agent: * Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php"
I then went to Google to submit the site and...
...oops! Seems that some "other" robots.txt file is blocking my site!
In fact, if I go to https://aceitesyesencias.com/robots.txt the file that is there is NOT my actual file...it says:
"User-agent: * Disallow: /"
I red that Wordpress uses something like a virtual robots.txt (which I guess it is what has been working in my site until today), and somehow the "real one" is not overriding the virtual one, and so the virtual one keeps blocking my site...
Coud you please help me with this?
How could I say wordpress something like... "Hey, serve the real file not the virtual one"..?
I would really appreciate your help :)
Maybe you having the option "Search Engine Visibility" active. You can change that on Settings > Reading and uncheck the option.