Welcome to the Treehouse Community

Want to collaborate on code errors? Have bugs you need feedback on? Looking for an extra set of eyes on your latest project? Get support with fellow developers, designers, and programmers of all backgrounds and skill levels here with the Treehouse Community! While you're at it, check out some resources Treehouse students have shared here.

Looking to learn something new?

Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and join thousands of Treehouse students and alumni in the community today.

Start your free trial

HTML

Posted by adamapperson
adamapperson
4,375 Points

Appropriate Usage of the Robots <meta> Tag

I'm trying to develop a better understanding of the different meta tags and their values. I understand that the Robots tag allows us to specifically disallow the indexing of pages, spidering of images, etc.. I guess what I don't have total clarity on is where and why we wouldn't want our site to be indexed, or links to be followed by robots. Can anyone provide some insight? Maybe different examples?

1 Answer

Paul Hubbard
Paul Hubbard
5,818 Points
Paul Hubbard
Paul Hubbard
5,818 Points

Basically, I feel it's more about control of what you want the search engine show when your website is up. I've found a couple of links that might be able to explain everything and hopefully make things clearer for you.

This explains robot.txt which is similar to <meta robot> and it has a link to Wikipedia which explains the origin and reason this exists. https://developer.mozilla.org/en-US/docs/Glossary/Robots.txt

This explains a few different meta tags in depth, and describes what they are and what they are used for. https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meta

I hope this helps!

Paul Hubbard
Paul Hubbard
5,818 Points

similar to the meta robot tag*