Meta Robots Tag & X-Robots-Tag Explained

Robots meta tags tell search engine bots how to crawl, index, and serve your site’s pages in the SERPs.