robots meta tag - Site Owners Forums

shanken · 09-25-2015, 12:40 AM

Quote:

Originally Posted by manpreetkaur

The robots meta tag lets you utilize a granular, page-specific approach to controlling how an individual page should be indexed and served to users in search results. Place the robots meta tag in the <head> section of a given page, like this:

<!DOCTYPE html>
<html><head>
<meta name="robots" content="noindex" />
(…)
</head>
<body>(…)</body>
</html>
The robots meta tag in the above example instructs all search engine not to show the page in search results. The value of the name attribute (robots) specifies that the directive applies to all crawlers. To address a specific crawler, replace the robots value of the name attribute with the name of the crawler that you are addressing. Specific crawlers are also known as user-agents (a crawler uses its user-agent to request a page.) Google's standard web crawler has the user-agent name Googlebot. To prevent only Googlebot from crawling your page, update the tag as follows:

<meta name="googlebot" content="noindex" />
This tag now instructs Google (but no other search engines) not to show this page in its web search results. Both the name and the content attributes are non-case sensitive.

Search engines may have different crawlers for different properties or purposes. See the complete list of Google's crawlers. For example, to show a page in Google's web search results, but not in Google News, use the following meta tag:

<meta name="googlebot-news" content="noindex" />
If you need to specify multiple crawlers individually, it's okay to use multiple robots meta tags:

<meta name="googlebot" content="noindex">
<meta name="googlebot-news" content="nosnippet">
If competing directives are encountered by our crawlers we will use the most restrictive directive we find.

Best answer of the whole lot.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
The Sticky for Meta Description, keywords, keyword density, and title tags	sinicline	Search Engine Optimization	2	01-06-2017 10:22 AM
Check the website robots file	china-jewels	Bing	4	10-11-2015 08:14 AM
If the meta robots tag has a value of "no index, no follow" what does it mean?	poojaseo	Search Engine Optimization	0	06-06-2014 03:08 AM
How to add a robots follow meta tag?	jamesranatte	Search Engine Optimization	1	10-11-2012 10:24 PM
What is Robots meta tag?	samlko	Search Engine Optimization	2	04-06-2012 12:10 AM

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)