Site Owners Forums - Webmaster Forums - View Single Post

uktweedjackets · 09-21-2017, 06:07 AM

Hello Friends,

The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots.

09-21-2017, 06:07 AM	#48
uktweedjackets Registered User Join Date: Aug 2017 Posts: 6	Hello Friends, The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots.