Site Owners Forums - Webmaster Forums - View Single Post - What is robot.txt file and what are the benefits of using it?

williamstinner · 04-11-2016, 10:51 PM

Robots.txt file is used to give instruction to the search engine crawler to which page has to crawled or not by allowing or disallowing.

04-11-2016, 10:51 PM	#14
williamstinner Registered User Join Date: Mar 2016 Posts: 20	Robots.txt file is used to give instruction to the search engine crawler to which page has to crawled or not by allowing or disallowing.