Site Owners Forums - Webmaster Forums - View Single Post

accutranskpo · 03-22-2016, 04:37 AM

robots.txt is a text file which is used by web-master to give crawling instructions to the search engine crawlers which page is to be crawl and which page is not.

03-22-2016, 04:37 AM	#10
accutranskpo Registered User Join Date: Mar 2016 Posts: 15	robots.txt is a text file which is used by web-master to give crawling instructions to the search engine crawlers which page is to be crawl and which page is not.