Site Owners Forums - Webmaster Forums - View Single Post

laceywilliams12 · 12-05-2012, 10:12 PM

Robot.txt is a text file which have set of instructions for crawlers ot crawl the website or a specific page. If you have it then it must be placed in root directory.....and for more help you can read lots Articles related to this...thanks

12-05-2012, 10:12 PM	#4
laceywilliams12 Registered User Join Date: Nov 2012 Posts: 71	Robot.txt is a text file which have set of instructions for crawlers ot crawl the website or a specific page. If you have it then it must be placed in root directory.....and for more help you can read lots Articles related to this...thanks