Thread: What is robots.txt?
View Single Post
Old 05-30-2017, 11:18 PM   #17
Shane Bentick
Registered User
 
Join Date: Apr 2017
Posts: 355
Robots.txt is a text file webmasters create;to instruct web robots how to crawl pages on their website.
Shane Bentick is offline   Reply With Quote