Thread: What’s the significance of Robots.txt file in a website?
View Single Post
Old 05-10-2017, 02:09 AM   #3
markb
Registered User
 
Join Date: Apr 2017
Posts: 32
robots.txt file is used to give instructions about the site to web robots to crawl which files .
markb is offline   Reply With Quote