Thread: What is the use of Robots.txt?
View Single Post
Old 02-17-2016, 11:26 PM   #7
mariajerek
Registered User
 
Join Date: Feb 2016
Posts: 35
Quote:
Originally Posted by Manishkumar12 View Post
Hello friends,

What is the use of Robots.txt?
Robots.txt file is used to instruct crawler which page has crawl or which page not. You can block a whole folder through this file.
mariajerek is offline   Reply With Quote