What Is A Robots.txt File?
 

Hi Friend,
Please can tell me about something about robots.txt file?

Robots.txt is a text file webmasters create to instruct robots how to crawl and index pages on their website.

yes by robots.txt file we can set permission which page is search by robots and which are not.

If your file are directories where you should not go then you can use robot txt file.

The robots.txt is a very simple text file that is placed on your root directory. An example would bewww.yourbdomain.com robots.txt. This file tells search engine and other robots which areas of your site they are allowed to visit and index.

Its Txt file by which you can tell search engine that what area of your site is allowed to crawl or store in a database or index.