robot.txt is a file which specifies a search engine bot, if it has to follow your or index your blog or website links or not...

Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.

It works likes this: a robot wants to vists a Web site URL, say http://www.example.com/welcome.html. Before it does so, it firsts checks for http://www.example.com/robots.txt, and finds:

Robots.txt help to block the crawler to crawl the web page which we mentioned those page URL on robots tag.

A Robots.txt file can control the Search Engine bots (eg. Google, bing, Yahoo, Ask, Yandex). If you want to prevent any directory/content of your website from indexing into search. Then using robots.txt you can block those content or directory.

Robots.txt is a text file in your web hosting files. Search engine see first, which page crawl or which not

In Robots.txt file you can allow or disallow the page you want.
Example:
User-agent : *
Disallow: /
Allow: /

Put your URL in Disallow and allow to crawl or not crawl your page

The Robots exclusion protocol(REP) or robots.txt is a text file, search engine spider first see that which page crawl or which not.

Robot.txt is the text file that helps the search engine crawler to check your website and its sub pages whether it is good or not. Primarily robot.txt helps the search engine crawlers to index the pages of a website quickly and easily. It is the important factor which a website must contain in it for getting good rank in different search engines.

here are two important considerations when using /robots.txt:

1- robots can ignore your /robots.txt. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.
2-the /robots.txt file is a publicly available file. Anyone can see what sections of your server you don't want robots to use

Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.

To Get More info - SEO Hats | SEO Updates

Robots.txt file will helps to block all web crawlers from all web pages or specific web crawlers from specific web pages.

robots.txt is a text file which is used by web-master to give crawling instructions to the search engine crawlers which page is to be crawl and which page is not be crawled

Robots.txt file help you to which web url will show on search engine or which not

robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.