|
What is Robots.txt File?
A Robots.txt file is an automatically generated by
Blogger for giving details information to search Robot about Blog posts. As a
result when Search Robots from various search engines like Google, Bing, Yahoo,
Yandex etc. visits your site then Robots.txt tells search crawler what to crawl
and what not to crawl.
|