![]() |
Robots.txt is very important txt file of our website. It is help to us to crawl the specific page of our website to be crawl or not.
|
Robot.txt is very important page.
|
It BLOCKS permission to crawl certain areas. zyxwvu was right about that
It is a simple text file that goes in the "root" folder of your website, the same folder that contains index.html robots.txt has no importance in SEO |
Robots.txt file is a very important file if you want to have a good ranking on search engines, many websites don't offer this file. A Robots.txt file is helpful to keep out unwanted search engine spiders like email retrievers, image strippers, etc. It defines which paths are off limits for spiders to visit. This is useful if you want to hide some personal information or some secret files.
|
User-Agent: *
Disallow: |
will it help if your only indexing your home page? thats where all my anchor text points to.
|
All times are GMT -7. The time now is 10:56 AM. |
Powered by vBulletin Copyright © 2020 vBulletin Solutions, Inc.