07-27-2012, 02:51 AM
|
#2 |
|
Registered User
Join Date: Mar 2012
Posts: 74
|
Hello
Robots.txt is the file which we used to block the content from crawler so that search engine will not crawled and index our data which we mention in robots file. Mainly the confidential data which we want to hide block with the help of robots file. Hope you understand. |
|
|
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Threaded Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Robots.txt | sameerseo | Search Engine Optimization | 26 | 09-28-2012 04:18 AM |
| issue in robots.txt file | davikerkrish | Search Engine Optimization | 0 | 07-20-2012 12:40 AM |
| Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
| What Is Robots.txt? | samlko | Search Engine Optimization | 13 | 03-09-2012 02:37 PM |
| sitemap.xml and robots.txt? | jamesranatte | Search Engine Optimization | 5 | 01-31-2012 11:16 PM |
All times are GMT -7. The time now is 06:09 AM.