![]() |
#7 |
Registered User
Join Date: Jan 2012
Posts: 177
|
Robots.txt will tell the search engine what to crawl and what not to. This helps if you do not want search engines to crawl and index some of your private data.
|
![]() |
![]() |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
sitemap.xml and robots.txt? | jamesranatte | Search Engine Optimization | 5 | 01-31-2012 11:16 PM |