![]() |
#3 |
Registered User
Join Date: Nov 2012
Posts: 5
|
Robots.txt is a text file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.
|
![]() |
![]() |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
Evil bots consuming all my sites traffic even after i disallowed them in robots.txt | Fking | General Discussion | 1 | 12-13-2013 11:27 PM |
Robots.txt | sameerseo | Search Engine Optimization | 26 | 09-28-2012 04:18 AM |
robots.txt | wheelbarrow001 | 2 | 07-27-2012 04:10 AM | |
Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
What Is Robots.txt? | samlko | Search Engine Optimization | 13 | 03-09-2012 02:37 PM |