12-19-2016, 01:26 AM
|
#15 |
|
Registered User
Join Date: Sep 2016
Posts: 206
|
Robots.txt is a text file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Threaded Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Check the website robots file | china-jewels | Bing | 4 | 10-11-2015 08:14 AM |
| Uses of robots? | jamesranatte | General Discussion | 2 | 12-11-2014 08:32 AM |
| Robots exclusion protocols? | jamesranatte | General Discussion | 7 | 12-26-2013 12:01 AM |
| what is difference between spider, crawler and robots? | noahwilson | Search Engine Optimization | 0 | 05-30-2013 03:04 AM |
| What is Robots meta tag? | samlko | Search Engine Optimization | 2 | 04-06-2012 12:10 AM |
All times are GMT -7. The time now is 07:49 PM.