|
|
04-22-2014, 03:29 AM
|
#1 |
|
Registered User
Join Date: Feb 2014
Location: india
Posts: 268
|
Crawlers, spider & robots.txt?
By default robots.txt is in port 80. There is any way to verify in case robots.txt it's not in port 80 to get it from any other port it may be. Thanks.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
 |
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Hybrid Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| What is robots.txt? | seoinheritx | Search Engine Optimization | 29 | 09-21-2022 11:57 PM |
| Evil bots consuming all my sites traffic even after i disallowed them in robots.txt | Fking | General Discussion | 1 | 12-13-2013 11:27 PM |
| Difference between Spider and Crawlers ? | noahwilson | Search Engine Optimization | 3 | 06-17-2013 07:04 PM |
| How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
| Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
All times are GMT -7. The time now is 11:57 AM.