|
|
01-03-2017, 10:48 PM
|
#1 |
|
Registered User
Join Date: Nov 2016
Location: USA
Posts: 32
|
The robots exclusion customary, conjointly referred to as the robots exclusion protocol or just robots.txt, could be a customary employed by netsites to speak with net crawlers and different web robots. the quality specifies the way to inform the net automaton regarding that areas of the web site mustn't be processed or scanned.
Click here for more details about how to bring users back to your app
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
 |
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Hybrid Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Robots.txt | jaysh4922 | Search Engine Optimization | 13 | 07-28-2016 05:10 AM |
| Crawlers, spider & robots.txt? | jackthomas087 | Search Engine Optimization | 0 | 04-22-2014 03:29 AM |
| Evil bots consuming all my sites traffic even after i disallowed them in robots.txt | Fking | General Discussion | 1 | 12-13-2013 11:27 PM |
| How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
| Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
All times are GMT -7. The time now is 03:35 AM.