11-20-2015, 03:47 AM
|
#5 |
|
Registered User
Join Date: Aug 2015
Posts: 434
|
Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol. The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Threaded Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Computer File Name Extensions | johnsoncla | Social Networks | 2 | 03-19-2015 03:09 AM |
| What do you mean by Robot. txt file? | davidwrenint | Search Engine Optimization | 2 | 08-25-2014 01:32 AM |
| telepresence robot | hemnchy | General Discussion | 1 | 01-02-2014 04:35 AM |
| RSS Feed | rajivwebads | Search Engine Optimization | 12 | 08-31-2012 07:15 AM |
| include file using html | ncjbhd | HTML / DHTML | 0 | 10-17-2011 12:21 AM |
All times are GMT -7. The time now is 01:30 AM.