03-27-2017, 02:17 AM
|
#9 |
|
Registered User
Join Date: Nov 2016
Posts: 412
|
The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.
You might be surprised to hear that one small text file, known as robots.txt, could be the downfall of your website. If you get the file wrong you could end up telling search engine robots not to crawl your site, meaning your web pages won’t appear in the search results. Therefore, it’s important that you understand the purpose of a robots.txt file and learn how to check you’re using it correctly.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
| Thread Tools | |
| Display Modes | Rate This Thread |
Threaded Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| What is robot.txt file and what are the benefits of using it? | Martinricky | Social Networks | 31 | 12-21-2016 02:46 AM |
| Which pages should be put in robot.txt file? | Martinricky | Search Engine Optimization | 1 | 07-24-2016 11:15 PM |
| Robot.txt? | siva SEO | Search Engine Optimization | 17 | 01-09-2015 07:02 PM |
| how to use robot.txt? | dean1122 | Search Engine Optimization | 11 | 01-05-2013 02:10 AM |
All times are GMT -7. The time now is 11:51 AM.