07-03-2014, 03:32 AM
|
#1 |
|
Registered User
Join Date: Jun 2014
Location: San Antonio
Posts: 67
|
What Is A Robots.txt File?
Hi Friend,
Please can tell me about something about robots.txt file? |
|
|
|
07-03-2014, 11:13 PM
|
#2 |
|
Registered User
Join Date: Jun 2014
Posts: 274
|
Robots.txt is a text file webmasters create to instruct robots how to crawl and index pages on their website.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
07-03-2014, 11:27 PM
|
#3 |
|
Registered User
Join Date: Feb 2014
Posts: 541
|
yes by robots.txt file we can set permission which page is search by robots and which are not.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
07-04-2014, 01:35 AM
|
#4 |
|
Registered User
Join Date: Jul 2014
Location: United Kingdom
Posts: 33
|
If your file are directories where you should not go then you can use robot txt file.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
07-04-2014, 04:11 AM
|
#5 |
|
Registered User
Join Date: Dec 2013
Posts: 146
|
The robots.txt is a very simple text file that is placed on your root directory. An example would bewww.yourbdomain.com robots.txt. This file tells search engine and other robots which areas of your site they are allowed to visit and index.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
07-22-2014, 04:28 AM
|
#6 |
|
Registered User
Join Date: Jul 2014
Posts: 648
|
Its Txt file by which you can tell search engine that what area of your site is allowed to crawl or store in a database or index.
|
|
|
|
 |
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Linear Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| What is the purpose of robots.txt file? | sahithya | Search Engine Optimization | 23 | 12-28-2021 03:48 AM |
| How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
| The robots.txt file | rubberfender | Yahoo | 0 | 12-02-2012 07:32 PM |
| Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
| Don�t exceed maximum file size for robots.txt file | ClaudiaSchayffe | Search Engine Optimization | 3 | 04-02-2012 04:28 AM |
All times are GMT -7. The time now is 05:14 AM.