03-14-2014, 04:25 AM
|
#1 |
|
Registered User
Join Date: Mar 2014
Posts: 11
|
Why is a Robots.txt File Used?
Why is a Robots.txt File Used?
matrimonial sites for hindu brahmin | indian matrimonial sites in new zealand |matrimony sites in gurgaon |
|
|
|
03-14-2014, 04:33 AM
|
#2 |
|
Registered User
Join Date: Jan 2014
Posts: 9
|
Robots.txt google robot allows better orientation
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
03-14-2014, 06:28 AM
|
#3 |
|
Registered User
Join Date: Mar 2014
Posts: 14
|
Use a show to gateway to for search engine bot to what to index what not to
|
|
|
|
|
03-23-2014, 12:58 AM
|
#4 |
|
Registered User
Join Date: Mar 2014
Location: ny
Posts: 12
    |
that is use for drive search engine bot
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
04-02-2014, 03:42 AM
|
#5 |
|
Registered User
Join Date: Mar 2014
Posts: 33
|
The basic format to write a robots.txt file is
User-agent: * Disallow: /url1/ Disallow: /url2/ Disallow: /url.html If you want to disallow few webpages crawling from search engines then you can add in this robots.txt file
__________________
Live TV Services - To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
 |
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Linear Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
| The robots.txt file | rubberfender | Yahoo | 0 | 12-02-2012 07:32 PM |
| RSS Feed | rajivwebads | Search Engine Optimization | 12 | 08-31-2012 07:15 AM |
| Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
| Don�t exceed maximum file size for robots.txt file | ClaudiaSchayffe | Search Engine Optimization | 3 | 04-02-2012 04:28 AM |
All times are GMT -7. The time now is 02:58 PM.