02-12-2016, 11:19 PM
|
#1 |
|
Registered User
Join Date: Feb 2014
Location: india
Posts: 268
|
What is the use of Robots.txt?
Hello friends,
What is the use of Robots.txt? Is it useful when it comes to SEO Please suggest me about this Thanks in advance
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
02-13-2016, 01:33 AM
|
#2 |
|
Registered User
Join Date: Feb 2016
Posts: 18
|
Please click on
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
02-16-2016, 12:34 AM
|
#3 |
|
Registered User
Join Date: Jan 2016
Location: Mumbai, India
Posts: 1,064
|
The most common usage of Robots.txt is to ban crawlers from visiting private folders or content that gives them no additional information.
-Robots.txt Allowing Access to Specific Crawlers. -Allow everything apart from certain patterns of URLs.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
02-18-2016, 12:59 AM
|
#4 |
|
Registered User
Join Date: May 2015
Location: Forum
Posts: 2,517
|
Robots.txt is a text file that is inserted into your website to contain instructions for search engine robots. The file lists webpages which are allowed and disallowed from search engine crawling.
__________________
█ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | $1 VPS Hosting █ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | Windows Server with Remote Desktop Access █ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | Servers with Free IPMI Setup |
|
|
|
|
02-20-2016, 07:13 AM
|
#5 | |
|
Registered User
Join Date: Dec 2015
Posts: 580
|
Quote:
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
|
02-20-2016, 10:06 AM
|
#6 |
|
Registered User
Join Date: Jan 2016
Location: karachi
Posts: 38
|
Robots.txt is a text file that is inserted into your website to contain instructions for search engine robots.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
02-26-2016, 07:51 AM
|
#7 |
|
Registered User
Join Date: Feb 2016
Posts: 9
|
Robots.txt
In simple word Robots.txt is one type of text file
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
03-01-2016, 05:57 AM
|
#8 |
|
Registered User
Join Date: Feb 2016
Location: Surat
Posts: 396
|
Yes it is usefull, its a text file
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
03-02-2016, 05:06 AM
|
#9 |
|
Registered User
Join Date: Feb 2016
Posts: 51
|
The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
03-04-2016, 03:18 AM
|
#10 |
|
Registered User
Join Date: Feb 2016
Location: Surat
Posts: 396
|
Robots.txt is a text file that is inserted into your website to contain instructions for search engine robots.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
04-07-2016, 09:25 PM
|
#11 |
|
Registered User
Join Date: Apr 2016
Posts: 81
|
Robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
04-07-2016, 09:38 PM
|
#12 |
|
Registered User
Join Date: Mar 2016
Posts: 75
|
Robots.txt file is text file which inserted in your web hosting file.
Robots.txt file is important for crawling of your webpage. which page you want to crawl or which not.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
02-13-2017, 04:58 AM
|
#13 |
|
Registered User
Join Date: Nov 2016
Posts: 372
|
Robots.txt is a text file. It is through this file, it gives instruction to search engine crawlers about indexing and caching of a web page, file of a website or directory, domain.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
02-13-2017, 10:36 PM
|
#14 |
|
Registered User
Join Date: Sep 2016
Posts: 429
|
It is one of the text file.
|
|
|
|
|
02-14-2017, 03:03 AM
|
#15 |
|
Registered User
Join Date: Jul 2016
Posts: 755
|
Robots.txt Allowing Access to Specific Crawlers.
Allow everything apart from certain patterns of URLs.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
 |
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
| Thread Tools | |
| Display Modes | Rate This Thread |
Linear Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Crawlers, spider & robots.txt? | jackthomas087 | Search Engine Optimization | 0 | 04-22-2014 03:29 AM |
| Evil bots consuming all my sites traffic even after i disallowed them in robots.txt | Fking | General Discussion | 1 | 12-13-2013 11:27 PM |
| How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
| The robots.txt file | rubberfender | Yahoo | 0 | 12-02-2012 07:32 PM |
| Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
All times are GMT -7. The time now is 09:25 AM.