|
||||||||
| Notices |
 |
|
|
Thread Tools | Rate Thread | Display Modes |
03-15-2016, 04:57 AM
|
#1 |
|
Registered User
Join Date: Jan 2016
Location: Mumbai, India
Posts: 1,064
|
What is robot.txt file and what are the benefits of using it..?
What is robot.txt file and what are the benefits of using it..?
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
03-15-2016, 05:55 AM
|
#2 |
|
Registered User
Join Date: Jul 2015
Location: Canada
Posts: 133
|
Robot.txt is a way to inform search engine that which file or folder on the website avoid to crawl. It is very important for website also.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. , To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
11-14-2016, 03:06 AM
|
#3 |
|
Registered User
Join Date: Oct 2016
Posts: 580
|
Robot.txt is a way to inform search engine that which file or folder on the website avoid to crawl. It is very important for website also.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
11-15-2016, 03:47 AM
|
#4 |
|
Registered User
Join Date: Aug 2016
Location: USA
Posts: 117
|
It is a text file which tells bot to what to crawl or what to not
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. || To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
11-24-2016, 04:03 AM
|
#5 |
|
Registered User
Join Date: Nov 2016
Posts: 372
|
"robots.txt" file is a very Useful file for webmasters and website owners who wish to have a Good control of their website control over Internet and to search engines.
Search Engines go through all the directories and Links of your website and they, sometimes, also go to the folders which you do not wish to let them go to, say website.com/administration/
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
11-24-2016, 09:00 AM
|
#6 |
|
Registered User
Join Date: Oct 2016
Posts: 136
|
Adplotter helps you easily create, organize and distribute advertisements and videos online that sell your products or services. visit- visit- http://tiny.cc/ocq5gy
Last edited by Sky Benson; 11-24-2016 at 09:35 AM.. |
|
|
|
|
11-24-2016, 09:42 PM
|
#7 |
|
Registered User
Join Date: May 2014
Posts: 670
|
Robot.txt file is a way for the website to communicate with the robots/crawlers, wherein it tells if a certain page should be indexed or not.
__________________
12BET To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
05-05-2017, 04:18 AM
|
#8 |
|
Registered User
Join Date: Apr 2017
Posts: 355
|
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.
The purpose of a robots.txt file is to tell search engines not to crawl the contents of specific directories or even individual pages within a website.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
05-05-2017, 04:30 AM
|
#9 |
|
Registered User
Join Date: Sep 2016
Posts: 246
|
A robots.txt file is a file at the root of your site that indicates those parts of your site you don’t want to be accessed by search engine crawlers. The file uses the Robots Exclusion Standard, which is a protocol with a small set of commands that can be used to indicate access to your site by section and by specific kinds of web crawlers (such as mobile crawlers vs desktop crawlers).
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Linear Mode
|
|
All times are GMT -7. The time now is 05:36 PM.