05-25-2016, 11:52 PM
|
#1 |
|
Registered User
Join Date: Mar 2016
Location: bangalore
Posts: 75
|
What is a robots.txt file?
What is a robots.txt file?
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
05-26-2016, 12:23 AM
|
#2 |
|
Registered User
Join Date: May 2016
Posts: 115
|
Robots.txt is a text file that is uploaded to a Website's root directory and linked in the html code of the website. The robots.txt file used to provide instructions about the website to webroots and spiders.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
05-26-2016, 05:02 AM
|
#3 |
|
Registered User
Join Date: May 2016
Location: Florida, United State
Posts: 36
|
Robot.txt is a text file.
It gives instruction to search engine crawlers about indexing and caching of a web page, file of website or directory, domain.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
05-26-2016, 05:38 AM
|
#4 |
|
Registered User
Join Date: May 2016
Posts: 551
|
robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
05-26-2016, 06:59 AM
|
#5 |
|
Registered User
Join Date: Sep 2015
Location: Bangalore
Posts: 548
 |
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
05-26-2016, 07:14 AM
|
#6 |
|
Registered User
Join Date: May 2016
Location: Dhaka
Posts: 55
|
What is Robots.txt File?
A Robots.txt file is an automatically generated by
Blogger for giving details information to search Robot about Blog posts. As a result when Search Robots from various search engines like Google, Bing, Yahoo, Yandex etc. visits your site then Robots.txt tells search crawler what to crawl and what not to crawl. |
|
|
|
|
05-26-2016, 07:41 AM
|
#7 |
|
Registered User
Join Date: May 2016
Location: New York City, NY, USA
Posts: 37
|
Site proprietors utilize the/robots.txt document to give guidelines about their webpage to web robots; this is known as The Robots Exclusion Protocol.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
|
05-26-2016, 09:48 PM
|
#8 |
|
Registered User
Join Date: May 2016
Posts: 22
|
Very Useful information on Robots.txt, thanks for sharing
|
|
|
|
|
05-26-2016, 11:14 PM
|
#9 |
|
Registered User
Join Date: Dec 2012
Posts: 657
|
Robots.txt is a text file that is inserted into your website and contains information for search engine robots. The file lists webpages that are allowed and disallowed from search engine crawling.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
 |
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Linear Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| What is the purpose of robots.txt file? | sahithya | Search Engine Optimization | 23 | 12-28-2021 03:48 AM |
| How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
| The robots.txt file | rubberfender | Yahoo | 0 | 12-02-2012 07:32 PM |
| Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
All times are GMT -7. The time now is 03:54 AM.