Register on the forum now to remove ALL ads + popups + get access to tons of hidden content for members only!

Go Back   Site Owners Forums - Webmaster Forums > Search Engine Optimization > Search Engine Optimization

Notices


Reply
 
Thread Tools Rate Thread Display Modes
Old 07-10-2017, 08:00 AM   #1
shopermart
Registered User
 
Join Date: Jun 2017
Location: Surat, Gujarat, India
Posts: 149
What is robots.txt?

What is robots.txt?
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
shopermart is offline   Reply With Quote

Old 07-10-2017, 11:11 PM   #2
quickyads
Registered User
 
Join Date: Jul 2017
Posts: 42
Robots.txt is a textual content report webmasters create to train web robots (typically search engine robots) how to crawl pages on their internet site.
quickyads is offline   Reply With Quote
Old 07-10-2017, 11:12 PM   #3
Jessianut
Registered User
 
Join Date: Jul 2017
Posts: 282
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Jessianut is offline   Reply With Quote
Old 07-10-2017, 11:42 PM   #4
esourcepart
Registered User
 
Join Date: Dec 2013
Posts: 667
The robots.txt file is primarily used to specify File used to direct or to tell web bots what pages and directories to index or not index. This file must be placed in the root directory on the server hosting your pages.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
esourcepart is offline   Reply With Quote
Old 07-11-2017, 12:02 AM   #5
jessicajesi
Registered User
 
Join Date: Feb 2017
Posts: 378
Robots.txt is a textual content report webmasters create to train web robots (typically search engine robots) how to crawl pages on their internet site.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
jessicajesi is offline   Reply With Quote
Old 07-11-2017, 12:08 AM   #6
jimmyjohnson
Registered User
 
Join Date: Apr 2017
Location: Mountain View, CA
Posts: 47
Robot.txt file is use to give instruction to the search engine bot if you want to hide some private pages which you don't want to indexed by search engine bot. This file is present in the root file of the website!
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
jimmyjohnson is offline   Reply With Quote
Old 07-11-2017, 12:17 AM   #7
Hashmicro
Registered User
 
Join Date: Nov 2016
Posts: 614
Robots.txt is common name of a text file that is uploaded to a Web site's root directory and linked in the html code of the Web site.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Hashmicro is offline   Reply With Quote
Old 07-11-2017, 12:26 AM   #8
jimken
Registered User
 
Join Date: May 2017
Location: Piscataway, NJ
Posts: 39
Robots.txt is a textual file that use to tell webmaster what to crawl or what not to inside the website.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
jimken is offline   Reply With Quote
Old 07-11-2017, 12:40 AM   #9
quickyadspro
Registered User
 
Join Date: Jul 2017
Posts: 113
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
quickyadspro is offline   Reply With Quote
Old 07-11-2017, 01:16 AM   #10
sharmaroshni012
Registered User
 
Join Date: Nov 2016
Posts: 510
A robots.txt file is a file at the root of your site that indicates those parts of your site you don’t want to be accessed by search engine crawlers. The file uses the Robots Exclusion Standard, which is a protocol with a small set of commands that can be used to indicate access to your site by section and by specific kinds of web crawlers (such as mobile crawlers vs desktop crawlers).
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
sharmaroshni012 is offline   Reply With Quote
Old 07-11-2017, 03:17 AM   #11
Jacklincy
Registered User
 
Join Date: Dec 2016
Posts: 142
Robot.txt file is use to give instruction to the search engine bot if you want to hide some private pages which you don't want to indexed by search engine bot. This file is present in the root file of the website!
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Jacklincy is offline   Reply With Quote
Old 07-11-2017, 03:51 AM   #12
sparkleoffice
Registered User
 
Join Date: Oct 2016
Posts: 38
The spiders exemption conventional, also known as the spiders exemption method or simply spiders.txt, is a conventional used by sites to talk with web spiders and other web spiders. The common identifies how to tell the web software about which areas of the website should not be prepared or examined.
sparkleoffice is offline   Reply With Quote
Old 07-11-2017, 03:55 AM   #13
Srinivasa
Registered User
 
Join Date: Jul 2017
Location: Chennai
Posts: 15
Hi,

The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.


To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Srinivasa is offline   Reply With Quote
Old 07-11-2017, 03:55 AM   #14
SophyMalkani
Registered User
 
Join Date: Mar 2017
Posts: 37
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
,
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
,
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
,
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
,
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
,
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
,
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
,
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
SophyMalkani is offline   Reply With Quote
Old 07-11-2017, 04:05 AM   #15
sachinkumar12
Registered User
 
Join Date: Apr 2016
Posts: 25
The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.
sachinkumar12 is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Robots.txt or Meta Robots.txt??? santoshkumar Search Engine Optimization 11 02-14-2017 03:03 AM
Robots.txt jaysh4922 Search Engine Optimization 15 07-29-2016 05:07 AM
Evil bots consuming all my sites traffic even after i disallowed them in robots.txt Fking General Discussion 1 12-13-2013 10:27 PM
How to edit virtual robots.txt file in wordpress? geniusoptimizer PHP / mySQL 1 04-12-2013 12:36 PM
Unable to delete robots.txt file notfake Yahoo 0 04-29-2012 02:19 AM


All times are GMT -7. The time now is 05:40 AM.


Powered by vBulletin Copyright © 2013 vBulletin Solutions, Inc.