Is it necessary to use Robots.txt file? - Site Owners Forums

latestnewsheadl · 02-16-2018, 04:44 AM

Hello friends- Is it necessary to use Robots.txt file? It can be affect on website ranking in search engine.

tanyaa · 02-16-2018, 04:46 AM

Yes, absolutely. Because Robosts.txt generator tells the search engines which pages to access and index on your website and which pages not to.

nathanleo · 07-21-2022, 05:19 AM

Warning: Don't use a robots. txt file as a means to hide your web pages from Google search results. If other pages point to your page with descriptive text, Google could still index the URL without visiting the page.

pharmasecure · 07-22-2022, 05:04 AM

Robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl & index pages on their website.

AzharSEO · 07-23-2022, 06:31 AM

This depends on us because, through robot.txt files, we can tell Google which website url you need to crawl and which page you need not to crawl.

02-16-2018, 04:44 AM	#1
latestnewsheadl Registered User Join Date: Jan 2017 Location: New Delhi Posts: 24	Is it necessary to use Robots.txt file? Hello friends- Is it necessary to use Robots.txt file? It can be affect on website ranking in search engine. __________________ Best Overseas Education Consultants in Delhi \| Study Abroad Consultancy in Delhi \| Study Abroad Consultants in Delhi \| Study in Ukraine \| Study MBBS in Ukraine

02-16-2018, 04:46 AM	#2
tanyaa Registered User Join Date: Feb 2018 Posts: 27	Yes, absolutely. Because Robosts.txt generator tells the search engines which pages to access and index on your website and which pages not to. __________________ Cookies plant Bakery Equipment Pasta Plant Packing Machines Potato baked chips plant Centre filled cookies

07-21-2022, 05:19 AM	#3
nathanleo Registered User Join Date: Jul 2017 Posts: 2,060	Warning: Don't use a robots. txt file as a means to hide your web pages from Google search results. If other pages point to your page with descriptive text, Google could still index the URL without visiting the page. __________________ English classes eastcobb Math classes eastcobb Math classes Marietta Critical Thinking Math Program Math Learning Centers Near Me Critical Thinking Math Classes Near Me Eye Level Critical Thinking Math

07-22-2022, 05:04 AM	#4
pharmasecure Registered User Join Date: Jun 2018 Posts: 508	Robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl & index pages on their website. __________________ Falsified medicines directive serialisation \| Serialization services for pharma manufacturing \| India serialization solution provider

07-23-2022, 06:31 AM	#5
AzharSEO Registered User Join Date: Aug 2014 Location: India Posts: 816	This depends on us because, through robot.txt files, we can tell Google which website url you need to crawl and which page you need not to crawl. __________________ It Consulting Services \| Enterprise Applications \| Business Solutions \| Iot Applications \| Iot \| It Consulting

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
How to edit virtual robots.txt file in wordpress?	geniusoptimizer	PHP / mySQL	1	04-12-2013 12:36 PM
The robots.txt file	rubberfender	Yahoo	0	12-02-2012 07:32 PM
Unable to delete robots.txt file	notfake	Yahoo	0	04-29-2012 02:19 AM