Is robert.txt file is important to Website? - Site Owners Forums

davisbusiness · 09-01-2016, 02:04 AM

Is robert.txt file is important to Website?

Nura · 09-01-2016, 02:32 AM

Sure its so important to website

davidsmith01 · 09-01-2016, 02:35 AM

It's very important,Robots.txt file is what tells the search engines which pages to access and index on your website on which pages not to.

Martinricky · 09-01-2016, 02:43 AM

Your Robots.txt file is what tells the search engines which pages to access and index on your website on which pages not to.

amanrai · 09-01-2016, 05:06 AM

Yes robot.txt is very much important for on page optimization. Google considers a website with robot.txt file. It tells the google bots which web page to crawl and which one to ignore.

anky · 09-01-2016, 05:19 AM

Yes, it is important if you want google crawlers/bots or other search engine bots to crawl and index your website.

alex.thomson · 09-01-2016, 05:55 AM

You might be surprised to hear that one small text file, known as robots.txt, could be the downfall of your website. If you get the file wrong you could end up telling search engine robots not to crawl your site, meaning your web pages won’t appear in the search results. Therefore, it’s important that you understand the purpose of a robots.txt file and learn how to check you’re using it correctly.

webiletech · 09-01-2016, 07:07 AM

Obviously Robots.txt file is very important because it tells search engines spiders that which page has to visit and which page has not visit.

FineGadgetsUK · 09-01-2016, 08:31 AM

Robots.txt file helps Google and other search engines to index your website properly.

timly · 09-01-2016, 08:46 AM

It is very important to your website, However if you don't want to crawl any particular folder or file you have to just put the Robert.txt in that file.

anna91 · 09-01-2016, 04:33 PM

yes sure very important you can search google to know detail about it

johnalandil · 09-01-2016, 06:18 PM

With robot.txt you can let google know where you want Google to crawl in your site and so you can index your content.

09-01-2016, 02:04 AM	#1
davisbusiness Registered User Join Date: Jun 2016 Posts: 38	Is robert.txt file is important to Website? Is robert.txt file is important to Website? __________________ Business Development, Business Coach

09-01-2016, 02:35 AM	#3
davidsmith01 Registered User Join Date: Apr 2016 Location: USA Posts: 219	It's very important,Robots.txt file is what tells the search engines which pages to access and index on your website on which pages not to. __________________ Taxi App Developers \| Online taxi booking software \| Uber Clone App \| Taxi app development \| truck driver app \|

09-01-2016, 02:43 AM	#4
Martinricky Registered User Join Date: Aug 2015 Posts: 434	Your Robots.txt file is what tells the search engines which pages to access and index on your website on which pages not to. __________________ Dentist in Delhi \| Best Dentist in Ashok Vihar \| Best Dentist in Shalimar Bagh \| Best Prosthodontist in Delhi

09-01-2016, 05:06 AM	#5
amanrai Registered User Join Date: Mar 2016 Posts: 343	Yes robot.txt is very much important for on page optimization. Google considers a website with robot.txt file. It tells the google bots which web page to crawl and which one to ignore. __________________ Packers and movers Ambikapur \| Packers and movers Raipur

09-01-2016, 05:19 AM	#6
anky Registered User Join Date: Jun 2016 Location: India Posts: 32	Yes, it is important if you want google crawlers/bots or other search engine bots to crawl and index your website. __________________ Mobile App Development Company \| Mobile App Development India \| iPhone App Development Company Android App Development Company

09-01-2016, 02:32 AM	#2
Nura Registered User Join Date: Aug 2016 Posts: 17	Sure its so important to website

09-01-2016, 05:55 AM	#7
alex.thomson Registered User Join Date: Jul 2016 Posts: 241	You might be surprised to hear that one small text file, known as robots.txt, could be the downfall of your website. If you get the file wrong you could end up telling search engine robots not to crawl your site, meaning your web pages won’t appear in the search results. Therefore, it’s important that you understand the purpose of a robots.txt file and learn how to check you’re using it correctly. __________________ Taxi Service From Bangalore Airport To Mysore Last edited by alex.thomson; 09-01-2016 at 06:00 AM..

09-01-2016, 07:07 AM	#8
webiletech Registered User Join Date: May 2016 Location: Ahmedabad Posts: 9	Obviously Robots.txt file is very important because it tells search engines spiders that which page has to visit and which page has not visit. __________________ iPhone App Development \| iPad App Development \| Android App Development \| iOS App Development

09-01-2016, 08:31 AM	#9
FineGadgetsUK Registered User Join Date: Aug 2016 Location: London Posts: 54	Robots.txt file helps Google and other search engines to index your website properly.

09-01-2016, 08:46 AM	#10
timly Registered User Join Date: May 2016 Posts: 105	It is very important to your website, However if you don't want to crawl any particular folder or file you have to just put the Robert.txt in that file. __________________ motivational quotes inspirational quotes funny quotes friendship quotes life quotes

09-01-2016, 04:33 PM	#11
anna91 Registered User Join Date: Aug 2016 Posts: 86	yes sure very important you can search google to know detail about it __________________ shopify store pro shopify store pro review Jame Review Dot Com

09-01-2016, 06:18 PM	#12
johnalandil Registered User Join Date: Sep 2016 Posts: 32	With robot.txt you can let google know where you want Google to crawl in your site and so you can index your content. __________________ ConnectAudience Bonus - Click Here\|ConnectAudience Review - Why should you buy it?

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Thread Tools
Show Printable Version Email this Page
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread:

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Secrete for seo	alina albert	Search Engine Optimization	87	03-11-2018 05:47 AM
What’s the significance of Robots.txt file in a website?	dyhair77	Search Engine Optimization	0	12-02-2015 04:01 AM
Check the website robots file	china-jewels	Bing	4	10-11-2015 08:14 AM
Why SMO is important for a website ?	bhavnasingh482	Search Engine Optimization	20	07-24-2015 08:25 AM
four important social media icons for a website?	arronmattwills	Graphic Design	5	05-26-2014 01:42 AM