Site Owners Forums - Webmaster Forums - View Single Post - Is robert.txt file is important to Website?

alex.thomson · 09-01-2016, 05:55 AM

You might be surprised to hear that one small text file, known as robots.txt, could be the downfall of your website. If you get the file wrong you could end up telling search engine robots not to crawl your site, meaning your web pages won’t appear in the search results. Therefore, it’s important that you understand the purpose of a robots.txt file and learn how to check you’re using it correctly.

09-01-2016, 05:55 AM	#7
alex.thomson Registered User Join Date: Jul 2016 Posts: 241	You might be surprised to hear that one small text file, known as robots.txt, could be the downfall of your website. If you get the file wrong you could end up telling search engine robots not to crawl your site, meaning your web pages won’t appear in the search results. Therefore, it’s important that you understand the purpose of a robots.txt file and learn how to check you’re using it correctly. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. Last edited by alex.thomson; 09-01-2016 at 06:00 AM..