Problem with robots.txt file - Site Owners Forums

Ajaysharma · 11-18-2016, 11:45 PM

"Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web. By defining a few rules in this text file, you can instruct robots to not crawl and index certain files, directories within your site, or at all. For example, you may not want Google to crawl the /images directory of your site, as it's both meaningless to you and a waste of your site's bandwidth.

basrtaj · 11-22-2016, 04:44 AM

Thank you all, but Should i add it to my website Host or no???

11-18-2016, 11:45 PM	#1
Ajaysharma Registered User Join Date: Nov 2016 Posts: 372	"Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web. By defining a few rules in this text file, you can instruct robots to not crawl and index certain files, directories within your site, or at all. For example, you may not want Google to crawl the /images directory of your site, as it's both meaningless to you and a waste of your site's bandwidth. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. \| To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

11-22-2016, 04:44 AM	#2
basrtaj Registered User Join Date: Nov 2014 Posts: 14	Thank you all, but Should i add it to my website Host or no??? __________________ Hi! Your Bro Mohamed Amine, a simple man who love blogging and search engin SEO. this is one of my websites: To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)