Site Owners Forums - Webmaster Forums - View Single Post

AshleyScott · 08-18-2012, 05:06 AM

Robots. txt file is necessary at time where you want to instruct the crawler for the pages it is allowed to crawl.

08-18-2012, 05:06 AM	#11
AshleyScott Registered User Join Date: Jul 2012 Posts: 9	Robots. txt file is necessary at time where you want to instruct the crawler for the pages it is allowed to crawl. __________________ To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.