Thread: What Is Robots.txt?
View Single Post
Old 03-07-2012, 04:28 AM   #5
Johnmark
Registered User
 
Join Date: Dec 2011
Posts: 153
it gives the instruction to not crawl the specific pages or file mention in txt
__________________

To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Johnmark is offline   Reply With Quote