Site Owners Forums - Webmaster Forums - View Single Post

danish00 · 03-20-2013, 11:50 PM

robots.txt file is very important to crawl or not crawls the pages. as we give coding
User-Agent: *
Disallow:
if you want to disallow the page
disallow:/
event management companies in kolkata

03-20-2013, 11:50 PM	#13
danish00 Registered User Join Date: Feb 2013 Location: delhi Posts: 103	robots.txt file is very important to crawl or not crawls the pages. as we give coding User-Agent: * Disallow: if you want to disallow the page disallow:/ event management companies in kolkata __________________ camp in kanatal