Quote:
Originally Posted by cpsi
Robot Visit your Website And Collect your Information. If you don't include robot meta tag then google not crawl your website.
|
Are you sure about that? You use the robots meta tag to stop Goolge from indexing a particular page and to set nofollow link atributes to all links on that page.
You can also use the robots meta tag to only allow certain spiders like Google bot to index your page.
If you dont ad the robots metaa tag then you are basically telling Google to go ahead and index my page.