Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web. By defining a few rules in this text file, you can instruct robots to not crawl and index certain files, directories within your site, or at all.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. - Shopping Cart- PrestaShop, Zencart; CMS - WordPress, Joomla; Custom Development
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. - SEO, Pay Per Click (PPC), SEM, Link Popularity Building, SEO Copywriting, SMO, SMM
|