|
robots.txt is a text file and if you don't want to crawl your webpage, you just write that file name and give robot.txt extension so crawler don't crawl that page....
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|