Robots.txt is a text file used by website to tell web crawler that what to follow and what to not and i found this online tool which can help you to generate robots.txt file.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
Last edited by imran.kkhan1; 07-27-2016 at 08:28 AM..
Reason: link
|