|
ROBOTS.TXT file is better because it is a file of that exact name stored in the root of your web server.
It provides instructions to spiders and other web page crawler about what pages of your site they provide and should not visit.
Thank you!
Happy to help you.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|