|
Msnbot, discobot, and Slurp are all called out specifically, so those user-agents will only pay attention to the directives in their sections of the robots.txt file. All other user-agents will follow the directives in the user-agent: * group.
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
/
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
/
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
/
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
/
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|