|
Robots.txt is a text file which you put on your website to tell crawler which pages you would like not to crawl.
You need to create a file and you have to enter that URL which you don't want to crawl. Have a look below.
User Agent: *
Disallow: //admin
Disallow: ///t141187-
Disallow: //file.html
Disallow: //purpose
__________________
Supreme Suppliers & Manufacturers of Plastic Furniture India - Find modern designer plastic moulded good quality Premium chairs/ Furnitures,dining Chair, tables, furniture.
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts.
|