01-08-2012, 02:06 AM
|
#1 |
|
Registered User
Join Date: Jan 2012
Posts: 17
|
How to slow the crawl rate of yahoo?
Hello members,
How do I slow the crawl rate of yahoo? its killing my bandwidth 700 bots a day and putting extra strain on my server. Please tell me How to do this?
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
02-05-2017, 02:52 AM
|
#2 |
|
Registered User
Join Date: Feb 2017
Posts: 149
|
Slurp is the Yahoo search bot for the crawl and index your web page.
It obeys the Robot Exclusion standard. If you don not want to crawl frequently your web page, create a robots.txt file in the root directory (Home folder of site and add a rule for "User-agent: Slurp" Code example in a robot.txt file: User-agent: Slurp<br>disallow:/cgi-bin/ |
|
|
|
 |
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Linear Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Yahoo Questions and Answers | jaxondavis37 | Yahoo | 11 | 05-22-2017 09:16 PM |
| Yahoo optimization � keywords by relevance | loganallen48 | Yahoo | 2 | 10-09-2014 11:57 PM |
| New mobile service by Yahoo! | gavindmello053 | Yahoo | 3 | 01-02-2012 10:14 PM |
| How can earn money through Yahoo! | haroldmacmillan | Yahoo | 3 | 12-08-2011 05:46 PM |
| Optimizing Yahoo | andysmith476 | Yahoo | 0 | 11-16-2011 02:04 AM |
All times are GMT -7. The time now is 07:56 AM.