02-05-2017, 02:52 AM
|
#2 |
|
Registered User
Join Date: Feb 2017
Posts: 149
|
Slurp is the Yahoo search bot for the crawl and index your web page.
It obeys the Robot Exclusion standard. If you don not want to crawl frequently your web page, create a robots.txt file in the root directory (Home folder of site and add a rule for "User-agent: Slurp" Code example in a robot.txt file: User-agent: Slurp<br>disallow:/cgi-bin/ |
|
|
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
| Thread Tools | |
| Display Modes | Rate This Thread |
Threaded Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Yahoo Questions and Answers | jaxondavis37 | Yahoo | 11 | 05-22-2017 09:16 PM |
| Yahoo optimization � keywords by relevance | loganallen48 | Yahoo | 2 | 10-09-2014 11:57 PM |
| New mobile service by Yahoo! | gavindmello053 | Yahoo | 3 | 01-02-2012 10:14 PM |
| How can earn money through Yahoo! | haroldmacmillan | Yahoo | 3 | 12-08-2011 05:46 PM |
| Optimizing Yahoo | andysmith476 | Yahoo | 0 | 11-16-2011 02:04 AM |
All times are GMT -7. The time now is 11:54 AM.