|
||||||||
| Notices |
 |
|
|
Thread Tools | Rate Thread | Display Modes |
|
|
02-23-2017, 01:57 AM
|
#1 |
|
Registered User
Join Date: Oct 2016
Posts: 580
|
Spiders is a name for the robots or crawlers that scan web and save the information in websites to a database. Hence, "Google spiders" is another name for "Google indexing bots". Spider - a browser like program that downloads web pages.
The Robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website. Crawler – a program that automatically follows all of the links on each web page
__________________
To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. | To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. |
|
|
|
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
| Thread Tools | |
| Display Modes | Rate This Thread |
Hybrid Mode
|
|
All times are GMT -7. The time now is 09:06 PM.