View Single Post
Old 05-26-2016, 03:41 AM   #4
SMTPL
Registered User
 
Join Date: May 2016
Posts: 56
Spider - a browser like program that downloads web pages.

Crawler – a program that automatically follows all of the links on each web page.

Robots - An automated computer program that visit websites & perform predefined task. They are guided by search engine algorithms & are able to perform different tasks instead of just one crawling task. They can combine the tasks of crawler & spider together and help in indexing and ranking of websites on any particular search engine.
SMTPL is offline   Reply With Quote