What are Spiders, Robots and Crawlers and what are their functions?
 

What are Spiders, Robots and Crawlers and what are their functions?

Spiders, robot and crawler, they are all same and referred by different names.
It is a software program that follows, or “Crawls” different links throughout the internet, and then grabs the content from the sites and adds to the search engine indexes.

Crawler: Also known as Robot, Bot or Spider. These are programs used by search engines to explore the Internet and automatically download web content available on web sites. They capture the text of the pages and the links found and thus enable search engine users to find new pages.

They are all the same and are automated search engine programs that are responsible to read through webpage sources to provide information to search engines.