It is the web crawler used to find the content present in the website. These web spiders are sent by the search engines to gather the content and after crawling content will be stored in the data base of search engine's.

A web crawler (also known as web spider) is a program which browses the World Wide Web in a methodical, automated manner.

Many search engines use spider software program to index website. Spider also ranks the content being indexed.

Spider is a crawler that crawl the web site.

a spider is a web crawler.

A spider is a program that visits Websites and reads their pages and other information in order to create entries for a search engine index.

Spiders are air-breathing arthropods that have eight legs, chelicerae with fangs generally able to inject venom, and spinnerets that extrude silk. They are the largest order of arachnids and rank seventh in total species diversity among all orders of organisms.

What is spider?
 

A spider is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index.