A crawler is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index. The major search engines on the Web all have such a program, which is also known as a "spider" or a "bot."

A crawler is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index. Crawlers are typically programmed to visit sites that have been submitted by their owners as new or updated.

Crawling is the process by virtue of which the search engines gather information about websites on world wide web.The crawlers are also known as spiders or bots , they visit website and send information to their respective parent websites.

Crawl is a set of programs through which google crawl your website and indexing their database.

A program that systematically browses the World Wide Web in order to create an index of data.