Site Owners Forums - Webmaster Forums - View Single Post

flumenk · 12-15-2013, 02:27 AM

A Web crawler is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing.

A Web crawler may also be called a Web spider, an ant, an automatic indexer, or (in the FOAF software context) a Web scutter.

12-15-2013, 02:27 AM	#3
flumenk Registered User Join Date: Dec 2013 Posts: 4	A Web crawler is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing. A Web crawler may also be called a Web spider, an ant, an automatic indexer, or (in the FOAF software context) a Web scutter.