Crawling and cashing - Site Owners Forums

satnam1980 · 07-10-2012, 01:52 PM

Web crawler is a computer program that browses the World Wide Web in a methodical, automated manner or in an orderly fashion. Other terms for Web crawlers are ants, automatic indexers, bots, Web spiders, Web robots,or especially in the FOAF community Web scutter..

Caching can happen at many places, including proxies (i.e. the user�s ISP) and the user�s local machine. The objective is to make efficient use of resources and speed the delivery of content to the end user.While caching can have a positive impact on the user�s experience, it can have a negative impact for site publishers, resulting in undercounts of page views and ad impressions. In response to this problem, sites have implemented various cache-busting techniques to better ensure that all performance statistics are accurately measured.

Thread Tools
Show Printable Version Email this Page
Display Modes	Rate This Thread
Switch to Linear Mode Switch to Hybrid Mode Threaded Mode	Rate This Thread:

07-10-2012, 01:52 PM	#4
satnam1980 Registered User Join Date: Jul 2012 Posts: 14	Web crawler is a computer program that browses the World Wide Web in a methodical, automated manner or in an orderly fashion. Other terms for Web crawlers are ants, automatic indexers, bots, Web spiders, Web robots,or especially in the FOAF community Web scutter.. Caching can happen at many places, including proxies (i.e. the user�s ISP) and the user�s local machine. The objective is to make efficient use of resources and speed the delivery of content to the end user.While caching can have a positive impact on the user�s experience, it can have a negative impact for site publishers, resulting in undercounts of page views and ad impressions. In response to this problem, sites have implemented various cache-busting techniques to better ensure that all performance statistics are accurately measured.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)