Site Owners Forums - Webmaster Forums

Site Owners Forums - Webmaster Forums (http://siteownersforums.com/index.php)

- Search Engine Optimization (http://siteownersforums.com/forumdisplay.php?f=16)

- - What is robots.txt? (http://siteownersforums.com/showthread.php?t=192376)

najishahmed

04-19-2017 03:08 AM

What is robots.txt?

Hlo friends,
Can anyone tell me, What is robots.txt?

autocarcovers

04-19-2017 03:49 AM

Robots.txt and Meta Robots. The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.

jamesvincent21

04-19-2017 05:43 AM

The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.

sofiasofi

04-19-2017 05:51 AM

Robots.text and Meta Robots. The robots exclusion protocol (REP), or robots.text is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.

All times are GMT -7. The time now is 01:48 PM.