Site Owners Forums - Webmaster Forums

Site Owners Forums - Webmaster Forums (http://siteownersforums.com/index.php)
-   Search Engine Optimization (http://siteownersforums.com/forumdisplay.php?f=16)
-   -   What is robot text (http://siteownersforums.com/showthread.php?t=63876)

jeffzeilenga 03-13-2013 04:05 AM

using Robots.txt is a file we can guide search engines to crawl or not to crawl certain sections of your website.

outure11 03-25-2013 10:17 PM

Actually the search engines handle nofollow differently. Google will not follow the link, assign PR to it, or give it weight in its algorithm. Yahoo will follow the link but give it no weight in their algorithm. MSN says they won't give the link any value but it is hard to confirm exactly what they're doing.

johnmartin9990 03-26-2013 01:46 AM

Well ... robots.text isa text file in which you difine instruction which page you want to be crawl or not to all robots of search engines.

hostsingh 03-27-2013 01:20 AM

Robots.txt is a text file you put it on your web site to tell search robots which pages you would like them not to visit. It is important to clarify that robots.txt is not a way from preventing search engines from crawling your site.


All times are GMT -7. The time now is 01:25 PM.


Powered by vBulletin Copyright © 2020 vBulletin Solutions, Inc.