Site Owners Forums - Webmaster Forums - Crawlers, spider & robots.txt?

Site Owners Forums - Webmaster Forums (http://siteownersforums.com/index.php)

- Search Engine Optimization (http://siteownersforums.com/forumdisplay.php?f=16)

- - Crawlers, spider & robots.txt? (http://siteownersforums.com/showthread.php?t=140498)

jackthomas087

04-22-2014 03:29 AM

Crawlers, spider & robots.txt?

By default robots.txt is in port 80. There is any way to verify in case robots.txt it's not in port 80 to get it from any other port it may be. Thanks.

All times are GMT -7. The time now is 11:08 PM.