Go Back   Site Owners Forums - Webmaster Forums > Search Engine Optimization > Search Engine Optimization

Notices


Reply
 
Thread Tools Rate Thread Display Modes
Old 12-31-2021, 03:26 AM   #16
nathanleo
Registered User
 
Join Date: Jul 2017
Posts: 2,060
A robots. txt file tells search engine crawlers which URLs the crawler can access on your site. This is used mainly to avoid overloading your site with requests; it is not a mechanism for keeping a web page out of Google.
nathanleo is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
What is the purpose of robots.txt file? sahithya Search Engine Optimization 23 12-28-2021 03:48 AM
How to edit virtual robots.txt file in wordpress? geniusoptimizer PHP / mySQL 1 04-12-2013 12:36 PM
The robots.txt file rubberfender Yahoo 0 12-02-2012 07:32 PM
Unable to delete robots.txt file notfake Yahoo 0 04-29-2012 02:19 AM


All times are GMT -7. The time now is 06:41 PM.


Powered by vBulletin Copyright © 2020 vBulletin Solutions, Inc.