05-23-2014, 03:36 AM
|
#7 |
|
Registered User
Join Date: May 2014
Location: Quebec City, Quebec, Canada
Posts: 23
|
By way of example, my website is css and html based, which means that it is not currently built with a CMS such as WordPress' and I use the robot.txt file to excule search engines from reaching the development directories I use to test drive CMS softwares such as WordPress. When my WordPress based site is ready for being seen by the world, I will remove the exclusion line from my robot.txt.
You can find the syntax supported by robot.txt files by doing a google search. They will also tell it where to put it on your site, which is the same directory where your index.html file is. If you use a CMS like WordPress, I believe the software generates the robot.txt file according to the parameters you choose within WordPress, but I'm less familiar with these than with the css and html based web sites.
__________________
Daniel Garneau To view links or images in signatures your post count must be 10 or greater. You currently have 0 posts. -- Knowing and Believing: Torn-apart, yet inseparable allies… Last edited by DanielGarneau; 05-23-2014 at 03:41 AM.. |
|
|
«
Previous Thread
|
Next Thread
»
| Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
| Thread Tools | |
| Display Modes | Rate This Thread |
Threaded Mode
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Crawlers, spider & robots.txt? | jackthomas087 | Search Engine Optimization | 0 | 04-22-2014 03:29 AM |
| Evil bots consuming all my sites traffic even after i disallowed them in robots.txt | Fking | General Discussion | 1 | 12-13-2013 11:27 PM |
| How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
| The robots.txt file | rubberfender | Yahoo | 0 | 12-02-2012 07:32 PM |
| Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
All times are GMT -7. The time now is 06:16 PM.