What is Robot.txt? - Site Owners Forums

		Site Owners Forums - Webmaster Forums > General Arena > General Discussion
What is Robot.txt?

FAQ

Calendar

Today's Posts

Notices

Welcome to SiteOwnersForums.com #1 Webmasters Community Online

You are currently viewing our forum as a guest which gives you limited access.
Membership is absolutely FREE and registration is FAST & SIMPLE so please, Register Today and join one of the friendliest communities on the net!
If you have any problems with the registration process or your account login, please contact us.

Thread Tools

Rate Thread

Display Modes

03-24-2017, 07:28 AM	#7
robinmym Registered User Join Date: Feb 2017 Location: Dhaka Posts: 486	The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned. The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.

« Previous Thread | Next Thread »

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Posting Rules
You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is Off HTML code is Off Forum Rules

Forum Jump

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
What is robot.txt file and what are the benefits of using it?	Martinricky	Social Networks	31	12-21-2016 02:46 AM
Which pages should be put in robot.txt file?	Martinricky	Search Engine Optimization	1	07-24-2016 11:15 PM
Robot.txt?	siva SEO	Search Engine Optimization	17	01-09-2015 07:02 PM
how to use robot.txt?	dean1122	Search Engine Optimization	11	01-05-2013 02:10 AM

All times are GMT -7. The time now is 02:42 AM.