|
Notices |
|
Thread Tools | Rate Thread | Display Modes |
![]() |
#3 |
Registered User
Join Date: Aug 2014
Posts: 14
|
XML is containing all sites links and Search engine will use to crawl and index all links from that XML. It really help in indexing all your website's links.
While robots.txt is used to tell search engines crawler that which page they should crawl or which page they should not crawl. |
![]() |
![]() |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
Evil bots consuming all my sites traffic even after i disallowed them in robots.txt | Fking | General Discussion | 1 | 12-13-2013 11:27 PM |
How to edit virtual robots.txt file in wordpress? | geniusoptimizer | PHP / mySQL | 1 | 04-12-2013 12:36 PM |
The robots.txt file | rubberfender | Yahoo | 0 | 12-02-2012 07:32 PM |
Unable to delete robots.txt file | notfake | Yahoo | 0 | 04-29-2012 02:19 AM |
getting hosting but not building a website - want domain to not resolve to anything | ncjbhd | Hosting Discussion | 2 | 11-14-2011 02:50 AM |