hello friend, Robots.txt is a text file resides in root DIR of host website. It permits us to block complete or partial website portion. It has three parts i,e. user agent, disallow section and sitemaps declarations section. Almost all major search engines like Google, Bing,
http://www.oko69.com ...... respect the presence of robots.txt.