hello friend, robots.txt is a text file resides in root DIR of website. It allow us to block complete or partial website portion. Make Sure to use a correct syntax of robots.txt. Almost all major search engines like Google, Bing,
http://www.oko69.com ...... respect the presence of robots.txt.
thanks