you dont have to worry about this unless you were not following any blackhat seo techniques. If you have placed links to your site in completely irrelevant or low ranking domains, you may be penalized, but its too soon to tell. Monitor your visitors and try to follow a legitimate back linking strategy, this should resolve the issue.
|