meta name=”robots” and robots.txt

indexed though blocked by robots.txt

While talking to bloggers and other webmasters, I find out that often the concepts for the meta-tag robots (meta name=”robots” …) and the robots.txt file are not understood and not seldom used in a way that hurts the rankings of the website. Index, noindex, follow, nofollow and┬ácombinations The meta-tag “robots” give you the ability to … Read more