Category: SEO

indexed though blocked by robots.txt 0

meta name=”robots” and robots.txt

While talking to bloggers and other webmasters, I find out that often the concepts for the meta-tag robots (meta name=”robots” …) and the robots.txt file are not understood and not seldom used in a way that hurts the rankings of the website. Index, noindex, follow,...