[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Demo of Sputnik with LuaUsers content
- From: Miles Bader <miles@...>
- Date: Fri, 22 Feb 2008 09:50:07 +0900
"Jerome Vuarand" <email@example.com> writes:
>>> However, as with robots.txt, Google honors noindex strictly when it
>>> feels like it.
>> What make you think that?
> I personnally have a blog with a robots.txt disallowing the whole
> domain, and still Google kept some links to it. I had to manually remove
> them in the Google webmaster console.
Did the links predate the robots.txt file being added? I wouldn't be
surprised if google handled robots.txt simply by having their crawler
abort when it saw it, so the effects wouldn't be retroactive.
Apologize, v. To lay the foundation for a future offense.