9 Oct
2003
9 Oct
'03
3:20 p.m.
On Thu, 9 Oct 2003, jim wrote:
Fred Cisin wrote:
I think that allowing Google and other search engines to crawl the
messages is actually a *good* idea. There are a number of mailing lists
that I've dealt with that denied search engines, and only had their own
search function. Those search functions never did work very well.
If not for Google, I'd have not ever found the list. I'm sure I'm not the
only one.
-Toth
I do NOT believe that they are spammers trolling
for addresses, nor
any other nefarious purpose. They seem to be just hapless folk
without a clue.
are the archive sites set to "no robots" so that they are not indexed on
the search engines?
If not, then anyone typing in PDP 11 on google will see all the PDP 11
posts in the archive.
Unless Jay has converted to an ASP or CGI script that won't serve up the
contents to robots.