30 Oct
2015
30 Oct
'15
2:43 p.m.
-----Original Message-----
From: cctalk [mailto:cctalk-bounces at classiccmp.org] On Behalf Of ANDY
HOLT
Sent: 30 October 2015 21:22
To: General Discussion: On-Topic and Off-Topic Posts
<cctalk at classiccmp.org>
Subject: Re: "Demystification" is just as important as "Abstraction"
in
"Computer Science" (Was: Know any Fortran
From: "Fred Cisin" <cisin at xenosoft.com>
About a year ago, I was asked by my employer to give them a list of the URLs accessed by
one member of staff. I sent them a list of the URL's that I had accessed.
It was totally useless, mainly because every page these day as is a composite. Add to that
allowing Outlook to download the content of an HTML e-mail ..
.. its almost a totally pointless exercise....
They literally refused to understand that a
dataset could exist that
would be too large to fit into memory.
In the UK, the Home Secretary wants to force all ISPs to store and keep
(reasonably) easily searchable logs of all URLs accessed by all their customers.
This, I think, is an example of "big data" ? possibly so big that backup is
impracticable.
When I did daily analysis of the urls accessed by the 2000 or so students at a
University*, that daily database was pushing what I could get into memory
on my desktop system (well specified) ? now imagine 20 million users for 365
days - I'd be doubtful that even todays mainframes and supercomputers
could do that.
* When I questioned the legality (and morality) of
doing this the answer was
"we believe it is legal" (and "stop complaining and keep digging").
Andy