30 Oct
2015
30 Oct
'15
2:37 p.m.
> They literally refused to understand that a
> dataset could exist that would be too large to fit into memory.
On Fri, 30 Oct
2015, ANDY HOLT wrote:
In the UK, the Home Secretary wants to force all ISPs
to store and
keep (reasonably) easily searchable logs of all URLs accessed by all
their customers.
This, I think, is an example of "big data" ? possibly so big that backup
is impracticable.
When I did daily analysis of the urls accessed by the 2000 or so students
at a University*, that daily database was pushing what I could get into memory
on my desktop system (well specified) ? now imagine 20 million users for 365 days -
I'd be doubtful
that even todays mainframes and supercomputers could do that.
* When I questioned the legality (and morality) of doing this the answer was
"we believe it is legal" (and "stop complaining and keep digging").
That would have been a much better example to bring up with the
Chancellor, since the Peralta Colleges administrators would LOVE
to keep track of all student computer usage!
He would have thought that it was a trivial requirement until walking him
through the arithmetic.
At one time, they had a "Chief Information Officer" who was joining
[temporarily?] various Yahoo email groups for the sole purpose of access
to archives to look for faculty posting "inappropriate" content (any
defamation of Peralta administration!) Yes, John Wxxxxxxx, eat shit and
die.