9 Mar
2011
9 Mar
'11
9:36 p.m.
Date: Wed, 09 Mar 2011 12:06:10 -0800
From: Al Kossow <aek at bitsavers.org>
Subject: Re: Who will be the last HD maker down the road?
To: General Discussion: On-Topic and Off-Topic Posts
<cctalk at classiccmp.org>
Message-ID: <4D77DDB2.5030004 at bitsavers.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
On 3/9/11 11:46 AM, vintagecoder at aol.com wrote:
I understand that and it makes sense "deleting" data or trying to overwrite
a filesystem record doesn't necessarily do what we think. But filling
the drive to its capacity with a utility like dd using zeros or random data
has to work because you can read the data back, so it's really there.
I don't know how that can be true. I can
understand deleting doesn't
work but is it true a simple dd if=/dev/zero of=/dev/sdx onto the flash
card doesn't fill it with zeros?
From reading the papers, there are many redundant cells, and the
controller between you and the flash is free to remap and lie about what
it has really done, including continuing to erase cells that have been
marked for garbage collection long after writing has stopped. This invalidates assumptions of repeatability of data
recovery assumed for
forensic evidence.
Interesting and I'm not sure I got what you mean.
Concern from the data security folks is what is really
still there
on the parts if you go directly to the chips and bypass the controller.
Right, for data that was "deleted", but new data that was written has to end
up on the chip somewhere...and this means it really does replace what
was there before, unless there's extra capacity there...in which case some
number of passes of filling it with random data has to work eventually, by
definition. Otherwise it's a bit bucket and not a storage device ;)
I'm concerned about the optimizations they mention
in the controller
firmware geared to proprietary file systems (NTFS). What if these are
accidentally performed on some file system that ISN'T and NTFS file
structure?
I saw a good piece on this point to the issue that most of the cards are
optimized towards FAT32. It explained it may be harmful performance wise to
other file systems that are based on spinning media and write small blocks,
since the flash works best with much larger fixed blocks depending on
manufacturer. I guess we will see some interesting file system development
as people address these issues both from hardware and software angles.
9 Mar
9 Mar
9:51 p.m.
vintagecoder wrote:
I understand that and it makes sense
"deleting" data or trying to
overwrite
a filesystem record doesn't necessarily do what we
think. But filling
the drive to its capacity with a utility like dd using zeros or
random data
has to work because you can read the data back, so
it's really there.
No, that doesn't "have to work". Even though the drive is "filled to
capacity", there still can be spared-out blocks containing old user
data. The capacity that the drive reports is less than the physical
number of blocks on the medium, specifically because there are blocks
reserved for use as spares.
Right, for data that was "deleted", but new
data that was written has
to end
up on the chip somewhere...and this means it really
does replace what
was there before, unless there's extra capacity there...
Which there is.
in which case some number of passes of filling it with
random data
has to work eventually, by definition.
If you do that, eventually enough blocks will fail that the drive can no
longer spare them. It will start reporting errors to the host.
However, this does NOT mean that you have succesfully erased any
previously spared blocks containing remnants of user data. Once they're
spared out, they generally stay spared out forever, so the remnants of
user data will still be there.
Eric
10 Mar
10 Mar
7:13 a.m.
At 11:36 PM 3/9/2011, vintagecoder at aol.com wrote:
I understand that and it makes sense
"deleting" data or trying to overwrite
a filesystem record doesn't necessarily do what we think. But filling
the drive to its capacity with a utility like dd using zeros or random data
has to work because you can read the data back, so it's really there.
If the "file system" was smart, after you did that it would just point you
to one zeroed block all day long, if it knew that's what you wanted to see.
In virtualized environments, there's great advantage to be gained
by letting an OS think it has much more space than it is actually
using. It might look like you have 1 TB, but your disk drive container
file is only as large as the storage you're actually using. This
"thin provisioning" allows many servers to run on a single server,
and reallocate the real storage between a dozen servers that think
they might need it. It's oversubscription at the file system level.
Technological advances and improvements are moving away from one user,
one hard drive, and the one-to-one correspondence between what was once
written and what you might find, if you look deeply at the low level.
Forensics may become much more difficult because of it. Cloud storage
throws it a loop, too. Removable storage does, too, along with encrypted
file systems within file systems, or wireless network storage, etc.
- John
5036
days inactive
5036
days old
2 comments
3 participants
participants (3)
-
eric＠brouhaha.com -
jfoust＠threedee.com -
vintagecoder＠aol.com