25 Mar
2018
25 Mar
'18
10:32 p.m.
The new sysadmin at work is clearing out closets full of junk^H^H^H^H
cool old stuff accumulated by the previous sysadmin. There's a big
carton full of PATA hard disks. Most of them are in the 4.3 GB - 20 GB
range, a few larger, a few smaller.
Anyone have any use for these? You can have them for the cost of
shipping, or free for local pickup in Bothell, WA. They're going to be
recycled as scrap if I don't find a home for them.
25 Mar
25 Mar
11:29 p.m.
those are the ibm server ones right?
On Sun, Mar 25, 2018 at 9:32 PM, Guy N. via cctalk <cctalk at classiccmp.org>
wrote:
The new sysadmin at work is clearing out closets full
of junk^H^H^H^H
cool old stuff accumulated by the previous sysadmin. There's a big
carton full of PATA hard disks. Most of them are in the 4.3 GB - 20 GB
range, a few larger, a few smaller.
Anyone have any use for these? You can have them for the cost of
shipping, or free for local pickup in Bothell, WA. They're going to be
recycled as scrap if I don't find a home for them.
26 Mar
26 Mar
8:05 a.m.
On Sun, Mar
25, 2018 at 9:32 PM, Guy N. via cctalk <cctalk at classiccmp.org>
wrote:
The new sysadmin at work is clearing out closets full of junk^H^H^H^H
cool old stuff accumulated by the previous sysadmin. There's a big
carton full of PATA hard disks. Most of them are in the 4.3 GB - 20 GB
range, a few larger, a few smaller.
On 26 March 2018 at 05:29, Adrian Stoness via cctalk <cctalk at classiccmp.org>
wrote:
those are the ibm server ones right?
9:22 a.m.
I've had more replies to this offer than I expected! I'll make a
general reply here, and contact everyone who responded off-list.
I think the simplest and most cost-effective way to ship these is a USPS
Priority Mail flat rate box.
I have to wipe the drives before I ship them -- my word of honor to the
sysadmin. I have probably a couple dozen done now that I can ship right
away. The rest will trickle out a little bit slower after that.
When I said "big carton" I meant it. There are plenty for everyone.
1:34 p.m.
On Mon, Mar 26, 2018 at 06:22:04AM -0700, Guy N. via cctalk wrote:
[...]
I have to wipe the drives before I ship them -- my
word of honor to the
sysadmin. I have probably a couple dozen done now that I can ship right
away. The rest will trickle out a little bit slower after that.
When I said "big carton" I meant it. There are plenty for everyone.
I have heard good things of MHDD diagnostic/repair program in the
context of low level format and generally checking health of spinning
disks. Strangely, I have never had a chance to use it, so far. It
seems like there is freeware version, able to boot from CD?
HTH
--
Regards,
Tomasz Rola
--
** A C programmer asked whether computer had Buddha's nature. **
** As the answer, master did "rm -rif" on the programmer's home **
** directory. And then the C programmer became enlightened... **
** **
** Tomasz Rola mailto:tomasz_rola at bigfoot.com **
1:44 p.m.
On 26 March 2018 at 19:34, Tomasz Rola via cctalk <cctalk at classiccmp.org> wrote:
I have heard good things of MHDD diagnostic/repair program in the
context of low level format and generally checking health of spinning
disks.
That's a new one on me.
I have occasionally used, and often recommend, DBAN -- Darik's Boot And Nuke.
https://dban.org/
--
Liam Proven ? Profile: https://about.me/liamproven
Email: lproven at cix.co.uk ? Google Mail/Hangouts/Plus: lproven at gmail.com
Twitter/Facebook/Flickr: lproven ? Skype/LinkedIn: liamproven
UK: +44 7939-087884 ? ?R (+ WhatsApp/Telegram/Signal): +420 702 829 053
9:24 p.m.
On Mon, Mar 26, 2018 at 07:44:58PM +0200, Liam Proven via cctalk wrote:
On 26 March 2018 at 19:34, Tomasz Rola via cctalk
<cctalk at classiccmp.org> wrote:
Interesting. But judging from descriptions, mhdd is very different
tool/beast. Not that I can recommend one over another, having no
experience (when in trouble, I just ran dd over the disk or parts of
it).
http://hddguru.com/software/2005.10.02-MHDD/
--
Regards,
Tomasz Rola
--
** A C programmer asked whether computer had Buddha's nature. **
** As the answer, master did "rm -rif" on the programmer's home **
** directory. And then the C programmer became enlightened... **
** **
** Tomasz Rola mailto:tomasz_rola at bigfoot.com **
I have heard good things of MHDD diagnostic/repair program in the
context of low level format and generally checking health of spinning
disks.
That's a new one on me.
I have occasionally used, and often recommend, DBAN -- Darik's Boot And Nuke.
https://dban.org/ 
2:18 p.m.
On 03/25/2018 07:32 PM, Guy N. via cctalk wrote:
The new sysadmin at work is clearing out closets full
of junk^H^H^H^H
cool old stuff accumulated by the previous sysadmin. There's a big
carton full of PATA hard disks. Most of them are in the 4.3 GB - 20 GB
range, a few larger, a few smaller.
Anyone have any use for these? You can have them for the cost of
shipping, or free for local pickup in Bothell, WA. They're going to be
recycled as scrap if I don't find a home for them.
I'm surprised that your sysadmin is allowing this to happen. Many large
outfits have a policy of sending any hard disk, regardless of content to
the industrial shredders.
--Chuck
2:23 p.m.
New subject: R: PATA hard disks, anyone?
Well, but with dban and/or other certified software the drives are as well as new. The
only issue would be the time/cost to sanitize them in house.
-----Messaggio originale-----
Da: cctalk [mailto:cctalk-bounces at classiccmp.org] Per conto di Chuck Guzis via cctalk
Inviato: luned? 26 marzo 2018 20:18
A: Guy N. via cctalk
Oggetto: Re: PATA hard disks, anyone?
On 03/25/2018 07:32 PM, Guy N. via cctalk wrote:
The new sysadmin at work is clearing out closets full
of junk^H^H^H^H
cool old stuff accumulated by the previous sysadmin. There's a big
carton full of PATA hard disks. Most of them are in the 4.3 GB - 20
GB range, a few larger, a few smaller.
Anyone have any use for these? You can have them for the cost of
shipping, or free for local pickup in Bothell, WA. They're going to
be recycled as scrap if I don't find a home for them.
I'm surprised that your sysadmin is allowing this to happen. Many large outfits have
a policy of sending any hard disk, regardless of content to the industrial shredders.
--Chuck
3:05 p.m.
New subject: R: PATA hard disks, anyone?
On 03/26/2018 11:23 AM, Mazzini Alessandro wrote:
Well, but with dban and/or other certified software
the drives are as well as new. The only issue would be the time/cost to sanitize them in
house.
Certainly, but it's fruitless to use logic in cases such as these.
Chances are that someone once read the paper from the 1990s that said it
was possible to recover overwritten data from a drive using, IIRC, an
STM--at a rate of what was it? 1 kbit per hour?
--Chuck
3:26 p.m.
New subject: R: PATA hard disks, anyone?
Certainly, but it's fruitless to use logic in
cases such as these.
Chances are that someone once read the paper from the 1990s that said it
was possible to recover overwritten data from a drive using, IIRC, an
STM--at a rate of what was it? 1 kbit per hour?
AFAIK there has been a bounty out to recover data with a single wipe that
hasn't been collected. I thought it was all theory and never done in
practice?
--
: Ethan O'Toole
4:08 p.m.
New subject: R: PATA hard disks, anyone?
On 03/26/2018 12:26 PM, ethan at 757.org wrote:
AFAIK there has been a bounty out to recover data with
a single wipe
that hasn't been collected. I thought it was all theory and never done
in practice?
Here's the Gutmann paper that many people cite:
https://www.usenix.org/legacy/publications/library/proceedings/sec96/full_p…
This may be applicable to early hard drives, but probably not practical
with later hard drives. However, like a bad penny, the legend has
persisted and IT management has, ever since, been irrationally paranoid.
--Chuck
4:09 p.m.
New subject: R: PATA hard disks, anyone?
> Certainly, but it's fruitless to use logic in
cases such as these.
> Chances are that someone once read the paper from the 1990s that said it
> was possible to recover overwritten data from a drive using, IIRC, an
> STM--at a rate of what was it? 1 kbit per hour?
On Mon, 26 Mar 2018, Ethan via cctalk wrote:
AFAIK there has been a bounty out to recover data with
a single wipe that
hasn't been collected. I thought it was all theory and never done in
practice?
NSA (No Such Agency) at Fort Meade, Maryland, does not discuss how much
success they have had with their extensive research into data recovery.
It is hard to make the data unrecoverable.
It is easy to make the recovery cost exceed the value of the data.
--
Grumpy Ol' Fred cisin at xenosoft.com
4:13 p.m.
New subject: R: PATA hard disks, anyone?
On 03/26/2018 04:09 PM, Fred Cisin via cctalk wrote:
NSA has been successful enough that the concept "DOD Wipe" as was found
in programs like Gformat from Ghost no longer exists.
bill
> Certainly,
but it's fruitless to use logic in cases such as these.
> Chances are that someone once read the paper from the 1990s that
> said it
> was possible to recover overwritten data from a drive using, IIRC, an
> STM--at a rate of what was it? 1 kbit per hour?
On Mon, 26 Mar 2018, Ethan via cctalk wrote:
AFAIK there has been a bounty out to recover data
with a single wipe
that hasn't been collected. I thought it was all theory and never
done in practice?
NSA (No Such Agency) at Fort Meade, Maryland, does not discuss how
much success they have had with their extensive research into data
recovery.
5:58 p.m.
New subject: R: PATA hard disks, anyone?
NSA has been successful enough that the concept
"DOD Wipe" as was found
in programs like Gformat from Ghost no longer exists.
Bill,
Are you saying that the NSA can recover data from a HDD that has been erased and been
written over multiple times?
-Ali
8:17 p.m.
New subject: R: PATA hard disks, anyone?
On 03/26/2018 05:58 PM, Ali via cctalk wrote:
That's what we were told when I was still with DOD.? No number of overwrites
made it impossible.? No disk that ever contained classified information
could
ever be let out of containment.? Other than melting them down the only
method
I ever heard of or saw that could truly erase a disk was passing them
through
the ship deguasser at the Navy Yard.? And, after that the disk was not
only clean,
it was unusable.
bill
NSA has been
successful enough that the concept "DOD Wipe" as was found
in programs like Gformat from Ghost no longer exists.
Bill,
Are you saying that the NSA can recover data from a HDD that has been erased and been
written over multiple times?
4:14 p.m.
Only somebody working for the NSA would bother trying to do that. Going from
theory to practice can be VERY expensive and time consuming.
-----Original Message-----
From: Ethan via cctalk
Sent: Monday, March 26, 2018 3:26 PM
To: Chuck Guzis ; General Discussion: On-Topic and Off-Topic Posts
Subject: Re: R: PATA hard disks, anyone?
Certainly, but it's fruitless to use logic in
cases such as these.
Chances are that someone once read the paper from the 1990s that said it
was possible to recover overwritten data from a drive using, IIRC, an
STM--at a rate of what was it? 1 kbit per hour?
AFAIK there has been a bounty out to recover data with a single wipe that
hasn't been collected. I thought it was all theory and never done in
practice?
--
: Ethan O'Toole
---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
27 Mar
27 Mar
12:05 a.m.
On Mon, 2018-03-26 at 11:18 -0700, Chuck Guzis via cctalk wrote:
I'm surprised that your sysadmin is allowing this
to happen. Many large
outfits have a policy of sending any hard disk, regardless of content to
the industrial shredders.
The advantages of working for a small company... the sysadmin is a
long-time employee who's just moved into that role, he and I are good
buddies. And there's not anything worth $$$ data recovery on them
anyway.
I hate seeing perfectly good working equipment reduced to low-value
scrap, so I'm wiping these drives at home on my own time to prevent
that.
12:19 p.m.
The advantages of working for a small company... the
sysadmin is a
long-time employee who's just moved into that role, he and I are good
buddies. And there's not anything worth $$$ data recovery on them
anyway.
I hate seeing perfectly good working equipment reduced
to low-value
scrap, so I'm wiping these drives at home on my own time to prevent
that.
Yep! I've watched thousand(s) of pounds of working hard drives get
shredded.
I have a hook-up to get some older drives from another company (1.5TB,
etc) and well... let's just say that "newer" used disks with 4 years on
them aren't very reliable. I'd imaigne the older ones hold up much better
since they were more expensive and less density.
--
: Ethan O'Toole
1:46 p.m.
On 03/27/2018 09:19 AM, Ethan via cctalk wrote:
Yep! I've watched thousand(s) of pounds of working
hard drives get
shredded.
This is nothing new. In the 1970s, the official CDC diktat was to
reduce any surplused equipment to scrap. That included taking a
sledgehammer to disk drives and other objects. I witnessed lots of
functional gear done in this way.
Apparently this was to prevent surplus parts finding their way into the
supply stream--and then having to service them.
--Chuck
3:19 p.m.
New subject: R: PATA hard disks, anyone?
"that" bad ? :S .....
-----Messaggio originale-----
Da: cctalk [mailto:cctalk-bounces at classiccmp.org] Per conto di Eric Smith via cctalk
Inviato: marted? 27 marzo 2018 20:48
A: ethan at 757.org; General Discussion: On-Topic and Off-Topic Posts
Oggetto: Re: PATA hard disks, anyone?
On Tue, Mar 27, 2018 at 10:19 AM, Ethan via cctalk <cctalk at classiccmp.org>
wrote:
and well... let's just say that "newer"
used disks with 4 years on
them aren't very reliable.
If anyone wants some Seagate ST3000DM001 drives (3TB SATA), I've got extras!
:-(
3:21 p.m.
Are they functional or decorative?
From: "cctalk" <cctalk at classiccmp.org>
To: ethan at 757.org, "cctalk" <cctalk at classiccmp.org>
Sent: Tuesday, March 27, 2018 11:48:22 AM
Subject: Re: PATA hard disks, anyone?
On Tue, Mar 27, 2018 at 10:19 AM, Ethan via cctalk <cctalk at classiccmp.org>
wrote:
and well... let's just say that "newer"
used disks with 4 years on them
aren't very reliable.
If anyone wants some Seagate ST3000DM001 drives (3TB SATA), I've got extras!
:-(
3:34 p.m.
From: "cctalk" <cctalk at
classiccmp.org>
To: ethan at 757.org, "cctalk" <cctalk at classiccmp.org>
Sent: Tuesday, March 27, 2018 11:48:22 AM
Subject: Re: PATA hard disks, anyone?
On Tue, Mar 27, 2018 at 10:19 AM, Ethan via cctalk <cctalk at classiccmp.org>
wrote:
I bet they'd fit great in a clay pidgeon thrower. :)
g.
--
Proud owner of F-15C 80-0007
http://www.f15sim.com - The only one of its kind.
http://www.diy-cockpits.org/coll - Go Collimated or Go Home.
Some people collect things for a hobby. Geeks collect hobbies.
ScarletDME - The red hot Data Management Environment
A Multi-Value database for the masses, not the classes.
http://scarlet.deltasoft.com - Get it _today_!
and well... let's just say that
"newer" used disks with 4 years on them
aren't very reliable.
If anyone wants some Seagate ST3000DM001 drives (3TB SATA), I've got extras!
:-(
4:05 p.m.
Are they functional or decorative?
3TB Seagate
They will likely fail. Defective model. Know someone that doesn't even RMA
them, straight to trash. Replaces them with WD.
(Note that all Seagate models have the issue, just something wrong with
a 3TB model.)
- Ethan
--
: Ethan O'Toole
4:02 p.m.
Eric,
my 17 yo son is building up a gaming computer out of a Mac Pro. If you are serious about
getting rid of these, I might encourage him to set up a Raid 5 with 3 of them and 1 or 2
spares. I would think it would read and write pretty fast until it broke, and then he
could transition the data to a more robust drive or set of drives. Meantime, if I?m wrong,
it?s not very critical data to lose.
Let me know if that seems like a reasonable deal, and if so what you?d want above
shipping.
- Mark
210-522-6025 office 210-379-4635 cell
On Mar 27, 2018, at 1:48 PM, Eric Smith via cctalk <cctalk at classiccmp.org>
wrote:
On Tue, Mar 27, 2018 at 10:19 AM, Ethan via cctalk
<cctalk at classiccmp.org>
wrote:
and well... let's just say that
"newer" used disks with 4 years on them
aren't very reliable.
If anyone wants some Seagate ST3000DM001 drives (3TB SATA), I've got extras!
:-(
7:43 p.m.
Digging around on the pointer from Al to backblaze, I found this, which,
to me is far more meaningful in terms of presentation of data:
https://hackernoon.com/applying-medical-statistics-to-the-backblaze-hard-dr…
--Chuck
8:51 p.m.
New subject: RAID? Was: PATA hard disks, anyone?
Well outside my realm of expertise (as if I had a realm!), . . .
How many drives would you need, to be able to set up a RAID, or hot
swappable RAUD (Redundant Array of Unreliable Drives), that could give
decent reliability with such drives?
How many to be able to not have data loss if a second one dies before the
first casualty is replaced?
How many to be able to avoid data loss if a third one dies before the
first two are replaced?
How many technicians with shopping carts full of spare drives?
(cf. early vaccum tube computers?)
28 Mar
28 Mar
6:57 a.m.
New subject: RAID? Was: PATA hard disks, anyone?
On 28 March 2018 at 02:51, Fred Cisin via cctalk <cctalk at classiccmp.org> wrote:
Well outside my realm of expertise (as if I had a
realm!), . . .
How many drives would you need, to be able to set up a RAID, or hot
swappable RAUD (Redundant Array of Unreliable Drives), that could give
decent reliability with such drives?
For what little it's worth, and little relevance, I had a hardware
RAID of 6 ? 80GB UltraIDE drives that ran faultlessly for several
years. Not full-time, to be fair, but surviving multiple power cycles.
In an old HP Proliant ML110 G1 -- space-heater Pentium 4 version --
with only about 2GB of RAM (because it used some kind of weird
expensive ECC RAM) and a hacked copy of Windows Server 2008.
It was a box made from free leftovers and proved to be one of my most
reliable workhorse PCs ever. Figures, really.
--
Liam Proven ? Profile: https://about.me/liamproven
Email: lproven at cix.co.uk ? Google Mail/Hangouts/Plus: lproven at gmail.com
Twitter/Facebook/Flickr: lproven ? Skype/LinkedIn: liamproven
UK: +44 7939-087884 ? ?R (+ WhatsApp/Telegram/Signal): +420 702 829 053
9:33 a.m.
New subject: RAID? Was: PATA hard disks, anyone?
On Mar 27, 2018, at 8:51 PM, Fred Cisin via cctalk
<cctalk at classiccmp.org> wrote:
Well outside my realm of expertise (as if I had a realm!), . . .
How many drives would you need, to be able to set up a RAID, or hot swappable RAUD
(Redundant Array of Unreliable Drives), that could give decent reliability with such
drives?
How many to be able to not have data loss if a second one dies before the first casualty
is replaced?
How many to be able to avoid data loss if a third one dies before the first two are
replaced?
These are straightforward questions of probability math, but it takes some time to get the
details right. For one thing, you need believable numbers for the underlying error
probabilities. And you have to analyze the cases carefully.
The basic assumption is that failures are "fail stop", i.e., a drive refuses to
deliver data. (In particular, it doesn't lie -- deliver wrong data. You can build
systems that deal with lying drives but RAID is not such a system.) The failure may be
the whole drive ("it's a door-stop") or individual blocks (hard read
errors).
In either case, RAID-1 and RAID-5 handle single faults. RAID-6 isn't a single
well-defined thing but as normally defined it is a system that handles double faults. So
a RAID-1 system with a double fault may fail to give you your data. (It may also be ok --
it depends on where the faults are.) RAID-5 ditto.
The tricky part is what happens when a drive breaks. Consider RAID-5 with a single dead
drive, and the others are 100% ok. Your data is still good. When the broken drive is
replaced, RAID rebuilds the bits that belong on that drive. Once that rebuild finishes,
you're once again fault tolerant. But a second failure prior to rebuild completion
means loss of data.
So one way to look at it: given the MTBF, calculate the probability of two drives failing
within N hours (where N is the time required to replace the failed drive and then rebuild
the data onto the new drive). But that is not the whole story.
The other part of the story is that drives have a non-zero probability of a hard read
error. So during rebuild, you may encounter a sector on one of the remaining drives that
can't be read. If so, that sector is lost.
The probability of hard read error varies with drive technology. And of course, the
larger the drive, the greater the probability (all else being equal) of having SOME sector
be unreadable. For drives small enough to have PATA interfaces, the probability of hard
read error is probably low enough that you can *usually* read the whole drive without
error. That translates to: RAID-1 and RAID-5 are generally adequate for PATA disks.
On the very large drives currently available, it's a different story, and the
published drive specs make this quite clear. This is why RAID-6 is much more popular now
than it was earlier. It isn't the probability of two nearly simultaneous drive
failures, but rather the probability of a hard sector read error while a drive has failed,
that argues for the use of RAID-6 in modern storage systems.
paul
1:02 p.m.
New subject: RAID? Was: PATA hard disks, anyone?
On 03/28/2018 06:33 AM, Paul Koning via cctalk wrote:
These are straightforward questions of probability
math, but it takes
some time to get the details right. For one thing, you need
believable numbers for the underlying error probabilities. And you
have to analyze the cases carefully.
A great discussion, Paul!
My biggest danger to data loss is--me. I've modified my working
procedures as much as possible and keep backups, but every once in a
while, I'll do something stupid, such as "rm -rf *", in the mistaken
assumption that the system I'm on contains a .bashrc file with "alias
rm='rm -i'" in it somewhere.
I know of no RAID setup that can save me from stupid.
--Chuck
2:32 p.m.
New subject: RAID? Was: PATA hard disks, anyone?
> How many drives would you need, to be able to set
up a RAID, or hot swappable RAUD (Redundant Array of Unreliable Drives), that could give
decent reliability with such drives?
> How many to be able to not have data loss if a second one dies before the first
casualty is replaced?
> How many to be able to avoid data loss if a third one dies before the first two are
replaced?
On Wed, 28 Mar 2018, Paul Koning wrote:
These are straightforward questions of probability
math, but it takes
some time to get the details right. For one thing, you need believable
numbers for the underlying error probabilities. And you have to analyze
the cases carefully.
THANK YOU for the detailed explanation!
The basic assumption is that failures are "fail
stop", i.e., a drive
refuses to deliver data. (In particular, it doesn't lie -- deliver
wrong data. You can build systems that deal with lying drives but RAID
is not such a system.) The failure may be the whole drive ("it's a
door-stop") or individual blocks (hard read errors).
So, in addition to the "RAID" configuration, you would also need
additional redundancy to compare multiple reads for error detection.
At the simplest level, if the reads don't match, then there is an error.
If a retry produces different dataa, then that drive has an error.
If two drives agree against a third, then there is a high probability that
the variant drive is in error.
In either case, RAID-1 and RAID-5 handle single
faults. RAID-6 isn't a
single well-defined thing but as normally defined it is a system that
handles double faults. So a RAID-1 system with a double fault may fail
to give you your data. (It may also be ok -- it depends on where the
faults are.) RAID-5 ditto.
The tricky part is what happens when a drive breaks. Consider RAID-5
with a single dead drive, and the others are 100% ok. Your data is
still good. When the broken drive is replaced, RAID rebuilds the bits
that belong on that drive. Once that rebuild finishes, you're once
again fault tolerant. But a second failure prior to rebuild completion
means loss of data.
With very unreliable drives, that isn't acceptable.
If each "drive" within the RAID were itself a RAID, . . .
Getting to be a complicated controller, or cascading controllers, . . .
So one way to look at it: given the MTBF, calculate
the probability of
two drives failing within N hours (where N is the time required to
replace the failed drive and then rebuild the data onto the new drive).
But that is not the whole story.
'course not. Besides MTBF for calculating the probability of a second
drive failing within N hours, must also consider other factors, such as
external influences causing more than one drive to go, and the essentially
non-linear aspect of a failure rate curve.
The other part of the story is that drives have a
non-zero probability
of a hard read error. So during rebuild, you may encounter a sector on
one of the remaining drives that can't be read. If so, that sector is
lost.
If we consider that to be a "drive failure", then we are back to designing
around multiple failures.
The probability of hard read error varies with drive
technology. And of
course, the larger the drive, the greater the probability (all else
being equal) of having SOME sector be unreadable. For drives small
enough to have PATA interfaces, the probability of hard read error is
probably low enough that you can *usually* read the whole drive without
error. That translates to: RAID-1 and RAID-5 are generally adequate for
PATA disks.
"generally".
The original thought behind this silly suggestion was whether it would be
possible to make use of MANY very unreliable drives.
On the very large drives currently available, it's
a different story,
and the published drive specs make this quite clear. This is why RAID-6
is much more popular now than it was earlier. It isn't the probability
of two nearly simultaneous drive failures, but rather the probability of
a hard sector read error while a drive has failed, that argues for the
use of RAID-6 in modern storage systems.
4:57 p.m.
New subject: RAID? Was: PATA hard disks, anyone?
On Mar 28, 2018, at 2:32 PM, Fred Cisin via cctalk
<cctalk at classiccmp.org> wrote:
If you don't trust drives to deliver correct data often enough, you need your own
error detection. Comparing redundant copies is possible. More efficient is various EDC
or ECC codes. Some file systems used hashes like SHA-1 to detect data corruption with
extremely high probability.
> How many drives would you need, to be able to
set up a RAID, or hot swappable RAUD (Redundant Array of Unreliable Drives), that could
give decent reliability with such drives?
> How many to be able to not have data loss if a second one dies before the first
casualty is replaced?
> How many to be able to avoid data loss if a third one dies before the first two are
replaced?
On Wed, 28 Mar 2018, Paul Koning wrote:
...
The basic assumption is that failures are
"fail stop", i.e., a drive refuses to deliver data. (In particular, it
doesn't lie -- deliver wrong data. You can build systems that deal with lying drives
but RAID is not such a system.) The failure may be the whole drive ("it's a
door-stop") or individual blocks (hard read errors).
So, in addition to the "RAID" configuration, you would also need additional
redundancy to compare multiple reads for error detection.
At the simplest level, if the reads don't match, then there is an error.
If a retry produces different dataa, then that drive has an error.
If two drives agree against a third, then there is a high probability that the variant
drive is in error. ...
Yes, RAID has an underlying assumption that drive failures are independent random events.
If that isn't valid then you have a big problem. This occasionally happens; there
have been drive enclosures with inadequate mechanical design, resulting in excessive
vibration which caused rapid and correlated drive failure. The answer to that is
"test it properly and don't ship stuff like that".
So one way to look at it: given the MTBF,
calculate the probability of two drives failing within N hours (where N is the time
required to replace the failed drive and then rebuild the data onto the new drive). But
that is not the whole story.
'course not. Besides MTBF for calculating the probability of a second drive failing
within N hours, must also consider other factors, such as external influences causing more
than one drive to go, and the essentially non-linear aspect of a failure rate curve.
The other part
of the story is that drives have a non-zero probability of a hard read error. So during
rebuild, you may encounter a sector on one of the remaining drives that can't be read.
If so, that sector is lost.
If we consider that to be a "drive failure", then we are back to designing
around multiple failures. The probability of hard read error varies with
drive technology. And of course, the larger the drive, the greater the probability (all
else being equal) of having SOME sector be unreadable. For drives small enough to have
PATA interfaces, the probability of hard read error is probably low enough that you can
*usually* read the whole drive without error. That translates to: RAID-1 and RAID-5 are
generally adequate for PATA disks.
"generally".
The original thought behind this silly suggestion was whether it would be possible to
make use of MANY very unreliable drives. 
6:06 p.m.
New subject: RAID? Was: PATA hard disks, anyone?
On 03/28/2018 12:32 PM, Fred Cisin via cctalk wrote:
With very unreliable drives, that isn't
acceptable. If each "drive"
within the RAID were itself a RAID, . . . Getting to be a complicated
controller, or cascading controllers, . . .
Many of the SCSI / SAS RAID controllers that I've worked with over the
last 10+ years have this cascading controller functionality. Most of
the RAID controlelrs that I've worked with would let you build a mirror
or stripe across some sort of underlying RAID. Typical examples are
striping (RAID 0) across mirrors (RAID 1) or multiple RAID 5 arrays.
'course not.? Besides MTBF for calculating the
probability of a second
drive failing within N hours, must also consider other factors, such as
external influences causing more than one drive to go, and the
essentially non-linear aspect of a failure rate curve.
You also need to take into
account the additional I/O load imposed on
the remaining drives during a rebuild.
I usedto routinely run into software (Solstice Disk Suite?) RAID 1
mirrors on Solaris boxen for the OS (/) where different parts of each
drive would fail. So we'd end up with a situation where we had a decent
RAID, but we couldn't replace either disk. This usually involved taking
an entire backup of the machine, replacing both disks, and restoring the
data.
--
Grant. . . .
unix || die
5:22 p.m.
New subject: RAID? Was: PATA hard disks, anyone?
On Wed, Mar 28, 2018 at 09:33:38AM -0400, Paul Koning via cctalk wrote:
[...]
The basic assumption is that failures are "fail
stop", i.e., a drive refuses
to deliver data. (In particular, it doesn't lie -- deliver wrong data. You
can build systems that deal with lying drives but RAID is not such a system.)
The failure may be the whole drive ("it's a door-stop") or individual
blocks
(hard read errors).
The assumption that disks don't lie is demonstrably false, and anybody who
still designs or sells a system in 2018 which makes that assumption is a
charlatan. I have hardware which proves it.
Sun's ZFS filesystem applies an extra "trust but verify" layer of
protection
using strong checksums. I have a server with a pair of mirrored 3TB enterprise
disks which are "zfs scrub"bed (surface-scanned and checksums verified) weekly.
Every few months, the scrub will hit a bad checksum which shows that the disk
read back different data to that which was written, even though the disk
claimed the read was OK. At best (and most likely) the problem was a single bit
flip, i.e. roughly a 1 in 1.8e13 error rate. So much for the manufacturer's
claim of less than 1 in 1e15 for that model of disk.
A workstation with a pair of 512GB consumer-grade SSDs has a half-dozen bad
stripes in every scrub performed after the machine has been powered down for a
week or so. The SSDs have just a few hundred hours on the clock and perhaps
three full drive writes. I love the performance of SSDs, but they are
appallingly unreliable for even medium-term storage.
Fortunately, ZFS can tell from the checksums which half of the mirror is lying,
and thus rewrite the stripe based on the known-good copy. It even handles the
case where both disks have some errors. Traditional RAID just cannot self-heal
like that.
8:40 p.m.
New subject: RAID? Was: PATA hard disks, anyone?
Hello all,
I have been kind of following this thread. I have a question about
MTBF. I have four HGST UltraStar Enterprise 2TB drives setup in a
Hardware RAID 10 configuration. If the the MTBF is 100,000 Hrs for each
drive does this mean that the total MTBF is 25,000 Hrs?
GOD Bless and Thanks,
rich!
On 3/28/2018 6:33 AM, Paul Koning via cctalk wrote:
On Mar 27, 2018, at 8:51 PM, Fred Cisin via
cctalk <cctalk at classiccmp.org> wrote:
Well outside my realm of expertise (as if I had a realm!), . . .
How many drives would you need, to be able to set up a RAID, or hot swappable RAUD
(Redundant Array of Unreliable Drives), that could give decent reliability with such
drives?
How many to be able to not have data loss if a second one dies before the first casualty
is replaced?
How many to be able to avoid data loss if a third one dies before the first two are
replaced?
These are straightforward questions of probability math, but it takes
some time to get the details right. For one thing, you need believable numbers for the
underlying error probabilities. And you have to analyze the cases carefully.
The basic assumption is that failures are "fail stop", i.e., a drive refuses to
deliver data. (In particular, it doesn't lie -- deliver wrong data. You can build
systems that deal with lying drives but RAID is not such a system.) The failure may be
the whole drive ("it's a door-stop") or individual blocks (hard read
errors).
In either case, RAID-1 and RAID-5 handle single faults. RAID-6 isn't a single
well-defined thing but as normally defined it is a system that handles double faults. So
a RAID-1 system with a double fault may fail to give you your data. (It may also be ok --
it depends on where the faults are.) RAID-5 ditto.
The tricky part is what happens when a drive breaks. Consider RAID-5 with a single dead
drive, and the others are 100% ok. Your data is still good. When the broken drive is
replaced, RAID rebuilds the bits that belong on that drive. Once that rebuild finishes,
you're once again fault tolerant. But a second failure prior to rebuild completion
means loss of data.
So one way to look at it: given the MTBF, calculate the probability of two drives failing
within N hours (where N is the time required to replace the failed drive and then rebuild
the data onto the new drive). But that is not the whole story.
The other part of the story is that drives have a non-zero probability of a hard read
error. So during rebuild, you may encounter a sector on one of the remaining drives that
can't be read. If so, that sector is lost.
The probability of hard read error varies with drive technology. And of course, the
larger the drive, the greater the probability (all else being equal) of having SOME sector
be unreadable. For drives small enough to have PATA interfaces, the probability of hard
read error is probably low enough that you can *usually* read the whole drive without
error. That translates to: RAID-1 and RAID-5 are generally adequate for PATA disks.
On the very large drives currently available, it's a different story, and the
published drive specs make this quite clear. This is why RAID-6 is much more popular now
than it was earlier. It isn't the probability of two nearly simultaneous drive
failures, but rather the probability of a hard sector read error while a drive has failed,
that argues for the use of RAID-6 in modern storage systems.
paul
7:43 p.m.
New subject: RAID? Was: PATA hard disks, anyone?
On Wed, 28 Mar 2018, Richard Pope via cctalk wrote:
I have been kind of following this thread. I have a
question about MTBF.
I have four HGST UltraStar Enterprise 2TB drives setup in a Hardware RAID 10
configuration. If the the MTBF is 100,000 Hrs for each drive does this mean
that the total MTBF is 25,000 Hrs?
<pedantic sadistics>
Probably NOT.
It depends extremely heavily on the shape of the curve of failure times.
MEAN Time Before Failure, of course, means that for a large enough sample,
half the drives fail before 100,000 hours, and half after. Thus, at
100,000 hours, half are dead.
But, how evenly distributed are the failures?
Besides the MTBF, it would help to know the variance or standard
deviation.
It is unlikely that the failures follow a "normal distribution"
(or "Laplace-Gauss") bell curve. And, other distributions are
certainly not ABnormal :-)
If the curve is symmetrical, then the mean, median, and mode will all be
the same. If it is not symmetrical, then they won't be. Hence the use of
MEDIAN - at that point half are dead, half are still alive.
In toxicology, there is a concept of an LD-50 dosage - the dosage that
will kill half, since for example, antibiotic resistant bacteria might
require an incredibly large dosage to get that last one, but LD-50
provides a convenient way to get a single number.
100,000 hours is the LD-50 of those drives.
If it turns out that the drives last 100,000 hours, plus or minus 10%,
then you have a curve with a very steep slope. It is still half dead at
100,000, but maybe hardly any dead until 90,000, hardly any left alive at
110,000.
OTOH, if the failures were evenly distributed throughout a life of 0 to
200,000 hours, with the same number going every day, then that also would
have a MTBF of 100,000. In THAT case, then yes, the MTBF of first
failure may well be 25,000.
They rarely work that way. Often our devices will have what is sometimes
called a "bathtub curve". There are a few failures IMMEDIATELY ("infant
mortality") falling off rapidly, and then few failures for quite a while,
and then, as random parts start to wear out, the failures rise.
In fact, with the same MTBF of 100,000, it could be that once the early
demise ones are discarded, that the MTBF of the REMAINDER might be
200,000.
IFF you are willing to deal with the DOA and infant mortality cases, then
by discarding or ignoring those outlying numbers, you might get a more
realistic evaluation of what to expect.
</pedantic sadistics>
--
Grumpy Ol' Fred cisin at xenosoft.com
9:57 p.m.
New subject: RAID? Was: PATA hard disks, anyone?
Fred,
I appreciate the explanation. So with out a 1,000, 10,000, or even
100,000 drives there is no way to know how long my drives in the RAID
will last. All I know for sure is that I can lose anyone drive and the
RAID can be rebuilt.
GOD Bless and Thanks,
rich!
On 3/28/2018 4:43 PM, Fred Cisin via cctalk wrote:
On Wed, 28 Mar 2018, Richard Pope via cctalk wrote:
I have been kind of following this thread. I
have a question about
MTBF. I have four HGST UltraStar Enterprise 2TB drives setup in a
Hardware RAID 10 configuration. If the the MTBF is 100,000 Hrs for
each drive does this mean that the total MTBF is 25,000 Hrs?
<pedantic sadistics>
Probably NOT.
It depends extremely heavily on the shape of the curve of failure times.
MEAN Time Before Failure, of course, means that for a large enough
sample, half the drives fail before 100,000 hours, and half after.
Thus, at 100,000 hours, half are dead.
But, how evenly distributed are the failures?
Besides the MTBF, it would help to know the variance or standard
deviation.
It is unlikely that the failures follow a "normal distribution" (or
"Laplace-Gauss") bell curve. And, other distributions are certainly
not ABnormal :-)
If the curve is symmetrical, then the mean, median, and mode will all
be the same. If it is not symmetrical, then they won't be. Hence the
use of MEDIAN - at that point half are dead, half are still alive.
In toxicology, there is a concept of an LD-50 dosage - the dosage that
will kill half, since for example, antibiotic resistant bacteria might
require an incredibly large dosage to get that last one, but LD-50
provides a convenient way to get a single number.
100,000 hours is the LD-50 of those drives.
If it turns out that the drives last 100,000 hours, plus or minus 10%,
then you have a curve with a very steep slope. It is still half dead
at 100,000, but maybe hardly any dead until 90,000, hardly any left
alive at 110,000.
OTOH, if the failures were evenly distributed throughout a life of 0
to 200,000 hours, with the same number going every day, then that also
would have a MTBF of 100,000. In THAT case, then yes, the MTBF of
first failure may well be 25,000.
They rarely work that way. Often our devices will have what is
sometimes called a "bathtub curve". There are a few failures
IMMEDIATELY ("infant mortality") falling off rapidly, and then few
failures for quite a while, and then, as random parts start to wear
out, the failures rise. In fact, with the same MTBF of 100,000, it
could be that once the early demise ones are discarded, that the MTBF
of the REMAINDER might be 200,000.
IFF you are willing to deal with the DOA and infant mortality cases,
then by discarding or ignoring those outlying numbers, you might get a
more realistic evaluation of what to expect.
</pedantic sadistics>
--
Grumpy Ol' Fred cisin at xenosoft.com
8:11 p.m.
New subject: RAID? Was: PATA hard disks, anyone?
It's not quite that bad. The answer is that the MTBF of four drives is probably not
simply the MTBF of one drive divided by four. If you have a good description of the
probability of failure as a function of drive age (i.e., a picture of its particular
"bathtub curve") you can then work out the corresponding curve for multiple
drives. I like to leave the details of how to do this to appropriate mathematicians.
If all you have is a data sheet that says "MTBF is 1M hours" then you don't
have enough information. You can assume some distribution and figure accordingly, but if
the actual distribution is sufficiently different from the guess then the answers you
calculated may be significantly off.
BTW, specified MTBF for modern drives is a whole lot higher than 100k hours. Real MTBF
may differ from specified, and derating the manufacturer's number according to your
preferred level of pessimism is probably a good idea.
paul
On Mar 28, 2018, at 9:57 PM, Richard Pope via cctalk
<cctalk at classiccmp.org> wrote:
Fred,
I appreciate the explanation. So with out a 1,000, 10,000, or even 100,000 drives
there is no way to know how long my drives in the RAID will last. All I know for sure is
that I can lose anyone drive and the RAID can be rebuilt.
GOD Bless and Thanks,
rich!
On 3/28/2018 4:43 PM, Fred Cisin via cctalk wrote:
> On Wed, 28 Mar 2018, Richard Pope via cctalk wrote:
>> I have been kind of following this thread. I have a question about MTBF. I have
four HGST UltraStar Enterprise 2TB drives setup in a Hardware RAID 10 configuration. If
the the MTBF is 100,000 Hrs for each drive does this mean that the total MTBF is 25,000
Hrs?
>
> <pedantic sadistics>
> Probably NOT.
> It depends extremely heavily on the shape of the curve of failure times.
> MEAN Time Before Failure, of course, means that for a large enough sample, half the
drives fail before 100,000 hours, and half after. Thus, at 100,000 hours, half are dead.
>
> But, how evenly distributed are the failures? ...
29 Mar
29 Mar
8:21 a.m.
New subject: RAID? Was: PATA hard disks, anyone?
On Wed, Mar 28, 2018 at 05:40:29PM -0700, Richard Pope via cctalk wrote:
I have been kind of following this thread. I have a
question about MTBF. I
have four HGST UltraStar Enterprise 2TB drives setup in a Hardware RAID 10
configuration. If the the MTBF is 100,000 Hrs for each drive does this mean
that the total MTBF is 25,000 Hrs?
That's the mean time before any one disk fails, but not the MTBF for the array
as a whole because failure of an individual disk doesn't cause the array to
fail. There needs to be at least one more disk failure for that to happen.
MTBF is also an overly simple measure which fails to account for the bathtub
curve and correlated failures. Attempts to compute the MTBF of an array from
the MTBF of the individual components will come up with a plausible number
which is technically correct yet bears no relation to the real world.
In practice, the only numbers on a typical hard disk datasheet which aren't
fantasy marketing puff are the physical dimensions and the number of sectors,
and even that is because those are industry-wide standards that disks must
conform to.
28 Mar
28 Mar
7:06 a.m.
On 28 March 2018 at 01:43, Chuck Guzis via cctalk <cctalk at classiccmp.org> wrote:
Digging around on the pointer from Al to backblaze, I
found this, which,
to me is far more meaningful in terms of presentation of data:
https://hackernoon.com/applying-medical-statistics-to-the-backblaze-hard-dr…
Remarkable and fascinating.
The charts for Seagate are especially reliable.
It came as a surprise to me. In 30y in the business, almost anyone
involved in selecting, specifying, purchasing, or maintaining hardware
inevitably has _strong_ opinions on the reliability, or lack thereof,
of certain brands of hard disk.
Personally, I've used them _all_. I've seen several-decades-old hard
disks working perfectly, I've seen brand new drives fail, I've watched
batches of them die progressively.
They can _all_ fail. I have no angels or demons -- I have seen random
sudden failures of every vendor known to humanity, and superb
longevity from every vendor too.
But if there's a general trend, it's that the bigger, the more
fragile. I have decade-old 300GB drives in routine use that are fine.
I've also had multiple failures of new multi-terabyte-class drives, to
my personal cost.
--
Liam Proven ? Profile: https://about.me/liamproven
Email: lproven at cix.co.uk ? Google Mail/Hangouts/Plus: lproven at gmail.com
Twitter/Facebook/Flickr: lproven ? Skype/LinkedIn: liamproven
UK: +44 7939-087884 ? ?R (+ WhatsApp/Telegram/Signal): +420 702 829 053
27 Mar
27 Mar
3:44 p.m.
On 03/27/2018 09:19 AM, Ethan via cctalk wrote:
I have a hook-up to get some older drives from another
company (1.5TB,
etc) and well... let's just say that "newer" used disks with 4 years on
them aren't very reliable. I'd imaigne the older ones hold up much
better since they were more expensive and less density.
Call it superstition on my part, but for other than a couple of 2TB
drives used for backup (i.e. powered and run only occasionally), my
working drives are mostly 500GB-1TB SATA models.
It's probably foolish and irrational, but I somehow just don't trust the
bargain-basement 4TB drives to perform long-term.
Am I being silly?
--Chuck
3:49 p.m.
On Tue, 27 Mar 2018, Chuck Guzis via cctalk wrote:
It's probably foolish and irrational, but I
somehow just don't trust the
bargain-basement 4TB drives to perform long-term.
Am I being silly?
Not really. Have you looked at the drive statistics published by
Backblaze? Here's a report for the 3rd quarter of 2017:
https://www.backblaze.com/blog/hard-drive-failure-rates-q3-2017/
Seagate drives are the worst out there, while HGST drives seem to last the
longest.
g.
--
Proud owner of F-15C 80-0007
http://www.f15sim.com - The only one of its kind.
http://www.diy-cockpits.org/coll - Go Collimated or Go Home.
Some people collect things for a hobby. Geeks collect hobbies.
ScarletDME - The red hot Data Management Environment
A Multi-Value database for the masses, not the classes.
http://scarlet.deltasoft.com - Get it _today_!
7:04 p.m.
On 3/27/18 12:44 PM, Chuck Guzis via cctalk wrote:
Am I being silly?
Hitachi 3 and 4tb are VERY good, as are the equivalent Toshiba from when WD divested
themselves
of the Hitachi hi-end line.
1tb was the switchover point to vertical recording, so those (and esp seagate 1.5tb) are
terrible
I have a few helium-filled drives spinning. They've been fine, but I don't know
for how long yet.
7:36 p.m.
On 03/27/2018 04:04 PM, Al Kossow via cctalk wrote:
Hitachi 3 and 4tb are VERY good, as are the equivalent Toshiba from when WD divested
themselves
of the Hitachi hi-end line.
Hmmm, I haven't actually installed a 1TB drive in any mission-critical
equipment yet--still sitting in original packaging on a shelf. Mostly,
I use 500GB drives. Haven't had a failure in years.
Maybe I'll leave the drives shelved. I suppose I could donate them to
some deserving kid, but that might be cruel.
--Chuck
9:20 p.m.
Hitachi 3 and 4tb are VERY good, as are the equivalent
Toshiba from when WD divested themselves
of the Hitachi hi-end line.
Been runnning 4 HGST 4TB for a long while now at home and have been really
happy. My best disks.
1tb was the switchover point to vertical recording, so
those (and esp seagate 1.5tb) are terrible
Ah, have a bunch of used 1.5TB seagates and there was about a 70% failure
rate but I assumed it was due to age on disk and going from 24x7 to off
for a few months then back on. Hoping to move off of those onto 30TB of WD
disks soon if the drives aren't totally dead.
I have a few helium-filled drives spinning.
They've been fine, but I don't know for how long yet.
Have a few HGST 8TB He drives at $work, out of 50 or so maybe 1 failure in
2 years. Very solid but our most expensive drives until you get into flash
storage.
I'm interested to hear how the 12TB Seagates perform. A friend I believe
got a few hundred in so it will be interesting in a year to hear if
they're reliable or not.
2 Apr
2 Apr
11:50 a.m.
I'll take them all please. Let me know the cost of shipping. Thanks.
Kip
-----Original Message-----
From: cctalk [mailto:cctalk-bounces at classiccmp.org] On Behalf Of Guy N. via cctalk
Sent: Sunday, March 25, 2018 10:33 PM
To: cctalk at classiccmp.org
Subject: PATA hard disks, anyone?
The new sysadmin at work is clearing out closets full of junk^H^H^H^H cool old stuff
accumulated by the previous sysadmin. There's a big carton full of PATA hard disks.
Most of them are in the 4.3 GB - 20 GB range, a few larger, a few smaller.
Anyone have any use for these? You can have them for the cost of shipping, or free for
local pickup in Bothell, WA. They're going to be recycled as scrap if I don't
find a home for them.
2303
days inactive
2310
days old
46 comments
21 participants
participants (21)
-
abuse＠cabal.org.uk -
aek＠bitsavers.org -
bill.gunshannon＠hotmail.com -
cclist＠sydex.com -
cctalk＠gtaylor.tnetconsulting.net -
cctalk＠ibm51xx.net -
cisin＠xenosoft.com -
computerdoc＠sc.rr.com -
decguy＠songdog.eskimo.com -
ethan＠757.org -
geneb＠deltasoft.com -
lproven＠gmail.com -
mazzinia＠tin.it -
mechanic_2＠charter.net -
mtapley＠swri.edu -
paulkoning＠comcast.net -
rtomek＠ceti.pl -
spacewar＠gmail.com -
tdk.knight＠gmail.com -
teoz＠neo.rr.com -
turing＠shaw.ca