RAID? Was: PATA hard disks, anyone?

[Fred Cisin]

>> How many drives would you need, to be able to set up a RAID, or hot swappable RAUD (Redundant Array of Unreliable Drives), that could give decent reliability with such drives?
>> How many to be able to not have data loss if a second one dies before the first casualty is replaced?
>> How many to be able to avoid data loss if a third one dies before the first two are replaced?

On Wed, 28 Mar 2018, Paul Koning wrote:
> These are straightforward questions of probability math, but it takes 
> some time to get the details right.  For one thing, you need believable 
> numbers for the underlying error probabilities.  And you have to analyze 
> the cases carefully.

THANK YOU for the detailed explanation!

> The basic assumption is that failures are "fail stop", i.e., a drive 
> refuses to deliver data.  (In particular, it doesn't lie -- deliver 
> wrong data.  You can build systems that deal with lying drives but RAID 
> is not such a system.)  The failure may be the whole drive ("it's a 
> door-stop") or individual blocks (hard read errors).

So, in addition to the "RAID" configuration, you would also need 
additional redundancy to compare multiple reads for error detection.
At the simplest level, if the reads don't match, then there is an error.
If a retry produces different dataa, then that drive has an error.
If two drives agree against a third, then there is a high probability that 
the variant drive is in error.

> In either case, RAID-1 and RAID-5 handle single faults.  RAID-6 isn't a 
> single well-defined thing but as normally defined it is a system that 
> handles double faults.  So a RAID-1 system with a double fault may fail 
> to give you your data.  (It may also be ok -- it depends on where the 
> faults are.)  RAID-5 ditto.
>
> The tricky part is what happens when a drive breaks.  Consider RAID-5 
> with a single dead drive, and the others are 100% ok.  Your data is 
> still good.  When the broken drive is replaced, RAID rebuilds the bits 
> that belong on that drive.  Once that rebuild finishes, you're once 
> again fault tolerant.  But a second failure prior to rebuild completion 
> means loss of data.

With very unreliable drives, that isn't acceptable.
If each "drive" within the RAID were itself a RAID, . . .
Getting to be a complicated controller, or cascading controllers, . . .

> So one way to look at it: given the MTBF, calculate the probability of 
> two drives failing within N hours (where N is the time required to 
> replace the failed drive and then rebuild the data onto the new drive). 
> But that is not the whole story.

'course not.  Besides MTBF for calculating the probability of a second 
drive failing within N hours, must also consider other factors, such as 
external influences causing more than one drive to go, and the essentially 
non-linear aspect of a failure rate curve.

> The other part of the story is that drives have a non-zero probability 
> of a hard read error.  So during rebuild, you may encounter a sector on 
> one of the remaining drives that can't be read.  If so, that sector is 
> lost.

If we consider that to be a "drive failure", then we are back to designing 
around multiple failures.

> The probability of hard read error varies with drive technology.  And of 
> course, the larger the drive, the greater the probability (all else 
> being equal) of having SOME sector be unreadable.  For drives small 
> enough to have PATA interfaces, the probability of hard read error is 
> probably low enough that you can *usually* read the whole drive without 
> error.  That translates to: RAID-1 and RAID-5 are generally adequate for 
> PATA disks.

"generally".
The original thought behind this silly suggestion was whether it would be 
possible to make use of MANY very unreliable drives.

> On the very large drives currently available, it's a different story, 
> and the published drive specs make this quite clear.  This is why RAID-6 
> is much more popular now than it was earlier.  It isn't the probability 
> of two nearly simultaneous drive failures, but rather the probability of 
> a hard sector read error while a drive has failed, that argues for the 
> use of RAID-6 in modern storage systems.