2 May
2002
2 May
'02
11:10 a.m.
> The emulator community is vigorously using a tape
image container
> format known as TAP for precisely this purpose.
...
Darn...sounds like a subset of what I use. I'd be
interested
in knowing more about TAP (with an eye towards adopting use of it),
and would suggest some possibly missing features might be:
- ability to flag if the current tape record had a recovered read
error (obviously, not all hardware/OS's support getting that information)
No. And I see no useful purpose in including that in the format.
If the read was recovered, you have the data; surely you would
*not* want to recreate a tape and make a previously marginal
block, marginal again on a clone?
- ability to flag if a particular tape record had a
hard error
while reading it
No (I can say this even though I'm about to ask): do you mean an
unrecoverable error, or one that was corrected by the hardware
(as opposed to being corrected by software)? Either way, if the
error was corrected, see my response above.
- ability to flag if an End-of-tape marker/indicator
was seen
while reading the the current record (again, not all
hardware/OS's support getting that information)
No, and again, I can't see how this data would be used to recreate
the tape or would be needed by an emulator. It's kind of like asking
if a particular hardware emulator also emulates memory parity errors
so that you can see memory parity errors in the logs. Why??
- overall header at start of file, recording
information about the tape
No, although I did consider suggesting some extensions like this.
One problem I have encountered with some old CDC tape formats
dealt with how blocking got handled. But my current feeling on
this is that if its blocked on taped, then the records will come
off the taoe with their proper lengths (I encountered this not
trying to read a tape, but trying to create an image of a tape
that no longer exists by a desctiption of its contents).
Anyway, I encode that kins of descriptive information (what kind
of computer, encoding, #tracks, etc) in the filename, or in the
name of the directory containing the file, or in a README.
I have a program called "tapedisk", which
reads an arbitrary tape
and "copies" it to a single disk file (or set of files, if the size of the
output file exceeds the maximum disk file size). The purpose of
tapedisk is to allow a future exact copy of the tape to be recreated
on another tape (of equal or larger size). (That future copy would be
made with the companion program, disktape.) Additionally, most of
my other tape-reading utilities know how to read a tapedisk format
disk file as though it were a tape.
This sounds familiar... is it on the web somewhere?
This is what I store on a per-tape-record basis:
...
So you really do want to recreate bad tapes as bad tapes...
Why?
This is some what I store as a "header" at
the start of the
overall output:
...
Yeah, this info is README fodder... why on earth program that?
As you can see...a lot of info, but all necessary to
faithfully
reproduce the tape later and/or to understand *why* the file might
(or can't) be used to completely reproduce the tape.
Uh, again, can't understand why you'd need to recreate bad tapes.
As to the understanding, README.DOC... etc...
This is part of a product we sell on MPE/iX systems,
but I'd be happy
to document the structures and/or work with other people on standards
in this area.
There's room for potential improvement. For example, is it worthwhile
to have a table at the front of the output file that specifies where
the first N EOFs are?
You just used that nasty, bad word "standard"... TAP is a convention,
not a standard.
;)
However, I'm not King of this Kingdom, we're a collective
(right out of Monty Python's Search for the Holy Grail).
You might try hooking up with the emulator community and
ask your questions there.
Regards,
-doug q
2 May
2 May
1 p.m.
Re:
NOOOO! It's *IMPORTANT* to know if there was a question about the
accuracy of the data! That's *WHY* some tape drives even *tell*
you there was a recovered error ... so the user could use that
information!
You clearly aren't going to try to create a marginal record if/when
you recreate the tape.
You *are* going to be able to truthfully and accurately answer
questions about the validity of your tape copy .. *ONLY* if
you record that information (if it's available).
It's like getting a recovered parity error on memory.
Should you know about it? DAMN STRAIGHT YOU SHOULD!
Without recording that information, you also can't answer
important questions like: is the orginal tape in good shape?
If it was corrected, then there was no error (other than a
recovered read, already correct recorded via the earlier bit :)
I meant a real, live, actual error. (E.g., missing oxide)
When recording the contents of a tape it's **IMPORTANT** to know
that there was an error there. If you simply skip the erorr and
keep reading, you've lost data.
Did you lose original user data? Possibly, and you've definitely
lost data about the tape (i.e., the fact there was an error).
By recording the fact an error occurred, along with any data
your tape drive/OS *may* have sent you, you preserve the ability
to act wisely in the future. For example, by recording the
fact there was an error at megabyte 100 in a 102 MB file,
the user would have confidence that when that file is eventually
extracted, the first 99 MB of it is in fine shape.
With 9-track tapes, software writing to the tapes generally watches
for an indication that the end-of-tape marker (a short silver
reflective strip) has been seen. At that time, it can prepare
to do a reel-switch. Although some people feel they can backspace
and write reel-switch info prior to the EOT marker on a 9-track
tape, that's technically not safe (and opens up the possibility
of getting what's called (in 1979) "crap in the gap"). Anyway,
the ANSI (?) standard for 9 track tapes specifies a minimum length
of tape be available for writing after the EOT marker ... a length
that should be sufficient (even with modest multiple outstanding
write-ahead requests) to write the reel switch information.
On at least one OS, Burroughs MCP, one could detect that EOT at
read time. Then, if one is doing multiple read-ahead requests,
one can moderate them appropriately.
Ok...the short answer: it's information. You don't throw away
information without a darn good reason. :)
Sorry...if you want a professional program, not something like
that toy cptape, you *WILL* have this information.
Been there, worked on that, for over 30 years. Trust me...you
*want* all the information you can get.
The MPE V version was/is (can't recall which), and was free.
The MPE/iX version is a product from our company.
You might be thinking of CodeBlue's TapeDisk, a PC disk
to tape backup mechanism. (I bought a copy about 5 years
ago, but had some problems with it: it didn't work on parallel port tape
drives as implied, IIRC. Once I got a different kind of tape drive,
it seemed to work, although we never had to recover with it.)
Since my first version tapedisk preceded theirs, I don't mind calling
mine that ... at least, not until the lawyers threaten me :)
Again...you're *NOT RECREATING BAD TAPES*.
- ability to
flag if the current tape record had a recovered read
error (obviously, not all hardware/OS's support getting that information)
No. And I see no useful purpose in including that in the format.
If the read was recovered, you have the data; surely you would
*not* want to recreate a tape and make a previously marginal
block, marginal again on a clone? - ability to
flag if a particular tape record had a hard error
while reading it
No (I can say this even though I'm about to ask): do you mean an
unrecoverable error, or one that was corrected by the hardware
(as opposed to being corrected by software)? Either way, if the
error was corrected, see my response above. - ability to
flag if an End-of-tape marker/indicator was seen
while reading the the current record (again, not all
hardware/OS's support getting that information)
No, and again, I can't see how this data would be used to recreate
the tape or would be needed by an emulator. It's kind of like asking
if a particular hardware emulator also emulates memory parity errors
so that you can see memory parity errors in the logs. Why?? - overall
header at start of file, recording information about the tape
No, although I did consider suggesting some extensions like this.
One problem I have encountered with some old CDC tape formats Anyway, I encode that kins of descriptive information
(what kind
of computer, encoding, #tracks, etc) in the filename, or in the
name of the directory containing the file, or in a README.
If two items of data are separated, what happens to them?
Yep. Should be 'nuff said :)
I have a
program called "tapedisk", which reads an arbitrary tape
This sounds familiar... is it on the web somewhere?
So you really do want to recreate bad tapes as bad tapes...
Why?
No...I want to know that a problem existed, and where that problem was.
Again...this is basic, vital, and indisputable. You *NEED* to
know if there was a problem with the tape, and if you're trying to
extract a particular file you need to know where any problems
are in that file. Otherwise, you'll be lying to the user.
If you know where within a file an error was found, then you've
got a chance at recovering most of the rest of the data successfully.
If you know an error occurred *somewhere* in a file, but not
where, then you have a large pile of bits which *might* be trash,
and *might* be useful ... but neither you or the user knows.
And, if you don't know where the errors were (and how many there
were), you can't reliably restore the data. That data might
be banking information that your company is being audited on;
it might be data that could clear someone of a crime...you simply
don't know.
Data is important...that's why we call it data and not junk :)
This is some what I store as a "header" at
the start of the
Uh, 'cuz someone had to do it and build a reliable, thorough
program?
overall output:
...
Yeah, this info is README fodder... why on earth program that? As you can
see...a lot of info, but all necessary to faithfully
reproduce the tape later and/or to understand *why* the file might
(or can't) be used to completely reproduce the tape.
Uh, again, can't understand why you'd need to recreate bad tapes. You might try hooking up with the emulator community
and
ask your questions there.
This being ClassicCmp, I thought they might be here, too :)
Besides, this isn't an emulator question so much as a data
preservation question.
Stan Sieler sieler(a)allegro.com
www.allegro.com/sieler/wanted/index.html www.allegro.com/sieler
2:54 p.m.
At 04:10 PM 5/2/2002 -0400, Douglas H. Quebbeman wrote:
No. And I see no useful purpose in including that in
the format.
If the read was recovered, you have the data; surely you would
*not* want to recreate a tape and make a previously marginal
block, marginal again on a clone?
Choosing between the loss of all the data, and a partial
copy, I'll take the partial copy.
I have many 8-inch disks with bad blocks. I'd like to preserve
the data if at all possible. A simplistic 'dd'-style read
won't preserve everything; it'll bail out when it has a problem.
It has nothing to do with any hope of recreating the original tape.
If all the data is recovered, sure you could make a copy from it.
If it isn't, then you have some of it, as opposed to none of it.
- John
5:24 p.m.
New subject: Tape dumping programs for Unix/Linux... 8-inch disks with bad blocks.
On Thursday 02 May 2002 18:54, you wrote:
I have many 8-inch disks with bad blocks. I'd
like to preserve
the data if at all possible. A simplistic 'dd'-style read
won't preserve everything; it'll bail out when it has a problem.
In dos you could do that with a batfile script of looping debug
commands
ie read sector > mem write sector mem > another disk
when you are done with a floppy you would have
all the readable sectors on a new floppy or whatever
with the bad ones left containing whatever bits
it could get if any along with the prev sectors data
You could at least spool the new disk to a file and
worry over the missing bits..
Ive done this before with bad PC floppies that
could not be mounted in any way, ie trashed
unreadble block 0, alien file system etc.
I suppose you could hack fdformat ( the verify code etc )
into a very respectable floppy recovery tool that would give
you all the good sectors to a file with the nonreadble sectors
filled with zeros..
one scheme perhaps .. have your set of "tools"
write a empty recovery file preinit with zeroed blocks
(easy to do with dd)
and then write only good reads into their respective
positions, that way in the attempt to get a good read
of bad sectors .. bad reads of once good sectors
wont trash good data you managed to get already
Hey .. its an idea. there is lots of "getsector" code out
there burried in a lot of things that you could wrap
your own code around .. you might not even wrap
it with much ...
The following is only a simulation
#getsector --help
Usage: getsector -d device -s relative_sector
fill in the rest with shell scripting ... wouldnt be fast
but simple brute force is often better
Isnt there a readtrack command on some controllers ?
You could come at this from a few angles i suppose.
Raymond
8:08 p.m.
New subject: Tape dumping programs for Unix/Linux... 8-inch disks with bad blocks.
It was thus said that the Great Raymond Moyers once stated:
Ive done this before with bad PC floppies that
could not be mounted in any way, ie trashed
unreadble block 0, alien file system etc.
I suppose you could hack fdformat ( the verify code etc )
into a very respectable floppy recovery tool that would give
you all the good sectors to a file with the nonreadble sectors
filled with zeros..
I remember using a disk utility for the Color Computer (forgot the name,
it's been nearly 10 years since I probably last used it) that would load the
sector into video memory (sectors on the Coco were 256 bytes in length, the
video screen was 512 bytes in length, so it would load the sector into the
lower half of video memory, leaving the top half for the menu/command
section).
It was interesting loading up a bad sector---often times only a few bytes
would change between reads, but rarely did the drive fail so bad that
nothing at all could be read (I suppose the disks weren't as bad as they
could get).
Based upon that, I might instead have the program detect a read that
failed, and instead of rereading the sector into the memory just used, grab
another sector's worth of memory, read into that; if there's an error, grab
yet another sectors worth of memory, read into that. Do that a preset
number of times (say, four times). Process it like:
for (i = 0 , bad_spot = 0; i < SECTOR_SIZE ; i++)
{
if
(
(sector[0][i] == sector[1][i])
&& (sector[1][i] == sector[2][i])
&& (sector[2][i] == sector[3][i])
)
{
good_sector[i] = sector[0][i];
}
else
{
record_bad_spot(head,track,sector,i);
bad_spot++;
good_sector[i] = 0;
}
}
if (((double)bad_spot / (double)SECTOR_SIZE) > THRESHHOLD)
{
record_bad_sector(head,track,sector);
}
You might be able to even isolate the bad bits in the bad byte, or take a
vote and use the most reported value to store into the good sector or some
other analysis. At the very least, I would record as much of the bad sector
as I could and write out (in a separate file) the known bad spots from the
disk being read.
-spc (Or am I way off base here?)
9:01 p.m.
New subject: Tape dumping, bad blocks.
On Friday 03 May 2002 00:08, you wrote:
You might be able to even isolate the bad bits in
the bad byte, or take a
vote and use the most reported value to store into the good sector or some
other analysis. At the very least, I would record as much of the bad
sector as I could and write out (in a separate file) the known bad spots
from the disk being read.
You might be able to do some of that with a readtrack ,, hard to say
depends on the controller i guess
The start of a sector is all FF's or rather, stream of 1 bits then FE
(or some other address mark) the controller then knows where it
is in the bytestream.
The problem with lost bits in a sector, is that all after that the controller
no longer knows what position its in .. is the next bit the high bit of the
next byte or the last bit of a current byte ? .. theres no way to tell
is there ? same with a tape, well modern scsi tapes anyway, they
are written in 512 byte blocks and here too, same thing.
So in the floppy case or the hard disk case or the tape case,
the next stream of 1's comes along, then the address mark
and the controler knows where it is again and can read more
stuff, but all after the error in the last block, floppy/disk/tape
ya get nothing but a short read.
Funny, on the 9 track tape you dont confront this issue.
9 track tape is 8 bits + parity right ? so here a bad bit dont
cause you to lose the ability to get the next byte and know
that its a valid byte, a physical superiority it has over the
serial streamers ?.
Or am i assuming too much again.
Raymond
8120
days inactive
8121
days old
5 comments
5 participants
participants (5)
-
dquebbeman＠acm.org -
jfoust＠threedee.com -
rmoyers＠nop.org -
sieler＠allegro.com -
spc＠conman.org