11 Aug
2004
11 Aug
'04
7:10 a.m.
Hi
Nether of these formats is totally a match for what we need.
These are all just data image formats for bits. None of these
describe the format of the bits. We can use the parts of these
specifications we like but what ever we use will still be
embedded as part of a larger file format. As Sellam says, that
format needs to be extendable to handle anything that comes
along. Both Intel HEX and Motorola S formats are such a small
part of the picture.
One thing that could be done but I'm sure many will think
I'm crazy. If the files contained OpenBoot source code to describe
the data, one can then create almost anything one wants.
This would be something like how postscript files are done now.
These file would have code that would transform the data into
whatever format the user needed. One could extract the bit
stream to feed as raw information to a drive or simply extract
the sector data. Using something like OpenBoot code has the
advantage that the source is human readable, just as postscript
is human readable ( although tedious ).
There could be some guide lines on standard ways to describe
common formats. These would also help other describe more
obscure formats.
We need to remember that what normally comes out of a floppy
controller chip is not the actual data on the disk. The information
on the disk is a more complex. It contains things like special
marks for indexing, headers and even errors. These need to
be represented as well. We need to be looking at capturing the
raw data from the disk. And at methods of post processing this
to a form similar to what comes from the floppy controller.
Dwight
From: "David V. Corbin"
<dvcorbin(a)optonline.net>
You might want to consider Motorola rather than Intel.
The only reason I suggest this is the motorola format alread has a (semi)
extensible record type description [the second char of each line].
S2 and S3 are already defined for 16MB and 4GB memory spaces...
A (non-standard) record type "could" be used for media format, etc...
Just my 16.3875 milli-EUR comment...
>> -----Original Message-----
>> From: cctalk-bounces(a)classiccmp.org
>> [mailto:cctalk-bounces@classiccmp.org] On Behalf Of Cini, Richard
>> Sent: Wednesday, August 11, 2004 10:02 AM
>> To: 'General Discussion: On-Topic and Off-Topic Posts'
>> Subject: RE: Let's develop an open-source media archive standard
>>
>> I haven't read the entire thread on this but I did read
>> Steve Thatcher's idea and it describes about where I was
>> coming out on this myself.
>>
>> I might have missed what the ultimate use of this archive
>> would be. Will the archive be used to (1) re-generate
>> original media; (2) operate with emualtors; (3) both?
>>
>> To ensure integrity of the data I would propose recording
>> the data in the Intel Hex format -- it's text-based and has
>> built-in CRC. Now, we'd have to modify the standard format
>> a bit to accommodate a larger address space and to add some
>> sort of standardized header (a "Hardware Descriptor"). This
>> data would be used by the de-archiver to interpret the
>> stream of data read from the data area (the "Hex Block").
>>
>> I agree that a multi-layer approach offers the best
>> combination of platform neutrality and portability. I don't
>> really know if we need two or three layers as Steve
>> described to describe the file in a standard fashion. Using
>> an Intel Hex-like format would increase the "de-archiving"
>> time, but in my view it's a fair trade-off. De-archiving
>> software could translate the platform-neutral file into
>> another format better suited for use in emulators.
>>
>> I think that we should start compiling a list of the
>> various media we want represented and how that media is
>> organized natively. I don't mean "well, it has blocks and
>> sectors" either. We should examine the exact format down to
>> the actual numbers (i.e., "2048 blocks of 256-bytes
>> recorded twice"). Seeing how the various data stores are
>> organized should bring some clarity to how we should represent it.
>>
>> Just my $0.02.
>>
>> Rich
>>
11 Aug
11 Aug
7:32 a.m.
On Wed, 11 Aug 2004, Dwight K. Elvey wrote:
One thing that could be done but I'm sure many
will think
I'm crazy. If the files contained OpenBoot source code to describe
the data, one can then create almost anything one wants.
I'm not familiar with OpenBoot. Can you explain this a bit more?
We need to remember that what normally comes out of a
floppy
controller chip is not the actual data on the disk. The information
on the disk is a more complex. It contains things like special
marks for indexing, headers and even errors. These need to
be represented as well. We need to be looking at capturing the
raw data from the disk. And at methods of post processing this
to a form similar to what comes from the floppy controller.
There are multiple levels at which someone may want to archive a medium.
There's the physical level, as you describe above. Then there's the level
in between physical and logical, which is, to use the example of floppy
disks again, tracks and sectors. And then there's the logical level which
is an actual filesystem with a directory and filenames, etc.
This spec should be designed to be able to handle all three types
simultaneously within the same image file.
What I mean is that it might be useful to have (to use a floppy disk as an
example again) most of the disk encoded in a track/sector representation,
but then have one particular track encoded at a bit level, because perhaps
it contains special signatures that are part of a copy protection scheme.
If the image is ever used to re-create the original physical disk, the
binary data will be essential if the disk is going to actually be expected
to work (unless the copy protection scheme is removed at that point).
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
9:19 a.m.
At 02:10 PM 8/11/2004, Dwight K. Elvey wrote:
We need to be looking at capturing the
raw data from the disk. And at methods of post processing this
to a form similar to what comes from the floppy controller.
And if you want to get really crazy, there's Jerome Fine's
methods of reconstructing faded magtapes by digitizing and
DSP-ing the raw data, or the obscure emulator tools that
can read the data from digitized cassette images of programs
and data.
I wish there was a mechanism like this for recovering
video from 8mm video tape; I have some munged tapes I'd like to
recover someday. Certainly others in the future and present world
could benefit from videotape recovery schemes. And then there's
the NSA's methods of reading erased data from hard disks.
I vote for ASCII and not binary or compressed binary in
the files. Compression is for afterwards and outside.
With the right code, an emulator could keep the
offsets to the "blocks" and rewrite the hex as needed
in-place. :-)
- John
11:53 a.m.
And if you want to get really crazy, there's
Jerome Fine's
methods of reconstructing faded magtapes by digitizing and
DSP-ing the raw data, or the obscure emulator tools that
can read the data from digitized cassette images of programs
and data.
Similar to the idea of recording C64 cassettes via a
soudcard and storing as WAV (or whatever) for later
playback to a C64 (which thinks the noise if coming
in from a cassette player). Sounds perfectly reasonable
to me :-)
I wish there was a mechanism like this for recovering
video from 8mm video tape; I have some munged tapes I'd like to
recover someday. Certainly others in the future and present world
could benefit from videotape recovery schemes. And then there's
the NSA's methods of reading erased data from hard disks.
Lots of places can do data recovery from dead and erased
disks. But doing it for a disk that has had a proper
security erase done (i.e. every data-storing physical
overwritten with the appropriate multiple patterns) is
very hard. I know that it was possible a decade or two ago
to go over the platter with an STM and effectively image
the off-track remnants of the data. But even then it was
time-consuming and not terribly reliable. These days
the disks have a hard enough time guessing at what is
on the platter :-)
I vote for ASCII and not binary or compressed binary
in
the files. Compression is for afterwards and outside.
Hear, hear.
With the right code, an emulator could keep the
offsets to the "blocks" and rewrite the hex as needed
in-place. :-)
This format should just be a way to preserve data. There
will be, of necessity, much more flexibility than any
emulator would ever need. The way to avoid complexity
for an emulator would be to produce a utility that
converts from the archive format (which may contain
a bistream or whatever) to an image format which the
emulator is happy with. Information is lost in the process,
but it's not information the emulator wants or needs.
(But don't throw the archive away afterwards :-)).
Antonio
--
---------------
Antonio Carlini arcarlini(a)iee.org
7442
days inactive
7442
days old
3 comments
4 participants
participants (4)
-
a.carlini＠ntlworld.com -
dwight.elvey＠amd.com -
jfoust＠threedee.com -
vcf＠siconic.com