17 May
2005
17 May
'05
3:06 p.m.
As coincidence would have it, I work at Indiana University's Digital
Library Program and there was a lecture on archiving audio which hits
many of the same issues that have come up here. The conclusions that
they came up with for the project included:
* There's no such thing as an eternal media: the data must be
transportable to the latest generation of storage
* Metadata should be bundled with the content
* Act like you get one chance to read the media :(
While this is a different context, the principle is basically the same.
I've got a pile of TK50 tapes I'm backing up using the SIMH tape format,
so this is relevant to that process as well.
I think the optimum format for doing this isn't a single file, but a
collection of files bundled into a single package. Someone mentioned
tar, I think, and zip would work just as well. The container could
contain these components:
* content metadata - info from the disk's label/sleeve, etc
* media metadata - the type of media this came from
* archivist metadata - who did it, methods used, notes, etc
* badblock information - 0 blocks which are actually bad.
* content - a bytestream of the data
I don't think there's any real need to document the physical properties
of the media for EVERY disk archived -- there should probably be a
repository of 'standard' media types (1541's different-sectors-per-track
info, FM vs MFM per track information, etc) plus overrides in the media
metadata (uses fat-tracks, is 40 track vs 35, etc).
Emulators could use the content part of the file as-is and collectors
would have enough information to recreate the original media. It would
also allow for cataloging fairly easily.
Brian
On Tue, 2005-05-17 at 21:04 +0000, Jules Richardson wrote:
On Tue, 2005-05-17 at 15:07 -0500, Randy McLaughlin
wrote:
I like and prefer media images as straight data
dumps but I want the
formatting information of the original media somewhere. I even want data
from media that is incomplete or has errors, also documented.
Yep, me too. From when we were bashing around ideas about this though a
few months back it seems that's a minority viewpoint; most people want
data embedded in the metadata.
For hard drive images I zero-pad any bad data but also include metadata
in a seperate file - including disk geometry, which blocks are bad,
resulting dump checksum, timestamp etc. along with anything else that
might be particularly useful. For floppy images things would be
significantly more complex though (due to factors as mentioned - variant
sectors/track, different encoding for different tracks etc.)
The idea behind futurekeep though was to make the metadata highly
structured and in a similar vein to HTML in that clients could handle as
much of the data as needed (eg. someone not dealing with variable bit
rate images wouldn't need a decoder that could handle them). Ideally
it'd be human-readable too (after a fashion) - e.g. XML - so that the
data could be reconstructed into a disk image "by hand" even if some
whizzy util to do it wasn't present. (understanding it at a file level
is obviously outside the scope)
That doesn't seem *too* much to ask; basic metadata can be created for
existing images without a lot of hassle *if desired*. To me such a
format's more useful for future image creation though, particularly in
the case of less-common systems; the popular machines are likely to be
covered by their own archive formats already and the following large
enough that lack of data is not (yet) a problem. Rather than messing
around with proprietary image formats for those, or formats that aren't
particularly descriptive, it'd be nice to start from day 1 using
something that allows us to capture all the useful stuff that goes along
with the raw data.
cheers
Jules