11 Aug
2004
11 Aug
'04
1:50 a.m.
Hi all, after reading all this morning's posts, I thought I would throw out some
thoughts.
XML as a readable format is a great idea. There is plenty of software available on most
platforms to deal with this format.
I looked at the CAPS format and in part that would be okay. I would like to throw in an
idea of whatever we create as a standard actually have three sections to it.
The first would be a data description and subsequent data (i.e. Data Container contains
Data Items). Block size, number of blocks, and block data...
The second would be a physical format description that would tell how the data description
data is actually stored on whatever media is being targeted (Format Container contains
Format Items). Device type (sequential, random), track, sector, virtual sector, head, bad
sectors, etc would be needed and then a mapping from each Data Item to each Format Item.
The third would simply be a "image file" that would simply be a sequential block
of data with length and CRC that contained the Data Container and Format Container.
This type of arrangement would give us a single file to send around that not only allowed
for the recreation of the physical format, but also allowed for direct file access (read
only of course). I suppose one could think of it as a smart library file that not only
contained one or more files, but also gave physical re-creation information.
best regards, Steve Thatcher
-----Original Message-----
From: Vintage Computer Festival <vcf(a)siconic.com>
Sent: Aug 10, 2004 7:04 PM
To: Classic Computers Mailing List <cctalk(a)classiccmp.org>
Subject: Let's develop an open-source media archive standard
Ok, instead of all the unproductive bickering over semantics (most of
which is illogical...come on, admit it) I'd rather we actually discuss
developing further an open-source, public domain, free (in every sense of
the word) standard for archiving data media of all types (magentic, paper,
or otherwise).
This is basically going to be a continuation of what we've already
discussed on the list, and what Hans and I have discussed in private.
First, let's start with the goals.
The format should be:
1) Well Documented (with such documentation actively preserved in all four
corners of the globe and beyond)
2) Not constrained to any particular hardware
3) Be inclusive of all physical (and logical?) manner of recording media
4) Be implementable on even the simplest architectures (because the
original media source will in many cases have to be read on the hardware
it is connected to)
5) Open source, public domain, etc. (although a copyright may be held if
it makes sense to do so)
6) Adaptable, expandable, revisable (for future extensions)
7) Text-based and storable in commonly accessible character formats (i.e.
a suitable subset of Unicode, i.e. ASCII)
8) Allow for the representation of media in either logical or physical
(raw bit stream) formats
This is a good start. Someone please continue adding to the definition.
I will establish a special session at the next VCF (November 6-7) where
we can commence a committee for formalizing this standard and getting it
recognized internationally in all the various relevant groups (i.e. ANSI
or ISO/IEC).
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
11 Aug
11 Aug
3:08 a.m.
On Wed, 2004-08-11 at 10:50, Steve Thatcher wrote:
Hi all, after reading all this morning's posts, I
thought I would throw out some thoughts.
XML as a readable format is a great idea.
I haven't done any serious playing with XML in the last couple of years,
but back when I did, my experience was that XML is not a good format for
mixing human-readable and binary data within the XML structure itself.
To make matters worse, the XML spec (at least at the time) did not
define whether it was possible to pass several XML documents down the
same data stream (or, as we'd likely need for this, XML documents mixed
with raw binary). Typically, parsers of the day expected to take control
of the data stream and expected it to contain one XML document only -
often closing the stream themselves afterwards.
I did end up writing my own parser in a couple of KB of code which was a
little more flexible in data stream handling (so XML's certainly not a
heavyweight format, and could likely be handled on pretty much any
machine), but it would be nice to make use of off-the-shelf parsers for
platforms that have them where possible.
As you've also said, my initial thought for a data format was to keep
human-readable config seperate from binary data. The human-readable
config would contain a table of lengths/offsets for the binary data
giving the actual definition. This does have the advantage that if the
binary data happens to be a linear sequence of blocks (sectors in the
case of a disk image) then the raw image can easily be extracted if
needs be (say, to allow conversion to a different format)
Personally, I'm not a fan of mixing binary data in with the
human-readable parts because then there are issues of character escaping
as well as the structure detracting from the readability. And if encoded
binary data is used instead (say, hexadecimal representation) then
there's still an issue of readability, plus the archive ends up bloated
and extra CPU cycles are needed to decode data. Neither of those two
approaches lend themselves to simply being able to use common
command-line utilities to extract the data, either. I'm prefectly
willing to be convinced, though :)
I looked at the CAPS format and in part that would be
okay. I would like
to throw in an idea of whatever we create as a standard actually have
three sections to it.
So, first section is all the 'fuzzy' data (author, date, version info,
description etc.), second section describes the layout of the binary
data (offsets, surfaces, etc.), and the third section is the raw binary
data itself? If so, I'm certainly happy with that :-)
One aside - what's the natural way of defining data on a GCR floppy? Do
heads/sectors/tracks still make sense as an addressing mode, but it's
just that the number of sectors per track varies according to the track
number? Or isn't it that simple?
cheers
Jules
3:40 a.m.
New subject: OT: Technical Explaination...
Hope the group will get a kick....
Why Computers Sometimes Crash! by Dr. Seuss. (Read this to yourself aloud -
it's great!)
If a packet hits a pocket on a socket on a port, and the bus is interrupted
at a very last resort, and the access of the memory makes your floppy disk
abort, then the socket packet pocket has an error to report.
If your cursor finds a menu item followed by a dash, and the double-clicking
icon puts your window in the trash, and your data is corrupted cause the
index doesn't hash,
then your situation's hopeless and your system's gonna crash!
If the label on the cable on the table at your house, says the network is
connected to the button on your mouse, but your packets want to tunnel to
another protocol, that's repeatedly rejected by the printer down the hall.
And your screen is all distorted by the side effects of gauss, so your icons
in the window are as wavy as a souse; then you may as well reboot and go out
with a bang, 'cuz sure as I'm a poet, the sucker's gonna hang.
When the copy on your floppy's getting sloppy in the disk, and the macro
code instructions is causing unnecessary risk, then you'll have to flash the
memory and you'll want to RAM your ROM, and then quickly turn off the
computer and be sure to tell your Mom!
Well, that certainly clears things up for me. How about you?
5:23 a.m.
New subject: OT: Technical Explaination...
In message
<!~!UENERkVCMDkAAQACAAAAAAAAAAAAAAAAABgAAAAAAAAAC6RJHIx+0kSwesJ7EUMCZcKAAAAQAAAA13B5rxvTF0GAmKUt9EKj7gEAAAAA(a)optonline.net>
"David V. Corbin" <dvcorbin(a)optonline.net> wrote:
Hope the group will get a kick....
Why Computers Sometimes Crash! by Dr. Seuss. (Read this to yourself aloud -
it's great!)
[snip]
It's even better when you post the whole thing...
source: <http://www.seuss.org/seuss/seuss.byte.html>
===> What If Dr. Seuss Wrote Technical Manuals? <===
===> Author: Gene Ziegler <===
=> A Grandchild's Guide to Using Grandpa's Computer <=
Bits Bytes Chips Clocks
Bits in bytes on chips in box.
Bytes with bits and chips with clocks.
Chips in box on ether-docks.
Chips with bits come. Chips with bytes come.
Chips with bits and bytes and clocks come.
Look, sir. Look, sir. read the book, sir.
Let's do tricks with bits and bytes, sir.
Let's do tricks with chips and clocks, sir.
First, I'll make a quick trick bit stack.
Then I'll make a quick trick byte stack.
You can make a quick trick chip stack.
You can make a quick trick clock stack.
And here's a new trick on the scene.
Bits in bytes for your machine.
Bytes in words to fill your screen.
Now we come to ticks and tocks, sir.
Try to say this by the clock, sir.
Clocks on chips tick.
Clocks on chips tock.
Eight byte bits tick.
Eight bit bytes tock.
Clocks on chips with eight bit bytes tick.
Chips with clocks and eight byte bits tock.
[ now for the good bit -Philpem ]
Here's an easy game to play.
Here's an easy thing to say....
If a packet hits a pocket on a socket on a port,
And the bus is interrupted as a very last resort,
And the address of the memory makes your floppy disk abort,
Then the socket packet pocket has an error to report!
If your cursor finds a menu item followed by a dash,
And the double-clicking icon puts your window in the trash,
And your data is corrupted 'cause the index doesn't hash,
Then your situation's hopeless, and your system's gonna crash!
You can't say this? What a shame sir!
We'll find you another game sir.
If the label on the cable on the table at your house,
Says the network is connected to the button on your mouse,
But your packets want to tunnel on another protocol,
That's repeatedly rejected by the printer down the hall,
And your screen is all distorted by the side effects of gauss
So your icons in the window are as wavy as a souse,
Then you may as well reboot and go out with a bang,
'Cause as sure as I'm a poet, the sucker's gonna hang!
When the copy of your floppy's getting sloppy on the disk,
And the microcode instructions cause unnecessary RISC,
Then you have to flash your memory and you'll want to RAM your ROM.
Quickly turn off the computer and be sure to tell your mom!
Later.
--
Phil. | Acorn Risc PC600 Mk3, SA202, 64MB, 6GB,
philpem(a)dsl.pipex.com | ViewFinder, 10BaseT Ethernet, 2-slice,
http://www.philpem.dsl.pipex.com/ | 48xCD, ARCINv6c IDE, SCSI
... I like Boolean logic. NOT!
4:17 a.m.
Am 11 Aug 2004 12:08 meinte Jules Richardson:
On Wed, 2004-08-11 at 10:50, Steve Thatcher wrote:
> Hi all, after reading all this morning's posts, I thought I would throw out some
thoughts.
> XML as a readable format is a great idea.
I haven't done any serious playing with XML in the
last couple of years,
but back when I did, my experience was that XML is not a good format for
mixing human-readable and binary data within the XML structure itself.
Only if you intend to keep it 100% human readable.
To make matters worse, the XML spec (at least at the
time) did not
define whether it was possible to pass several XML documents down the
same data stream (or, as we'd likely need for this, XML documents mixed
with raw binary). Typically, parsers of the day expected to take control
of the data stream and expected it to contain one XML document only -
often closing the stream themselves afterwards.
Now, that's a feature of the reading application. XML does not
stat what happens next since this is outside the scope. It is
perfectly op to look for the next Document, or the next start
tag of the same document type, or for whatever.
I did end up writing my own parser in a couple of KB
of code which was a
little more flexible in data stream handling (so XML's certainly not a
heavyweight format, and could likely be handled on pretty much any
machine), but it would be nice to make use of off-the-shelf parsers for
platforms that have them where possible.
Right, but especialy when we're coming down to classic platforms,
such building blocks are not always usable, and in general way
oversized. On a 48k Apple (or a 64 K 4 MHz CP/M machine) we don't
have the space to just port a C-app that has 'only' 100k of code
size. So reader/writer applications for the original environment
have to be small and special to type.
As you've also said, my initial thought for a data
format was to keep
human-readable config seperate from binary data. The human-readable
config would contain a table of lengths/offsets for the binary data
giving the actual definition. This does have the advantage that if the
binary data happens to be a linear sequence of blocks (sectors in the
case of a disk image) then the raw image can easily be extracted if
needs be (say, to allow conversion to a different format)
Well, that is only true if you define binary data as 8 Bit and
all means of transport as 100% transparent. Just, this hasn't
worked that way in the past, and I doubt that we will be safe
from changes in the future.
As for the character size: we had in the past everything from
6 to 12 Bit (ok, I can't remember 11 Bit characters/words) as
'binary' characters. Of course 6,7 and 8 Bit Bytes can be easy
stored in a 8 Bit Byte, but what about 9 Bit (Bull) or 12 (DEC)?
At that point you already have to incooperate speciual trans-
formation rules which are not necersary transparent.
Also for the requirement of a transparent transport: When
transfering files between different architectures we usualy
have code or even format conversions. Most notable code
conversion would be, for example, ISO 8859-1 <-> EBCDIC which
totally destroys the 'binary' part. Or take format conversions
as done on the way between Unix style files and (Win-)DOS, LF
vs CR/LF. Whenever you leave the A-Z and 0-9 range we are
likely to encounter such problems.
Shure, one could code an app capable to read ASCII/Binary on
a EBCDIC Machine and vice versa, but to my experience (doing
programming since 25 years in mixed environments) it's not
only a boring job, but also one of the most sensitive to
errors.
Any kind of standard format must be true machine independent.
Thus (at least when using the recommended representation) be
able to be transferred across all platforms thinkable of.
> I looked at the CAPS format and in part that
would be okay. I would like
> to throw in an idea of whatever we create as a standard actually have
> three sections to it.
So, first section is all the 'fuzzy' data
(author, date, version info,
description etc.), second section describes the layout of the binary
data (offsets, surfaces, etc.), and the third section is the raw binary
data itself? If so, I'm certainly happy with that :-)
I would rather go for an anoted format, where more detailed
information can be added at any point, and not necersary
in certain sections. Especialy since the 'fuzzy' data is
usualy not needed for the job itself.
One aside - what's the natural way of defining
data on a GCR floppy? Do
heads/sectors/tracks still make sense as an addressing mode, but it's
just that the number of sectors per track varies according to the track
number? Or isn't it that simple?
Well, that's already outside of what a standard definition
can define without doubt.
To my understanding interpretation of Data is always part
of a real application. As soon as it touches machine or
format specific implementation details a standard may only
give guidelines how to store them properly, but not how to
interprete. That's part of an actual reader implementation.
And each rader will of course only understand parts he's
made for - e.g. a Apple DOS 3.3 reader will have no idea
what a tape label for a IBM tape is not to mention be able
to differentiate between the various header types.
Reader/Writer apps will always be as specific as they are
right now, when handling a proprietary format. The big
advantage is that intermediate tools, like archiving,
indexing, etc.pp can be shared. Well, in fact it's the
only advantage, except the fact that one doesn't have to
figure out a new format each time, and the simple format
does allow the ad hoc inclusion of new machines/systems.
Gruss
H.
--
VCF Europa 6.0 am 30.April und 01.Mai 2005 in Muenchen
http://www.vcfe.org/
11:57 a.m.
On Wed, 11 Aug 2004, Jules Richardson wrote:
One aside - what's the natural way of defining
data on a GCR floppy? Do
heads/sectors/tracks still make sense as an addressing mode, but it's
just that the number of sectors per track varies according to the track
number? Or isn't it that simple?
GCR is also usually done with tracks and sectors.
Having a different number of sectors on different tracks is
irrelevant to whether it is GCR or MFM or ?
Instead, that often becomes a matter of
CAV (constant angular velocity) v CLV (constant linear velocity)
It can be implemented by a variable motor speed, OR
by a variable data transfer rate.
Many systems that do it do it in "zones", rather than
changing every track.
It WILL require another parameter in the spec.
7267
days inactive
7267
days old
5 comments
6 participants
participants (6)
-
cisin＠xenosoft.com -
dvcorbin＠optonline.net -
Hans.Franke＠mch20.sbs.de -
julesrichardsonuk＠yahoo.co.uk -
melamy＠earthlink.net -
philpem＠dsl.pipex.com