10 Aug
2004
10 Aug
'04
11:04 a.m.
Ok, instead of all the unproductive bickering over semantics (most of
which is illogical...come on, admit it) I'd rather we actually discuss
developing further an open-source, public domain, free (in every sense of
the word) standard for archiving data media of all types (magentic, paper,
or otherwise).
This is basically going to be a continuation of what we've already
discussed on the list, and what Hans and I have discussed in private.
First, let's start with the goals.
The format should be:
1) Well Documented (with such documentation actively preserved in all four
corners of the globe and beyond)
2) Not constrained to any particular hardware
3) Be inclusive of all physical (and logical?) manner of recording media
4) Be implementable on even the simplest architectures (because the
original media source will in many cases have to be read on the hardware
it is connected to)
5) Open source, public domain, etc. (although a copyright may be held if
it makes sense to do so)
6) Adaptable, expandable, revisable (for future extensions)
7) Text-based and storable in commonly accessible character formats (i.e.
a suitable subset of Unicode, i.e. ASCII)
8) Allow for the representation of media in either logical or physical
(raw bit stream) formats
This is a good start. Someone please continue adding to the definition.
I will establish a special session at the next VCF (November 6-7) where
we can commence a committee for formalizing this standard and getting it
recognized internationally in all the various relevant groups (i.e. ANSI
or ISO/IEC).
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
10 Aug
10 Aug
12:17 p.m.
>>>> "Vintage" == Vintage
Computer Festival <vcf(a)siconic.com> writes:
Vintage> Ok, instead of all the unproductive bickering over semantics
Vintage> (most of which is illogical...come on, admit it) I'd rather
Vintage> we actually discuss developing further an open-source,
Vintage> public domain, free (in every sense of the word) standard
Vintage> for archiving data media of all types (magentic, paper, or
Vintage> otherwise).
Vintage> This is basically going to be a continuation of what we've
Vintage> already discussed on the list, and what Hans and I have
Vintage> discussed in private.
Vintage> First, let's start with the goals.
When you say "archiving" what span of time do you mean? A year? A
decade? A century? Longer?
You will get completely different answers depending on how you answer
that question.
(See also www.longnow.org)
paul
12:50 p.m.
On Tue, 10 Aug 2004, Paul Koning wrote:
When you say "archiving" what span of time
do you mean? A year? A
decade? A century? Longer?
Is forever too ambitious? :)
The idea is to create a standard for representing the data, not a medium.
Media with near-infinite longevity will be the next undertaking :)
(See also www.longnow.org)
Is there a specific section you want to refer me to?
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
1:22 p.m.
----- Original Message -----
From: "Vintage Computer Festival" <vcf(a)siconic.com>
To: "General Discussion: On-Topic and Off-Topic Posts"
<cctalk(a)classiccmp.org>
Sent: Tuesday, August 10, 2004 8:50 PM
Subject: Re: Let's develop an open-source media archive standard
Here is an example of a society that preserves media on the Amiga platform
(complete images including the copy protection as used by the developer).
http://www.caps-project.org/
They have spent allot of time deciphering custom disk formats and saving all
of the information digitally so it can be reproduced exactly at a later
date.
TZ
On Tue, 10 Aug 2004, Paul Koning wrote:
Festival
When you say "archiving" what span of
time do you mean? A year? A
decade? A century? Longer?
Is forever too ambitious? :)
The idea is to create a standard for representing the data, not a medium.
Media with near-infinite longevity will be the next undertaking :)
(See also www.longnow.org)
Is there a specific section you want to refer me to?
--
Sellam Ismail Vintage Computer
--------------------------------------------------------------------------
----
International Man of Intrigue and Danger
http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage
mputers ]
[ and academia at www.VintageTech.com || at
http://marketplace.vintage.org ]
3:07 p.m.
On Tue, 10 Aug 2004, Teo Zenios wrote:
Here is an example of a society that preserves media
on the Amiga platform
(complete images including the copy protection as used by the developer).
http://www.caps-project.org/
They have spent allot of time deciphering custom disk formats and saving all
of the information digitally so it can be reproduced exactly at a later
date.
That's a very cool project. The archives they are making could be
converted into the specification I'm talking about by writing a
translation program to take the raw bit images they are producing and
convert it into the specification markup language. Quite easily.
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
3:18 p.m.
On Tue, 10 Aug 2004, Vintage Computer Festival wrote:
On Tue, 10 Aug 2004, Teo Zenios wrote:
Very interesting:
http://www.caps-project.org/glossary.php?term=ipf
IPF (Interchangeable Preservation Format)
Glossary: 100, Last Updated: 2004-02-20
IPF stands for Interchangeable Preservation Format, and is the file format
we use to preserve content. It is a name for what is we use to store
(currently Amiga) floppy disk images. It is abstract for very good
reasons:
It is not meant just for the Amiga: The image format was designed to be
used for anything, and as such, it is possible for any platform to have
its media supported. In fact, some of the track formats used for Amiga
copy protection are actually PC/Atari anyway.
It is not just for disks: Any type of digital media can be contained by
the format. Currently it is used for disk images and ROM files, but in the
future it may possibly be used for CDROMs and information contained in
"dongle" protections.
The underlying file format has many similarities to the IFF
(Interchangeable File Format) standard on the Amiga, which was a very
forward-thinking format. The same techniques are used today in other file
formats such as PNG.
--
This is pretty much what I'm proposing.
I wonder where the description of their format is though? I can't find it
anywhere on their website.
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
Here is an example of a society that preserves
media on the Amiga platform
(complete images including the copy protection as used by the developer).
http://www.caps-project.org/
They have spent allot of time deciphering custom disk formats and saving all
of the information digitally so it can be reproduced exactly at a later
date.
That's a very cool project. The archives they are making could be
converted into the specification I'm talking about by writing a
translation program to take the raw bit images they are producing and
convert it into the specification markup language. Quite easily.
3:32 p.m.
----- Original Message -----
From: "Vintage Computer Festival" <vcf(a)siconic.com>
To: "General Discussion: On-Topic and Off-Topic Posts"
<cctalk(a)classiccmp.org>
Sent: Tuesday, August 10, 2004 11:18 PM
Subject: Re: Let's develop an open-source media archive standard
.
http://www.caps-project.org/download.shtml
The API and assorted documents are all there.
I wonder where the description of their format is though? I can't find it
anywhere on their website.
--
Sellam Ismail Vintage Computer
Festival
--------------------------------------------------------------------------
----
International Man of Intrigue and Danger
http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage
mputers ]
[ and academia at www.VintageTech.com || at
http://marketplace.vintage.org ]
3:35 p.m.
On Tue, 10 Aug 2004, Vintage Computer Festival wrote:
I wonder where the description of their format is
though? I can't find it
anywhere on their website.
Did you try:
http://www.caps-project.org/wip-jul-2002.shtml
3:28 p.m.
----- Original Message -----
From: "Vintage Computer Festival" <vcf(a)siconic.com>
To: "General Discussion: On-Topic and Off-Topic Posts"
<cctalk(a)classiccmp.org>
Sent: Tuesday, August 10, 2004 11:07 PM
Subject: Re: Let's develop an open-source media archive standard
Festival
On Tue, 10 Aug 2004, Teo Zenios wrote:
> Here is an example of a society that preserves media on the Amiga
platform
> (complete images including the copy protection as
used by the
developer).
>
> http://www.caps-project.org/
>
> They have spent allot of time deciphering custom disk formats and saving
all
of the
information digitally so it can be reproduced exactly at a later
date.
That's a very cool project. The archives they are making could be
converted into the specification I'm talking about by writing a
translation program to take the raw bit images they are producing and
convert it into the specification markup language. Quite easily.
--
Sellam Ismail Vintage Computer
--------------------------------------------------------------------------
----
The images they made can be run on WinUAE, and there is talk of using the
recently announced Catweasel V 4 for dumping the images back to floppy.
They have also started looking at doing the same preservation on the Atari
ST line, which ST people have problems with for some reason (namely Amiga
people doing the standards).
Without support from the people of the different platforms it would be hard
to get a universal program to work. While TOSEC.org has done a good job of
cataloging image files from multiple systems, most of their archives are for
programs that have been cracked, while CAPs is trying to preserver the
programs as they were sold with the copy protection intact.
4:18 p.m.
On Tue, 10 Aug 2004, Teo Zenios wrote:
The images they made can be run on WinUAE, and there
is talk of using the
recently announced Catweasel V 4 for dumping the images back to floppy.
They have also started looking at doing the same preservation on the Atari
ST line, which ST people have problems with for some reason (namely Amiga
people doing the standards).
Not surprising. I wish people could not act like infants all the time.
Another benefit of creating a universal standard is that it takes platform
wars out of the equation.
Without support from the people of the different
platforms it would be hard
to get a universal program to work. While TOSEC.org has done a good job of
cataloging image files from multiple systems, most of their archives are for
programs that have been cracked, while CAPs is trying to preserver the
programs as they were sold with the copy protection intact.
Platforms do not figure into the specification I have in mind. It would
be a specification. It could then be implemented on whatever platform
anyone cared to. As long as the various applications follow the spec,
images will be able to be stored where ever. Apple ][ users could store
Amiga images if they wanted to.
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
4:53 p.m.
----- Original Message -----
From: "Vintage Computer Festival" <vcf(a)siconic.com>
To: "General Discussion: On-Topic and Off-Topic Posts"
<cctalk(a)classiccmp.org>
Sent: Wednesday, August 11, 2004 12:18 AM
Subject: Re: Let's develop an open-source media archive standard
Platforms do not figure into the specification I have
in mind. It would
be a specification. It could then be implemented on whatever platform
anyone cared to. As long as the various applications follow the spec,
images will be able to be stored where ever. Apple ][ users could store
Amiga images if they wanted to.
--
Sellam Ismail Vintage Computer
Festival
--------------------------------------------------------------------------
----
Specifications are nice, but unless you get the people who still have the
hardware and software to join in it will get nowhere. The problem is getting
the people to follow the standards and image their disks, once the images
have been collected you can do what you want with them and not worry about
the platform they came from. CAPS software uses an Amiga 1200 platform to
get images from the original disks, Commodore 64 users can grab images from
a 1541/1571 drive connected to a PC running DOS connected to the drive with
a xe1541 cable and some software. As far as I know there is no hardware that
can precisely read all of the different disk formats and sizes let alone
deal with all copy protection methods (such as laser hole in the media
itself). You still need the original hardware of the platform it ran on to
get the images.. so you have to get the people of the different platforms
behind you (good luck).
5:36 p.m.
Teo Zenios wrote:
----- Original Message -----
From: "Vintage Computer Festival" <vcf(a)siconic.com>
To: "General Discussion: On-Topic and Off-Topic Posts"
<cctalk(a)classiccmp.org>
Sent: Wednesday, August 11, 2004 12:18 AM
Subject: Re: Let's develop an open-source media archive standard
I have an Atari Mega STE, a Commodore 128D with 1541, several Amigas
(but no Amiga 3.5" HD drives, dammit), an Amiga Catweasel, an AMAX-II
and Mac 5.25" external drive, and 68K Macs with 800K and 1.4MB floppy
drives. I have PCs with 5.25" and 3.5" drives that will do any of the
PC-compatible formats. Linux, MS-DOS, DR-DOS, NetBSD, and Win32 OSs
available.
I also have various Alphas, VAXen and PDP-11s with RX02, RX50, RX33,
and RX23 disk drives, as well as the later 2.88MB 3.5" floppy drive.
Add in RS/6000s with their various 3.5" formats ("PC standard" 720K,
1.44MB and 2.88MB, right?) and an Indigo with a known-good Floptical drive.
I don't write code, I don't know doodoo about floppy hardware that I
didn't learn here on CC, and I'm not always real quick to get projects
done, but I'm damned good at beta testing. I can break most anything,
but I can always tell *how* I broke it, and I can usually tell you where
and why it broke.
Long-winded way to say "I wanna help!" :)
Doc
Platforms do not figure into the specification I
have in mind. It would
be a specification. It could then be implemented on whatever platform
anyone cared to. As long as the various applications follow the spec,
images will be able to be stored where ever. Apple ][ users could store
Amiga images if they wanted to.
--
Sellam Ismail Vintage Computer
Festival
--------------------------------------------------------------------------
----
Specifications are nice, but unless you get the people who still have the
hardware and software to join in it will get nowhere. The problem is getting
the people to follow the standards and image their disks, once the images
have been collected you can do what you want with them and not worry about
the platform they came from. CAPS software uses an Amiga 1200 platform to
get images from the original disks, Commodore 64 users can grab images from
a 1541/1571 drive connected to a PC running DOS connected to the drive with
a xe1541 cable and some software. As far as I know there is no hardware that
can precisely read all of the different disk formats and sizes let alone
deal with all copy protection methods (such as laser hole in the media
itself). You still need the original hardware of the platform it ran on to
get the images.. so you have to get the people of the different platforms
behind you (good luck).
6:38 p.m.
On Wed, 11 Aug 2004, Teo Zenios wrote:
Specifications are nice, but unless you get the people
who still have the
hardware and software to join in it will get nowhere.
Well, that's part of developing a specification, isn't it? Getting buy-in
is of course critical, which is why I'm going to approach this from the
standpoint of creating an internationally recognized standard that has an
RFC and everything.
The problem is getting
the people to follow the standards and image their disks, once the images
have been collected you can do what you want with them and not worry about
the platform they came from.
Once the standard is in place then people can choose to use it or not. If
not, that's their loss...their archives will have a lesser chance of being
propagated, unless they get a lot of other people to do whatever it is
they're doing.
I would hope that people would see the value in a standard like I am
proposing and use it. It's not my intention to have just another format
that people can choose. No, this will be THE standard.
CAPS software uses an Amiga 1200 platform to
get images from the original disks, Commodore 64 users can grab images from
a 1541/1571 drive connected to a PC running DOS connected to the drive with
a xe1541 cable and some software. As far as I know there is no hardware that
can precisely read all of the different disk formats and sizes let alone
deal with all copy protection methods (such as laser hole in the media
itself). You still need the original hardware of the platform it ran on to
get the images.. so you have to get the people of the different platforms
behind you (good luck).
Exactly. And it's not a matter of luck, but rather a matter of survival.
Do you want the images of YOUR favorite platform to survive into the next
decade/century/millenium? Great, use the standard.
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
4:10 p.m.
At 08:22 PM 8/10/2004, Teo Zenios wrote:
Here is an example of a society that preserves media on
the Amiga platform
(complete images including the copy protection as used by the developer).
http://www.caps-project.org/
They have spent allot of time deciphering custom disk formats and saving all
of the information digitally so it can be reproduced exactly at a later
date.
Astounding! Will that computer never die? And I say that
as someone who Believed, '85-92.
I'm tempted to say that we should leave copy protection
hacks out of the spec for now, but if it was extensible,
that would be great.
- John
4:26 p.m.
On Tue, 10 Aug 2004, John Foust wrote:
Astounding! Will that computer never die? And I say
that
as someone who Believed, '85-92.
The Amiga is still going strong in some circles. More power to them.
I'm tempted to say that we should leave copy
protection
hacks out of the spec for now, but if it was extensible,
that would be great.
Yes, copy protection will defintely be able to be described naturally in
the specification I have in mind. The spec should be able to define
several layers of bit storage: logical (files, directories, etc.), byte
(e.g. tracks, sectors), and raw (bit streams). In this way, copy
protection schemes can be preserved by storing the image in the raw
format.
This will of course have to be thought out, and it may not even be
included in the first revision of the spec, but as I declared originally,
the spec will be extensible.
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
4:38 p.m.
I'm
tempted to say that we should leave copy protection hacks out of
the spec for now, but if it was extensible, that would be great.
Yes, copy
protection will defintely be able to be described naturally
in the specification I have in mind. The spec should be able to
define several layers of bit storage: logical (files, directories,
etc.), byte (e.g. tracks, sectors), and raw (bit streams). In this
way, copy protection schemes can be preserved by storing the image in
the raw format.
4:59 p.m.
On Wed, 11 Aug 2004, der Mouse wrote:
What about copy-protection schemes that work by
damaging the media to
the point where it doesn't _have_ a bit stream at a particular place?
That's been considered as well. There will simply be a header that
describes the error that occured at that particular point on the media
(either a specific hardware code or other).
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
4:44 p.m.
----- Original Message -----
From: "John Foust" <jfoust(a)threedee.com>
To: <cctalk(a)classiccmp.org>
Sent: Wednesday, August 11, 2004 12:10 AM
Subject: Re: Let's develop an open-source media archive standard
I went from a C64 to a 286/12 PC in the late 80's. In the last 2 years I
have acquired 3 Amigas (A500, A1200,A2000), they are interesting systems.
Backing up discs or programs that have been 'cracked' leads to problems when
you try to run update patches to old programs or when the 'cracker' did a
bad job on the program and it doesn't work like it should. I never seen a
crowd into preserving their machines like the Amiga crowd.
Examples:
The Largest Amiga hardware reference on the net:
http://www.amiga-hardware.com/
Amiga Hall of Light, database of Amiga games showing box scans and in game
screens (very well done):
http://hol.abime.net/
CAPS Preserving the original media:
http://www.caps-project.org/
Aminet, the largest internet based collection of freeware/shareware/GPL
Amiga software
http://ftp.uni-bremen.de/aminet/
WinUAE emulator for Amiga systems:
http://www.winuae.net/
WHD Load where people got together and reworked the older software so that
it ran on newer hardware, fixed bugs from the original programs, and allowed
the program to run from the HD (instead of flipping floppies as needed):
http://www.whdload.de/
Plus don't forget the active forums and other websites supporting a platform
where the manufacturer died 10 years ago.
If I want to add something to my collection the original media, manuals,
box, and working disks are what I look for, a perfect image will keep me
happy until I run into the original (or have the money to buy it if its is
rare). Since your going to the trouble of preserving something you might as
well preserve it exactly as it was released.
At 08:22 PM 8/10/2004, Teo Zenios wrote:
>Here is an example of a society that preserves media on the Amiga
platform
>(complete images including the copy protection as
used by the developer).
>http://www.caps-project.org/
>They have spent allot of time deciphering custom disk formats and saving
all
of the
information digitally so it can be reproduced exactly at a later
date.
Astounding! Will that computer never die? And I say that
as someone who Believed, '85-92.
I'm tempted to say that we should leave copy protection
hacks out of the spec for now, but if it was extensible,
that would be great.
- John
5:08 p.m.
On Tue, Aug 10, 2004 at 11:10:49PM -0500, John Foust wrote:
Astounding! Will that computer never die? And I say
that
as someone who Believed, '85-92.
I still enjoy playing the odd Amiga game (Black Crypt being one of the
best ones never ported to another machine, 'though the color version of
Larn is kinda nice compared to the vt100-compatible version that's available
for most UNIX machines).
I'm tempted to say that we should leave copy
protection
hacks out of the spec for now, but if it was extensible,
that would be great.
Extensible is good... one of the advantages of the Amiga IFF structure they
chose to emulate/use as a starting point, is that according to the spec, the
structure, tag formats, checksums, are all described in advance; what is
malleable is that if your application doesn't understand a tag, you are
obligated to ignore it, not puke 'cause it's in an unknown format. PICTs
are similar in that regard, a structured collection of (binary) records.
All programs know the structure, but not all programs can interpret all
record types.
These days, of course, XML can serve much the same function, plus have
the advantage of being printable. You could end up with files that could
never fit on the medium they describe, but unless you are trying to do
100% of your computing on classic platforms with tiny media, that shouldn't
be a problem (and even if you are... you _could_ fit a 1541 description
file on a 1581-compatible floppy, so even the C=64 could create and restore
these files to and from real media, given the right setup).
One suggestion... for the data portion of sectors/tracks/etc... let's spec
in a run-length encoding scheme so large portions of unused media compress
down really tightly (lots of zeros, or in the case of a real 1541 disc,
lots of hex 0x01s). If it's XMLish, it could look like
'<r="256">01</r>'
or some such.
-ethan
--
Ethan Dicks, A-130-S Current South Pole Weather at 11-Aug-2004 05:00 Z
South Pole Station
PSC 468 Box 400 Temp -65.4 F (-54.2 C) Windchill -97.8 F (-72.2 C)
APO AP 96598 Wind 9.1 kts Grid 019 Barometer 683.9 mb (10480. ft)
Ethan.Dicks(a)amanda.spole.gov http://penguincentral.com/penguincentral.html
6:41 p.m.
On Wed, 11 Aug 2004, Ethan Dicks wrote:
These days, of course, XML can serve much the same
function, plus have
the advantage of being printable. You could end up with files that could
never fit on the medium they describe, but unless you are trying to do
100% of your computing on classic platforms with tiny media, that shouldn't
be a problem (and even if you are... you _could_ fit a 1541 description
file on a 1581-compatible floppy, so even the C=64 could create and restore
these files to and from real media, given the right setup).
Keep in mind that the standard I'm proposing is completely human readable.
If it came to it and was necessary, a human could decode the image data.
The idea is that the images will be stored on modern machines with very
large data stores, though the standard will be simple enough to allow any
computer to implement it and, providing they have enough data storage, to
store the images as well.
One suggestion... for the data portion of
sectors/tracks/etc... let's spec
in a run-length encoding scheme so large portions of unused media compress
down really tightly (lots of zeros, or in the case of a real 1541 disc,
lots of hex 0x01s). If it's XMLish, it could look like
'<r="256">01</r>'
or some such.
Totally agreed.
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
11:53 p.m.
At 12:08 AM 8/11/2004, Ethan Dicks wrote:
Extensible is good... one of the advantages of the
Amiga IFF structure they
chose to emulate/use as a starting point, is that according to the spec, the
structure, tag formats, checksums, are all described in advance; what is
malleable is that if your application doesn't understand a tag, you are
obligated to ignore it, not puke 'cause it's in an unknown format.
In theory. It does little good if your app depends on something
that's not there, or if the truly vital bits are skipped by your
appplication.
PICTs
are similar in that regard, a structured collection of (binary) records.
All programs know the structure, but not all programs can interpret all
record types.
Yes, uhm, I wrote what were among the first color PICT image translators
on non-Mac platforms, licensed to ASDG and NewTek for their
paint programs, and another B/W version to SPC for an early version
of Harvard Draw. :-)
These days, of course, XML can serve much the same
function, plus have
the advantage of being printable. You could end up with files that could
never fit on the medium they describe,
Yes, and after a bit of browsing on that CAPS-Amiga site, I was amazed
at the XML and debugging work they've done to preserve copy protection
on obscure games. They were also quite aware that their data format
work would be able to record other disk formats. The site was a bit
over-grown, though, and I never found the spec itself.
- John
11 Aug
11 Aug
8:11 p.m.
New subject: PICT files
On Wed, Aug 11, 2004 at 06:53:35AM -0500, John Foust wrote:
Yes, uhm, I wrote what were among the first color PICT
image translators
on non-Mac platforms, licensed to ASDG and NewTek for their
paint programs, and another B/W version to SPC for an early version
of Harvard Draw. :-)
Then you'd be a good resource to ask... I have this Quicktake 150 camera...
when dumped on a Mac, it produces (I'm told) Quicktime Compressed PICT
files. In the past, when dumping this camera, any app (Photoshop,
ImageConverter, etc.) can handle these files _if_ you have loaded the QT150
software on there first. I found no way to process these files with Open
Source tools. There were tools that could handle PICTs, but they puked on
the payload portion of the file. They _knew_ there was a chunk there, but
didn't know what to do with it.
Do you happen to know of any way to digest these files except with the right
libraries loaded on a real Mac?
-ethan
--
Ethan Dicks, A-130-S Current South Pole Weather at 12-Aug-2004 08:00 Z
South Pole Station
PSC 468 Box 400 Temp -65.7 F (-54.3 C) Windchill -98.3 F (-72.40 C)
APO AP 96598 Wind 9.19 kts Grid 077 Barometer 685.2 mb (10432. ft)
Ethan.Dicks(a)amanda.spole.gov http://penguincentral.com/penguincentral.html
4:33 a.m.
Ethan Dicks wrote:
One suggestion... for the data portion of
sectors/tracks/etc... let's spec
in a run-length encoding scheme so large portions of unused media compress
down really tightly (lots of zeros, or in the case of a real 1541 disc,
lots of hex 0x01s). If it's XMLish, it could look like
'<r="256">01</r>'
or some such.
Also timing needs to be done too of the media. A laser hole in the disk
could
rely on software timer that is interupted when the floppy disk controler
gets
a I/O error. I know the Color Computer uses software and hardware wait
instruction
to do the dma transfer, and other sneeky stuff. Also what about a drum
memory
where the timeing track needs to recorded.
-ethan
Ben.
6:32 a.m.
On Wed, 11 Aug 2004, ben franchuk wrote:
Also timing needs to be done too of the media. A laser
hole in the disk
could rely on software timer that is interupted when the floppy disk
controler gets a I/O error. I know the Color Computer uses software and
hardware wait instruction to do the dma transfer, and other sneeky
stuff. Also what about a drum memory where the timeing track needs to
recorded.
Interesting. Worth keeping in mind.
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
7:03 a.m.
On Wed, 11 Aug 2004, Ethan Dicks wrote:
. . . You could end up with files that could
never fit on the medium they describe,
. . .
One suggestion... for the data portion of sectors/tracks/etc... let's spec
in a run-length encoding scheme so large portions of unused media compress
down really tightly (lots of zeros, or in the case of a real 1541 disc,
lots of hex 0x01s). If it's XMLish, it could look like
'<r="256">01</r>'
or some such.
Initially, I was uncomfortable about the added complications
involved in compression. But,... the ability to store an image on
the same kind of media that it was created from is a MAJOR bonus.
Consider that there are some machines for which their basic
floppy media is the ONLY practical way [short of Dyna-Typer] to
get data or disk images into them. Therefore, an important
extension to consider would be any form of lossless compression.
LZW would be good for the entire image file, but not well
suited for compression in individual blocks.
1:18 a.m.
>>>> "Vintage" == Vintage
Computer Festival <vcf(a)siconic.com> writes:
Vintage> On Tue, 10 Aug 2004, Paul Koning wrote:
> When you say "archiving" what span of
time do you mean? A year?
> A decade? A century? Longer?
Vintage> Is forever too ambitious? :)
Vintage> The idea is to create a standard for representing the data,
Vintage> not a medium. Media with near-infinite longevity will be
Vintage> the next undertaking :)
The point of the question: what assumptions are you making about the
people who are trying to read your archive N years from now? Are they
assumed to be using computers? If so, what kind?
To look at it differently -- if you handed someone a CDROM, and a
stack of standards describing it (ISO 9660, the Red Book, etc.) but no
computer and no CDROM drive, is that person likely to have enough
information to recover the data on the disk? I expect the answer is
"no, not even close".
> (See also www.longnow.org)
Vintage> Is there a specific section you want to refer me to?
It's all worth reading, but
http://www.longnow.org/10klibrary/library.htm seems directly relevant
to the discussion.
paul
5:50 a.m.
On Wed, 11 Aug 2004, Paul Koning wrote:
I read that, and I understand the concerns expressed. But a first step is
to actually develop a universally recognized format to archive the data
The point of the question: what assumptions are you
making about the
people who are trying to read your archive N years from now? Are they
assumed to be using computers? If so, what kind?
To look at it differently -- if you handed someone a CDROM, and a
stack of standards describing it (ISO 9660, the Red Book, etc.) but no
computer and no CDROM drive, is that person likely to have enough
information to recover the data on the disk? I expect the answer is
"no, not even close".
Well, that's hard to say really. Whose to know what technology people
will have access to in the future? Whose to say they will be speaking any
language we use today? There are a lot of ifs. All we can do is make a
standard and live by it, and then hope its useful to future generations.
> (See also
www.longnow.org)
Vintage> Is there a specific section you want to refer me to?
It's all worth reading, but
http://www.longnow.org/10klibrary/library.htm seems directly relevant
to the discussion. from various media into. The hard part is coming up
with either a
technology that can last millenia, or a system that is foolproof and
will
make sure the archive data is propagated throughout generations.
I'm focusing right now on the easy part.
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
11:23 a.m.
I read that, and I understand the concerns expressed.
But a
first step is
to actually develop a universally recognized format to
archive the data
from various media into. The hard part is coming up with either a
technology that can last millenia, or a system that is
foolproof and will
make sure the archive data is propagated throughout generations.
There someone suggesting that the Aliens SETI is looking
for might well have stored there messages right under our
noses - or rather in them, right in the DNA.
Pretty low bandwidth propogation mechanism, but I guess
if we design a small creature with lots of junk DNA we
can store stuff in ... Of course, we'd need a pretty
powerful ECC to be able to retrieve the data in a
few thousand years!
Antonio
--
---------------
Antonio Carlini arcarlini(a)iee.org
7:26 a.m.
On Wed, 11 Aug 2004, Paul Koning wrote:
To look at it differently -- if you handed someone a
CDROM, and a
stack of standards describing it (ISO 9660, the Red Book, etc.) but no
computer and no CDROM drive, is that person likely to have enough
information to recover the data on the disk? I expect the answer is
"no, not even close".
If they have retained sufficient technological competence
(not another "dark ages"), and IFF the need is enough to
warrant devoting enough resources to the project, then YES.
Neither a CDROM, nor a floppy drive, is a particularly
difficult item to build with appropriate facilities.
But "enough resources" might indeed be fairly extreme, and
one goal is to reduce the amount of resources needed.
'course they could always send a time traveller, and in
addition to the 5100, have him make another stop (at FRY's??)
to shop for some hardware.
11:13 a.m.
To look at it differently -- if you handed someone a
CDROM, and a
stack of standards describing it (ISO 9660, the Red Book, etc.) but no
computer and no CDROM drive, is that person likely to have enough
information to recover the data on the disk? I expect the answer is
"no, not even close".
I would hope that the standards give you enough information
to be able to build a CDROM drive and read back the bits
on there. I'm assuming that the stack of standards includes
information on the way the bits hang together too (the
file-system or whatever).
You ould probably not need to build the CDROM drive
for just one CD. All you need, in principle, is a
high resolution microscope and either a lot of time
or some image processing software.
What is it that you think is not properly described in
the standards?
Antonio
--
---------------
Antonio Carlini arcarlini(a)iee.org
10 Aug
10 Aug
9:41 p.m.
On Tue, 2004-08-10 at 23:04, Vintage Computer Festival wrote:
The format should be:
1) Well Documented (with such documentation actively preserved in all four
corners of the globe and beyond)
2) Not constrained to any particular hardware
3) Be inclusive of all physical (and logical?) manner of recording media
4) Be implementable on even the simplest architectures (because the
original media source will in many cases have to be read on the hardware
it is connected to)
5) Open source, public domain, etc. (although a copyright may be held if
it makes sense to do so)
6) Adaptable, expandable, revisable (for future extensions)
7) Text-based and storable in commonly accessible character formats (i.e.
a suitable subset of Unicode, i.e. ASCII)
8) Allow for the representation of media in either logical or physical
(raw bit stream) formats
Agreed with all those. Can I add some important things:
o) The format should be able to note media errors (I still want to
archive damaged disks whilst some data can be recovered from them)
o) The format should be able to cope with hard disk images too (i.e. not
make assumptions about only 1 or 2 media surfaces!)
o) Aside from the archive obviously recording geometry of the source
media, the following data is important per archive:
who created it
when it was created
what hardware it was created on (at least drive model,
and any protocol bridge that the drive was connected to)
what the archive represents
what platform / system the archive is for
freeform description field (for any other notes)
o) Archive format should not be tied to any particular software (i.e. no
assumption is made about what utility is used to read/write the archive
files)
o) Each archive should record the version of the spec against which it
was created. Yes, you said that yourself in point 6, but it's darn
important :)
I'm actually currently playing around with this sort of thing anyway,
and for the human-readable side of configuration I'm favouring simple
parameter = value ASCII data contained within declared sections.
Currently sections are non-nestable, and I'm hoping they don't need to
be as it makes things a lot cleaner; we shall see. Configuration is
line-based (UNIX \n only), whitespace and case for parameter names is
*not* significant (so "A Parameter", "aparameter" and "a paraM
eter"
are all equivalent as far as parsing's concerned - can't remember if I
borrowed that idea from the apache or samba config!). Comments are
supported, and yadda yadda yadda...
I've only got around to defining the config for the SCSI floppy
controller board to which the floppy drives are connected, so that I can
read the hardware configuration from a config file - but my plan is/was
to use the same config style for the floppy disk images.
Disk image archive format would be seperate from the hardware of course;
indeed I've got ST506 drives hooked up to various SCSI-ST506 bridge
boards in old machines (Omti, Adaptec etc.) and intend to use the same
archive format for those.
I still like the idea of embedding the file format definition itself as
commented data within each archive file - or at least a subset of it.
That way in 30 years when someone comes across an archive in this format
but the spec's long disappeared off the web, there's enough information
within the file itself telling them how to read it.
Ahh, fond memories of the days of upsetting customers because I designed
everything to have a lifetime of at least decades - and they couldn't
see beyond the next six months :-)
cheers
Jules
11 Aug
11 Aug
10:52 a.m.
Agreed with all those. Can I add some important
things:
o) The format should be able to note media errors (I still want to
archive damaged disks whilst some data can be recovered from them)
The format should be extendable!. We can't possibly think of everything
now, so it's essential that more stuff can be added later.
o) The format should be able to cope with hard disk images too (i.e. not
make assumptions about only 1 or 2 media surfaces!)
Another useful thing would be to handle devices that are regarded as just
a linear array of blocks (in other words with no explicit tracks and
sectors).
Also, to be able to note things like file marks, etc so that magtapes can
be archived in the same type of file.
o) Archive format should not be tied to any particular
software (i.e. no
assumption is made about what utility is used to read/write the archive
files)
Agreed. And to go along with that, there should be no assumption about
big/little endian machines, word size, or even what character set is used
(I think we're agreed that only printable characters, newlines, and not
much else should be used).
I think others have said this, but the binary data in a sector/block
should be recorded in some hex format. Yes, this wastes space in the
archive, but it makes it human readable. And if you want to save space on
the machine you're storign the archives on, you can always gzip them or
whatever.
-tony
7441
days inactive
7443
days old
31 comments
12 participants
participants (12)
-
a.carlini＠ntlworld.com -
ard＠p850ug1.demon.co.uk -
bfranchuk＠jetnet.ab.ca -
cisin＠xenosoft.com -
dickset＠amanda.spole.gov -
doc＠mdrconsult.com -
jfoust＠threedee.com -
julesrichardsonuk＠yahoo.co.uk -
mouse＠Rodents.Montreal.QC.CA -
pkoning＠equallogic.com -
teoz＠neo.rr.com -
vcf＠siconic.com