Hello, all:
I wanted to drop everyone a note to let you know that the latest version of
the Altair32 Emulator was released this evening. It has been more than a
year since the last update and a lot of work has gone into bug hunting and
enhancing operational performance. Scott made the timeslicer multi-threaded,
so now the Altair32 can achieve prototypical performance with a machine as
slow as a Pentium-III/500MHz (the slowest machine available for testing) at
a greatly reduced host system load (10-20%).
There are a few things on the drawing board for the next version. First, we
will be adding support for a to-be-named color graphics board. Second,
hopefully, we will be adding support for an operational Altair (or IMSAI)
front panel with a machined metal dress panel in the case of the Altair, or
a screenprinted plexiglass one for the IMSAI.
Thanks again for all of your support with this project, and enjoy. As
always, if anyone finds any bugs or anomalies in the code, please let me
know.
Rich
Rich Cini
Collector of classic computers
Build Master for the Altair32 Emulation Project
Web site: http://highgate.comm.sfu.ca/~rcini/classiccmp/
Altair32 Page: http://highgate.comm.sfu.ca/~rcini/classiccmp/Altair32.htm
/************************************************************/
> Decmate Software, copyright 1985, DM/WPS utility
> diskette 2.0 and DM/WPS system diskette v2.0. Don't know which version of
> decmate they are for, but they are 5.25 inch, not 8 inch.
These are the basic wordprocessing which is called WPS and should work on
the Decmate II and III.
Since the RX50s held 400K on each floppy they were fine for wordprocessing
and two RX50s was a common configuration. You could keep DM/WPS in drive A,
the Utility disk or another in Drive B and Data disks in C and D.
CPM was an option with a piggyback processing card as was a hard drive.
Neither are commonly found in Decmate IIs
Paxton
Astoria, OR
I agree in all what Hans had to say except for how data would be stored. This is supposed to be an archive format which would in my view preclude getting data from the outside world. My only other concern as I have stated before was that data should not be an integral part of the media. If a device happens to be a tape drive, the data on the tape still separates into "file" type data and overhead data required for the tape physical format. Enbedding the "file" tyupe data inside of the physical format makes the data inaccessible without special knowledge.
best regards, Steve Thatcher
-----Original Message-----
From: Hans Franke <Hans.Franke(a)siemens.com>
Sent: Aug 12, 2004 9:30 AM
To: "General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
Subject: Re: archive file format exmaple
> I'm not so sure about having the actual data within the archive element;
> I'd rather have it afterwards.
I would prefer an aproach where both ways are possible, includeing
a third wich allows to put the data even into another file. In the
CCDD definition almost every element (except META) may be empty and
have a HREF attribute pointing to it's content:
A take header record could have it's data included:
<DATA SIZE="80" ENCODING="CHAR" FILLER=" ">HDR2U020480204841 00</DATA>
or just point to the data
<DATA HREF="#DB1"/>
<DATA SIZE="80" ENCODING="CHAR" FILLER=" " ID="DB1">HDR2U020480204841 00</DATA>
which of course may even be in another file:
<DATA HREF="http://cc-archive.org/~hans/mytapes/T43782.xml#DB1"/>
I looked through my floppy library and came across three floppies that
might help.
They are labelled Decmate Software, copyright 1985, DM/WPS utility
diskette 2.0 and DM/WPS system diskette v2.0. Don't know which version of
decmate they are for, but they are 5.25 inch, not 8 inch. I don't think
I'll ever need them, but maybe somebody can put them out on the web?
Also, I haven't tried to read them, they may be bad, but I don't know.
Joe Heck
I have a MicroTechnology MLV11M, which connects to a MFM ST412 type of disk
and emulates a DEC RL02. I used to have the manual and formatting software
for this but they got lost.... It's a long story.
What I am looking for is a utility called RL02DG.SAV which runs under RT11
and allows you to format a disk connected to the controller. Anyone have
this very old software? Circa 1985 perhaps.
Doug
On Aug 12, 2004, at 3:34 PM, cctalk-request(a)classiccmp.org wrote:
[...rip...]
> There were tools that could handle PICTs, but they puked on
> the payload portion of the file. They _knew_ there was a chunk there,
> but
> didn't know what to do with it.
>
> Do you happen to know of any way to digest these files except with the
> right
> libraries loaded on a real Mac?
>
> -ethan
[...too cold to even consider...]
For the Mac environment get ye a copy of *Graphic Converter*
<http://www.lemkesoft.de/en/graphcon.htm> shareware. Takes in about
anything format-wise and will put out anything else.
CRC
I saw this stuff at the surplus dealer today. I don't know anything about
this, but it sounded/looked list stuff some here may be interested in.
1) Motorola EVB/EVM/EVS Software disk with an M68HC05EVM board
2) P&E Microsystems ICS05JPW In Circuit Simulator for Windows diskette, plus
M68ICS05JP board
3) MMDS08 or MMD508 software and lots of manuals, plus M68EM08 board
4) M68EVB912B32 circuit board
They had a price tag of $25 on each of the 4 items above. Each one seemed
pretty complete, with lots of papers, diskettes, PLCC adapter with ribbon
cable, power supplies, serial cable to hook up to host PC, etc.
Anyways, thought folks might have interest.
Jay
A guy contacted me back on the 10th offering me a complete line of Vector
Graphic Computers. Hardware, manuals (tech & user), and sales literature for
all of it, free for pickup. Asked him the same day when and were could I
pick it all up and today I get my answer back saying it's all gone to a
startup computer museum at the UT-Arlington campus. Talk about bad luck and
everyone here in Texas wanting to start a computer museum.
that is real simple. You have a physical section that says it is a sequential block of data. You point to the data block that has a length. End of image file in the case.
If you know nothing about the format, all you can do is save a sequential data block.
I would still separate the two out for strictly consistency sake, so you can get the data easiliy to analyze. Once anlyzed, the a new image file could be generated with more info.
best regards, Steve Thatcher
-----Original Message-----
From: Vintage Computer Festival <vcf(a)siconic.com>
Sent: Aug 12, 2004 5:59 PM
To: Steve Thatcher <melamy(a)earthlink.net>,
"General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
Subject: Re: archive file format exmaple
On Thu, 12 Aug 2004, Steve Thatcher wrote:
> I agree in all what Hans had to say except for how data would be stored.
> This is supposed to be an archive format which would in my view preclude
> getting data from the outside world. My only other concern as I have
> stated before was that data should not be an integral part of the media.
> If a device happens to be a tape drive, the data on the tape still
> separates into "file" type data and overhead data required for the tape
> physical format. Enbedding the "file" tyupe data inside of the physical
> format makes the data inaccessible without special knowledge.
Steve,
What if you don't know what a tape you are archiving contains in the first
place? What if you just want to preserve it in case someone else in the
future can figure it out?
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
comments below...
best regards, Steve Thatcher
-----Original Message-----
From: Vintage Computer Festival <vcf(a)siconic.com>
Sent: Aug 12, 2004 3:44 PM
To: "General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
Subject: RE: Let's develop an open-source media archive standard
On Wed, 11 Aug 2004, Steve Thatcher wrote:
> I did not say it was the primary purpose. The data blocks work hand in
> hand with the formatting information. It is fine to make a standard
> extensible, but what good does it do if the (I hate the use the word)
> file can't be gotten without jumping through major hoops because how the
> data was stored in the image file wasn't extended out to make blocks. If
> you have to iterate through formatting information to get data then you
> have to be intimately familiar with the disk format in use. It means any
> GENERAL utility to read an image file for data access will have to KNOW
> about all machione supported rather than just getting some type of data
> identifier and reading the data out.
>
> I think part of the problem here is that the word file is being taken
> literally to mean filename, size, data, etc. I am using in the context
> of a block of data. It has NOTHING to do with how the data is on the
> disk, where it is stored, the recording format, etc. It is just a piece
> of data...
One way to resolve this is to put comments in the image file pointing out
where a block of data ("file") starts and how long it is. This can also
be built into the spec as a tag, rather than relying on human readable
comments to effect this.
*** I agree comments could help explain how to get info out of the image file, but if you don't create data blocks in a fashion that implies knowledge of the file system, then you can not just simply point at the data block and say how long it is. If you are talking about my propsed concept then you can just point at a data block and go get it.
> I am talking about two separate sections inside the same image file. One
> of the sections contains data blocks. The other has specific formatting
> information and POINTS to the data in the data blocks. A utility program
> could then SIMPLY get any type of data out of the image file without the
> utility being out of date as soon as someone added a new physical
> format.
The spec COULD be made to allow for this, and I don't see a reason why
not. It's just an organizational attribute. I'll add it to the notes.
*** great, I would hope that people who are archiving disks will use the data blocks rather than embedding the data in the physical format. It just makes life easier for everyone.
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
comments below...
best regards, Steve Thatcher
-----Original Message-----
From: "Dwight K. Elvey" <dwight.elvey(a)amd.com>
Sent: Aug 12, 2004 3:09 PM
To: cctalk(a)classiccmp.org
Subject: RE: Let's develop an open-source media archive standard
>From: "melamy(a)earthlink.net" <melamy(a)earthlink.net>
>
---snip---
>
>The image file that you are discussing is only good as a "how to" to write
>arbitrary data to arbitrary media. It is only good for creating a final
>media image because the format being contemplated (mixing data into the
>physical spec) will preclude the file from standing on its own.
No, that is not true. The description of the physical data in the archive
should be sufficient to recover the system side data. I can state
this because the system side data is just a subset of the physical
data.
*** if all you are describing is tracks, sectors and heads then you have not included ANY information as to how to get at a specific piece of data in the image file. You can certainly get a sector piece but you will have no clue how to connect individual pieces together to make a block of data. Only platform specific information will get you to that point and that means text has to be included to tell someone how to write software that accesses a particular file structure on a particular OS. If you have that file information when the image file is created, it would be easy to save the info and then be able to get at any file inside of the image file without ANY knowledge of the physical file structure or OS.
I recommend that the archive should include enough information to
translate from the physical to the system but for archiving purposes,
that is not absolutely necessary.
*** this information is not trivial, but if we make the data separate then it is if we save the file information too.
It is not that hard to describe as
part of the header what FM looks like. Still, if we restrict all archived
information to include this, some unknown formats may not be archived
because the person with the data doesn't know that format. It is
better to capture the data first.
It may also be that the only way that person has to capture the
data is the output of a controller chip. The archiving should allow
this as well ( more in the format the Steve would like it all to
be in ).
*** not sure why this relates to the format I was proposing. What I was desdcribing was a way to do both low level bytes as well as blocks of data
This file could later be combined with the more physical
information when that was available, just as a archive file that
originally had only physical information could be appended with the
data as seen by the system.
What this means is that some of the archived files may not be directly
useable by Steve and his tools without some more in depth knowledge
about the encoding use. It doesn't make it impossible to use, just
difficult.
*** is is not a encoding issue with regards to the hardware level. There are three standards just for 8" disks - FM, MFM, and M2M. ISIS DD uses M2M and I for one want to be able to use my ISIS software independent of the media it is on. I also don't want to have to write a major utility (in other words a small OS) just to read the image file.
Dwight
>I was
>proposing identifying data blocks as files if that is what they were. The
>data blocks would have been sequential and trivial to retrieve. That is
>hardly what I would call defining a file system as part of the image file.
>
>best regards, Steve Thatcher
>
no mention of age and unfortunately I am old enough to have worked on and designed predicatbale systems...
-----Original Message-----
From: "Dwight K. Elvey" <dwight.elvey(a)amd.com>
Sent: Aug 12, 2004 6:41 PM
To: cctalk(a)classiccmp.org
Subject: Re: Let's develop an open-source media archive standard
Hi
I was more referring to a modern PC than specific
older hardware. The issue with doing direct I/O
is the it needs to be predictable and on time.
Most modern machines can no longer to that reliably
without using DMA. They have too may other things
that they are expected to do simultaneously. Also
because of multiple levels of caching, real-time
predictability is not practical.
In the N*, they dedicate a predictable processor
to do just the disk I/O and nothing else.
Dwight
>From: "Steve Thatcher" <melamy(a)earthlink.net>
>
>I seem to recall that my slow old N* Horizon was doing dd at 4mhz with no dma -
in fact it was polled I/O because their wait for I/O available kept locking up
so I modified it.
>
>best regards, Steve Thatcher
>
>-----Original Message-----
>From: Fred Cisin <cisin(a)xenosoft.com>
>Sent: Aug 12, 2004 5:12 PM
>To: "General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
>Subject: Re: Let's develop an open-source media archive standard
>
>On Thu, 12 Aug 2004, Dwight K. Elvey wrote:
>> Hi Jules
>> Here is what I've found. It is a disk drive emulator.
>> Unless a PC is DMA driven, bit banging a floppy is not practical.
>
>Disk I/O without DMA is not practical. But it IS possible.
>Consider the PCJr and Tandy 1000, both of which do disk I/O
>without DMA on 4.77MHz? machines.
>
>
>
Hi
I was more referring to a modern PC than specific
older hardware. The issue with doing direct I/O
is the it needs to be predictable and on time.
Most modern machines can no longer to that reliably
without using DMA. They have too may other things
that they are expected to do simultaneously. Also
because of multiple levels of caching, real-time
predictability is not practical.
In the N*, they dedicate a predictable processor
to do just the disk I/O and nothing else.
Dwight
>From: "Steve Thatcher" <melamy(a)earthlink.net>
>
>I seem to recall that my slow old N* Horizon was doing dd at 4mhz with no dma -
in fact it was polled I/O because their wait for I/O available kept locking up
so I modified it.
>
>best regards, Steve Thatcher
>
>-----Original Message-----
>From: Fred Cisin <cisin(a)xenosoft.com>
>Sent: Aug 12, 2004 5:12 PM
>To: "General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
>Subject: Re: Let's develop an open-source media archive standard
>
>On Thu, 12 Aug 2004, Dwight K. Elvey wrote:
>> Hi Jules
>> Here is what I've found. It is a disk drive emulator.
>> Unless a PC is DMA driven, bit banging a floppy is not practical.
>
>Disk I/O without DMA is not practical. But it IS possible.
>Consider the PCJr and Tandy 1000, both of which do disk I/O
>without DMA on 4.77MHz? machines.
>
>
>
Ok, Here's another pet project of mine.. I Have a simple DOS6.22
installed in one of my laptops, and
I want to make it look like as much like an IBM (it's an IBM laptop too
btw) mainframe as possible. I am working
on a vauge memory from a job over 10 years ago when I had an account on
the R&D little mainframe.
I think the machine was an early AS400, It filled 3 Equipment Racks.
This was about the time of the Loma-Preata Earthquake (for non
california readers, I forget the year but
both San Francisco and Oakland, and San Jose were all effected by it,
and that was the one where the
freeway collapsed and many of the buildings in SF's Marina District
suffered from liquifacation)
I think it ran somthing like VM/OS, anyway it had Xedit for a system
editor and I have "the heisling
editor" which is very much like xedit.
Can any one capture screens from a Mainframe like this? I'd love to see
them..
Esp. the login screen, A directory/datset screen (where in you put an
X near a file and hit
enter to edit the file.) and any other sorts of menu screens.
Thanks.
>From: "Fred Cisin" <cisin(a)xenosoft.com>
>
>On Thu, 12 Aug 2004, Dwight K. Elvey wrote:
>> Hi Jules
>> Here is what I've found. It is a disk drive emulator.
>> Unless a PC is DMA driven, bit banging a floppy is not practical.
>
>Disk I/O without DMA is not practical. But it IS possible.
>Consider the PCJr and Tandy 1000, both of which do disk I/O
>without DMA on 4.77MHz? machines.
>
>
True, but it does require the hardware to be setup for it.
I have a Forth processor ( NC4000 ) that I talk directly
to the controller chip without DMA. Newer processors are
faster but may have latency issues. Using a dedicated uP
or some FPLA is the right way to handle it.
Of course the problem is multiplies when you have no controller
chip and you are talking about raw bit streamed data.
It was mentioned earlier that for logging bit stream data,
the gaps my have trash in them that causes loss of sync. This
should also be handled in the specification.
Dwight
How would one actually go about re-generating an original media from
the metafile? Do we contemplate connecting some future computer's I/O port
to a 34-pin ribbon cable connected to a 1980's vintage floppy drive? At some
point in this process we're going to have to make some detailed assumptions
on how the metadata will be used 50 or 100 years from now.
Also, the metafile not only has to include information about the
"user" data areas of the disk but also the system areas (the stuff written
to the media by the controller -- address marks, gaps, sync bits, etc.).
This would require us to not only compile the general media format
data but also data on the controller used to generate the media (chip specs,
data gleaned from examining the "format" programs used, etc.).
The reason why I ask is that somehow we're going to have to test the
archival/restoration process to see that it works. It's like making tape
backups but never testing them with a restore.
This might be obvious, but I've been accused of stating that before
:-)
Rich
-----Original Message-----
From: cctalk-bounces(a)classiccmp.org
[mailto:cctalk-bounces@classiccmp.org]On Behalf Of Steve Thatcher
Sent: Thursday, August 12, 2004 10:17 AM
To: General Discussion: On-Topic and Off-Topic Posts
Subject: RE: Let's develop an open-source media archive standard
comments embedded below...
best regards, Steve Thatcher
-----Original Message-----
From: Hans Franke <Hans.Franke(a)siemens.com>
Sent: Aug 12, 2004 10:06 AM
To: Steve Thatcher <melamy(a)earthlink.net>,
"General Discussion: On-Topic and Off-Topic Posts"
<cctalk(a)classiccmp.org>
Subject: RE: Let's develop an open-source media archive standard
Am 11 Aug 2004 14:47 meinte Steve Thatcher:
> I realize that the idea is to create a format to make re-creation
> of media possible for a variety of platforms. We can certainly do
> that and have its only function be to maintain a physical data
> format. My added idea is that the data and the formatting be
> separated so that a simple utility on a non-target platform could
> extract data from the image file.
Well, physical format and data is not always seperable. In some
circumstances the physical format is part of the onformation an
application needs, and differs form media to media (e.g. copy
protection schemes)
*** the separated data that I am talking about is what would be accessed
through normal OS channels. Copy
protection schemes, special destroyed sectors, etc are not accessible
through the OS.
> If we create a physical description only and do not abstract
> the data then any emulator must understand the OS file
> structure in order to retrieve any internal file representation.
> My idea would make the file re-creation simple in that the xml
> image file would be parsed for the actual file data that an
> emulator would need. This makes the emulator easier.
But what if the emulator needs the physical format information?
*** I have not proposed that the physical format information be excluded...
> To retrieve a file from the physical layout that it at the end
> of this message, the emulator must know the actual disk format
> that is used on the target system (the one the image file was
> made for). I have seen cp/m systems where the actual physical
> sectors were sequential on disk and the OS file sector was
> actually virtual to increase speed. Not my idea of the way to
> do it. It is much easier to make the physical sectors slewed
> so that a physical sector is a file sector. These are the types
> of issues you will have to overcome if an emulator must totally
> understand each and every file system for a cp/m version for
> example.
At least within a CP/M system it usualy doesn't matter at all
how the files are stored on a disk. Except for some odd apps
who tried to implement system specific copy protection schemes,
all and every CP/M app accesses files just via BDOS which already
hides the real disk strukture.
*** talk to other people here and one of their arguements for keeping things
all together was being able to
access track and sector directly. As for BDOS, that is fine, but keep in
mind that BDOS was customized for nearly every cp/m platform
>From: "Fred Cisin" <cisin(a)xenosoft.com>
>
>I think that capability of including comments is essential!
>
>For example:
>
><COMMENT>CRC error on disk. Not yet determined whether it is
>a read error, or a deliberate component of copy protection
></COMMENT>
>
>or
><COMMENT>Note that HEAD NUMBER field in sector header is wrong.
>Machine uses WD controller, and doesn't care about that field,
>therefore, the incorrect value does not need to be replicated
>for normal use. </COMMENT>
>
>
Hi
A must.
Dwight
>From: "Steve Thatcher" <melamy(a)earthlink.net>
From: "Dwight K. Elvey" <dwight.elvey(a)amd.com>
---snip---
> It may also be that the only way that person has to capture the
>data is the output of a controller chip. The archiving should allow
>this as well ( more in the format the Steve would like it all to
>be in ).
>
>*** not sure why this relates to the format I was proposing. What I was
desdcribing was a way to do both low level bytes as well as blocks of data
>
Hi Steve
Again, you've missed the point here. The information may not
have anything other than the bit stream and the track. The
person extracting the data may have no knowledge of the
format ( FM, MFM, M2FM, RLL or whatever ). They are just
archiving the data of the disk. I agree that if there is
sufficient information available to include such things
as sector boundaries, that should be encoded in some method.
This may not be in the actual data but as part of the format
description ( requires some work to extract from the data ).
At the lowest level, the archive may only contain a bit stream
that correlates to the signal coming from the drive. Without
information as to the encoding method, this may be useless
to you. As Sellam has stated, your application is secondary
to actually capturing a reproduceable medium.
Obviously, most people will not be able to create such information
( I believe I will ). Most will be creating such things as the
output data from some standard disk reading chip. These will
surely have some form of partioning, either in the header or
embedded in the data.
This could still be a valid archive input for standard formats.
Note the word 'could' and not must.
Dwight
here is a "crude" example of what I was talking about.
try:
<archive image file version="1.0">
<definitions>
a place to put whatever relevant information is needed about the format or achive
or your favorite poem or joke
</defintions>
<target system>Tandy Model 100</target system>
<author>Ian Blindly</author>
<media>
<format>5.25" floppy</format>
<default-sector-data value="$AA"/>
<encoding>MFM? RLL? something ... </encoding>
<tracks>35</tracks>
<heads>1</heads>
<sectors size="256">18</sectors>
<wordsize>8</wordsize>
</media>
<datamap>
<head physical="0">
<track physical="0">
<sector logical="1" physical="4" datablock="DB1"
dataitemid="SB1" />
<sector logical="2" physical="8" datablock="DB2"
dataitemid="SB1"/>
<sector logical="3" physical="12" fill="$00"
/>
<sector logical="4" physical="16" fill="$00"
/>
...
</track>
<track physical="1">
<sector logical="1" physical="4" fill="$FF"
/>
<sector logical="9" physical="12" datablock="DB3"
dataitemid="SB1"/>
</track>
<track physical="22" datablock="DB3" dataitemid="SB2"
/>
</head>
</datamap>
<datablock id="DB1" type="boot">
<dataitem id="SB1" encoding="HEX" crc="1234">456789ABCDEF...</dataitem>
<dataitem id="SB2" encoding="HEX" crc="2341">1234567890A...</dataitem>
<dataitem id="SB3" encoding="HEX" crc="3412">890ABCDEF01245...</dataitem>
<dataitem id="SB4" encoding="HEX" crc="4123">456789ABCDEF...</dataitem>
</datablock>
<datablock id="DB2" type="OS">
<dataitem id="SB1" encoding="HEX" crc="1234">456789ABCDEF...</dataitem>
<dataitem id="SB2" encoding="HEX" crc="2341">1234567890A...</dataitem>
<dataitem id="SB3" encoding="HEX" crc="3412">890ABCDEF01245...</dataitem>
<dataitem id="SB4" encoding="HEX" crc="4123">456789ABCDEF...</dataitem>
</datablock>
<datablock id="DB3" type="file" name="DUMP.ASM">
<dataitem id="SB1" encoding="HEX" crc="1234">456789ABCDEF...</dataitem>
<dataitem id="SB2" encoding="HEX" crc="2341">1234567890A...</dataitem>
<dataitem id="SB3" encoding="HEX" crc="3412">890ABCDEF01245...</dataitem>
<dataitem id="SB4" encoding="HEX" crc="4123">456789ABCDEF...</dataitem>
</datablock>
</archive image file>
please don't get hung up on names, etc, The basic structure is what I have been
talking about. A utility can go into the archive file, find a <datablock>,
know what it is and extract it without having to know anything about the OS. Another
utility can read the media, datamap, and datablocks to re-create tracks in memory
to then write out to disk. There was earlier talk about years from now being able
to still re-create disks. I think that is a fine ambition, but that still requires
all the hardware and intimate OS knowledge to still be around. My idea was to at
least be able to extract the data without having to have ANY knowledge of the OS.
best regards, Steve Thatcher
I seem to recall that my slow old N* Horizon was doing dd at 4mhz with no dma - in fact it was polled I/O because their wait for I/O available kept locking up so I modified it.
best regards, Steve Thatcher
-----Original Message-----
From: Fred Cisin <cisin(a)xenosoft.com>
Sent: Aug 12, 2004 5:12 PM
To: "General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
Subject: Re: Let's develop an open-source media archive standard
On Thu, 12 Aug 2004, Dwight K. Elvey wrote:
> Hi Jules
> Here is what I've found. It is a disk drive emulator.
> Unless a PC is DMA driven, bit banging a floppy is not practical.
Disk I/O without DMA is not practical. But it IS possible.
Consider the PCJr and Tandy 1000, both of which do disk I/O
without DMA on 4.77MHz? machines.
my crude example was a form of picture because words just didn't seem to be conveying what I had in mind... :)
-----Original Message-----
From: Vintage Computer Festival <vcf(a)siconic.com>
Sent: Aug 12, 2004 5:33 PM
To: "General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
Subject: Re: archive file format exmaple
On Thu, 12 Aug 2004, Jules Richardson wrote:
> On Thu, 2004-08-12 at 11:11, Steve Thatcher wrote:
> > here is a "crude" example of what I was talking about.
> > [snip]
>
> That looks good. How's about moving things like author into the
> definitions section? I'm not sure that it belongs at the top level, but
> more in the section containing archive info (description, creation date
> etc.)
I think it's rather premature to be discussing implementation :(
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
Hi Jules
Here is what I've found. It is a disk drive emulator.
Unless a PC is DMA driven, bit banging a floppy is not practical.
There are just too many other things that the PC is doing on
the side. Having a separate dedicated uP is the best way.
That device can then either communicate with the host through serial
or parallel.
Dwight
Web pointer:
http://www.rothfus.com/SVD/
>From: "Jules Richardson" <julesrichardsonuk(a)yahoo.co.uk>
>
>On Thu, 2004-08-12 at 00:13, Dwight K. Elvey wrote:
>> >From: "Fred Cisin" <cisin(a)xenosoft.com>
>> >
>> >truly demented idea:
>> >build a hardware device that can read the image file, connects
>> >via 34 or 50 pin cable to an FDC, and that produces pulses
>> >that look like disk data to the FDC.
>> >
>> Hi
>> It has already been done. I can look up the web page if you like.
>> Dwight
>
>yes, please :-)
>
>I remember asking about this a while back, specifically wondering if a
>PC parallel port was fast enough to drive it (not without buffering at
>some sort of level, it seemed)
>
>I wouldn't mind seeing what someone else has come up with.
>
>cheers
>
>Jules
>
>
I did not say it was the primary purpose. The data blocks work hand in hand with the formatting information.
It is fine to make a standard extensible, but what good does it do if the (I hate the use the word) file can't be gotten without jumping through major hoops because how the data was stored in the image file wasn't extended out to make blocks. If you have to iterate through formatting information to get data then you have to be intimately familiar with the disk format in use. It means any GENERAL utility to read an image file for data access will have to KNOW about all machione supported rather than just getting some type of data identifier and reading the data out.
I think part of the problem here is that the word file is being taken literally to mean filename, size, data, etc. I am using in the context of a block of data. It has NOTHING to do with how the data is on the disk, where it is stored, the recording format, etc. It is just a piece of data...
Sellam in a later email thought that I was proposing that the data be duplicated twice - one for file access and one for format data - NO, that is not what I was talking about. The formatting information contains no data other that what may be necessary for things outside of what is considered a sector on a drive such as address mark, etc(note: this does not preclude sequential data - I am only using this as a specific example in this email).
I am talking about two separate sections inside the same image file. One of the sections contains data blocks. The other has specific formatting information and POINTS to the data in the data blocks. A utility program could then SIMPLY get any type of data out of the image file without the utility being out of date as soon as someone added a new physical format.
I have tried to make this email as clear as possible, so there is no misunderstanding of what I have proposed.
best regards, Steve Thatcher
-----Original Message-----
From: "Dwight K. Elvey" <dwight.elvey(a)amd.com>
Sent: Aug 11, 2004 8:11 PM
To: cctalk(a)classiccmp.org
Subject: RE: Let's develop an open-source media archive standard
>From: "Steve Thatcher" <melamy(a)earthlink.net>
>
>what is wrong with making things easier?
---snip---
Hi
I'm not saying to make it impossible to do, just that it shouldn't
be considered as a primary purpose. Using an extendable language
like I've suggested, one can add such features. Part of the problem
is that when someone creates the archive, they may not even know
the file structure of the disk. I would expect the specification
to be broad enough to allow such. Still, the primary thing is
to be able to recreate the original material.
To me this means that any input to some emulator may mean that
it requires some post processing. How would one know what some
some format a particular emulator wanted? How would a person always
know how to read the directory structure and be able to extract
files? If one wanted to work in all cases, I'd expect that the
person writing the emulator would provide the needed post processing
to extract such information. Otherwise, they'd only be able to
read archives of disk that were specifically created for their
system. Those archives that the person didn't know the file
structure would be useless unless.
Such things are secondary functions. They shouldn't be restricted
>from being used, it is just that the primary function should
be to capture the entire information in as close to the original
format as possible. Creating post processors could easily be
done as a separate outside function for special purposes.
Dwight
>>XML is platform neutral because it's basically ASCII, right?
Yes, true, but I think of XML more as a Web technology requiring a complex
parsing engine. I'm not a Web programmer so my thoughts on XML are probably
somewhat broken.
Another comment was made about the difference between what's actually on the
media versus what the CPU actually sees. We would thus need to capture the
raw data stream from the "heads" side of the controller in order to
regenerate a usable original media from the metafile.
For emulator use, we can grind this metafile through a translation program
to get the bytestream. OR, the metafile could contain both types of data
(using the container file and metadirectory idea from earlier).
What we really need is PDF for magnetic media :-)
-----Original Message-----
From: cctalk-bounces(a)classiccmp.org
[mailto:cctalk-bounces@classiccmp.org]On Behalf Of Vintage Computer
Festival
Sent: Wednesday, August 11, 2004 3:23 PM
To: General Discussion: On-Topic and Off-Topic Posts
Subject: RE: Let's develop an open-source media archive standard
On Wed, 11 Aug 2004, Cini, Richard wrote:
> This example represents the block data using metatags...I guess along the
> "XML" part of the thread.
>
> I was thinking similarly to you but not using XML metadata:
>
> ;Hardware descriptor
> MFGR
> MACHINE
> SUBTYPE
> DRIVETYPE (this of course defines what follows)
> ;for floppy
> DRIVESIZE
> ENCODING
> TRACKS
> SECTORS
> SECTSIZE
> ;HexData
> ; Each record or group of records contains the related media data. The
> address record would be used for encoding the metadata
> 00TTSSHH: (00-track-sector-head)
>
> I looked to Intel Hex (or Motorola) because it had built-in CRC facilities
> and it was human-readable ASCII. The drive and machine description could
be
> encoded in special MOT records probably.
I like the XML style because it's more explicit; more human-readable.
> XML is more a more "current" technology but I was trying to keep with the
> platform neutrality by sticking to text-only and not assuming the use of
any
> other technology like XML.
XML is platform neutral because it's basically ASCII, right?
--
Sellam Ismail Vintage Computer
Festival
----------------------------------------------------------------------------
--
International Man of Intrigue and Danger
http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers
]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org
]
as Sellam had suggested, the data size we represent the information in is not that important. I would encode binary data as hex to keep everything ascii. Data size would expand, but the data would also be compressable so things could be kept in ZIP files of whatever choice a person would want to for their archiving purposes.
XML has become a storage format choice for a lot of different commercial packages. My knowledge is more based on the windows world, but I would doubt that other computer software houses are avoiding XML. Sun/Java certainly embraces it.
I don't quite understand why representing binary as hex would affect the ability to have command line utilitities. Certainly more cpu cycles are needed for conversion and image file size is larger, but we need a readable format and I would think that cpu cycles is not as much of a concern or file size. A utility to create a disk would only have to run through the conversion once to buffer a representation of the floppy disk (unless we are talking about a hard drive image of course). The file size to re-create a floppy disk is only going to be 2 to 3meg at the most (if thinking about a 1.2meg floppy with fortmatting info).
The only difference I see in the sections that were described is that the first one encompasses the format info and the data. My description had the first one as being a big block that contained the two other sections as well as length and CRC info to verify data consistency. Adding author, etc to the big block would make perfect sense.
As for GCR, that would have been covered under etc... I am not familiar with GCR, but I would guess that it has to deal at least with physical tracks and heads. In this case, a track would consist of whatever the format needed plus the data blocks required for the track.
best regards, Steve Thatcher
-----Original Message-----
From: Jules Richardson <julesrichardsonuk(a)yahoo.co.uk>
Sent: Aug 11, 2004 8:08 AM
To: "General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
Subject: Re: Let's develop an open-source media archive standard
On Wed, 2004-08-11 at 10:50, Steve Thatcher wrote:
> Hi all, after reading all this morning's posts, I thought I would throw out some thoughts.
>
> XML as a readable format is a great idea.
I haven't done any serious playing with XML in the last couple of years,
but back when I did, my experience was that XML is not a good format for
mixing human-readable and binary data within the XML structure itself.
To make matters worse, the XML spec (at least at the time) did not
define whether it was possible to pass several XML documents down the
same data stream (or, as we'd likely need for this, XML documents mixed
with raw binary). Typically, parsers of the day expected to take control
of the data stream and expected it to contain one XML document only -
often closing the stream themselves afterwards.
I did end up writing my own parser in a couple of KB of code which was a
little more flexible in data stream handling (so XML's certainly not a
heavyweight format, and could likely be handled on pretty much any
machine), but it would be nice to make use of off-the-shelf parsers for
platforms that have them where possible.
As you've also said, my initial thought for a data format was to keep
human-readable config seperate from binary data. The human-readable
config would contain a table of lengths/offsets for the binary data
giving the actual definition. This does have the advantage that if the
binary data happens to be a linear sequence of blocks (sectors in the
case of a disk image) then the raw image can easily be extracted if
needs be (say, to allow conversion to a different format)
Personally, I'm not a fan of mixing binary data in with the
human-readable parts because then there are issues of character escaping
as well as the structure detracting from the readability. And if encoded
binary data is used instead (say, hexadecimal representation) then
there's still an issue of readability, plus the archive ends up bloated
and extra CPU cycles are needed to decode data. Neither of those two
approaches lend themselves to simply being able to use common
command-line utilities to extract the data, either. I'm prefectly
willing to be convinced, though :)
> I looked at the CAPS format and in part that would be okay. I would like
> to throw in an idea of whatever we create as a standard actually have
> three sections to it.
So, first section is all the 'fuzzy' data (author, date, version info,
description etc.), second section describes the layout of the binary
data (offsets, surfaces, etc.), and the third section is the raw binary
data itself? If so, I'm certainly happy with that :-)
One aside - what's the natural way of defining data on a GCR floppy? Do
heads/sectors/tracks still make sense as an addressing mode, but it's
just that the number of sectors per track varies according to the track
number? Or isn't it that simple?
cheers
Jules
I'm looking for ROMs (or ROM images would be better) for a TQK50
controller. The ROMs I want are 23-330E5 and 23-331E5. Can anyone
help?
--
Pete Peter Turnbull
Network Manager
University of York
would you rather have one format that allows for easy data access and re-creation of media or only be able to re-create media and have a difficult interface in EVERY emulator that needs to access the data? From a usage standpoint, I think putting the burden on the emulators is not a reasonable approach. There are a variety of emulators available do not have to deal with track sector access of information. One uses an import command to bring ms-dos files in for example. To have to write an outside utility that has to know what the image file type is, know the physical layout of the disk, know how the OS accesses files on the disk just to retrieve file data is not a good idea. leave the OS access to the real platform or emulators that want to deal with actual tracks and sectors.
best regards, Steve Thatcher
-----Original Message-----
From: "Dwight K. Elvey" <dwight.elvey(a)amd.com>
Sent: Aug 11, 2004 3:05 PM
To: cctalk(a)classiccmp.org
Subject: Re: Let's develop an open-source media archive standard
>From: spc(a)conman.org
>
>It was thus said that the Great Vintage Computer Festival once stated:
>>
>> HOWEVER, this makes it very difficult to use the imagefile on an emulator.
>> To use the floppy disk example again, if the emulator wants Track 14
>> Sector 8 (or Block 417) but it has not been explicitly laid out in the
>> imagefile because it was originally zeroes, then the emulator, if poorly
>> designed, may crap out.
>
> Are you trying to create an archive format, or a format that is to be used
>by emulators? I say skip the emulators and concentrate on archival
>purposes. An emulator can then use the archive format to create a disk
>image in whatever internal format it requires.
>
> Don't complicate the problem.
I agree. We only need to provide a format that could be converted
for a specific emulator, not necessarily one that can be conveniently
read by an emulator.
Dwight
>
> -spc (And don't try to become everything for everybody ... )
>
>
>
>
> -----Original Message-----
> From: cctalk-bounces(a)classiccmp.org
> [mailto:cctalk-bounces@classiccmp.org] On Behalf Of Jules Richardson
> Sent: 02 August 2004 23:10
> To: General Discussion: On-Topic and Off-Topic Posts
> Subject: Re: rarest computers. was: RE: Xerox Alto
> Restoration + Emulation
>
>
> On Mon, 2004-08-02 at 19:56, Vintage Computer Festival wrote:
> > On Mon, 2 Aug 2004, Jules Richardson wrote:
> > > Apple /// (possibly lots worldwide; there don't seem to be many
> > > people this side of the pond who have seen one though)
> >
> > Very common.
>
> I figured they might be - just not in this part of the world, it seems
> :-) Wonder how many were actually sold in the UK...
At least 3 - you've got 1, I've got 2 and I bet Tony's got one
somewhere...
What might be uncommon these days is a *working* /// :)
Cheers
w
Hi
I suspect that most metal cased N*'s were destined to be used
in rack mount applications.
Dwight
>From: "Marvin Johnston" <marvin(a)rain.org>
>
>Actually, it would be nice to have pictures of all three Northstar
>chassis styles. Right now, mine are all buried and I won't get a chance
>to get near them for another month or so. But I'll keep my eyes open for
>one of the beige ones now :).
>
>Adrian Graham wrote:
>>
>> > On Wed, 4 Aug 2004, Marvin Johnston wrote:
>> >
>> > > Someone mentioned some time ago a metal case instead of the
>> > standard
>> > > case for the Northstar Horizon. Can anyone confirm if they
>> > did put out
>> > > a computer in a metal case, and if so, (wishful thinking) how many
>> > > they might have produced? I have one here in a powder blue
>> > metal case
>>
>> There's at least one of the metal cased ones in 'storage' at Bletchley
>> Park - IMSAI blue with an (I assume) aluminium front, 2 vertical floppy
>> drives etc. Got a pic somewhere if anyone's interested, I can't attach
>> it to this mail 'cos the list software will bin it.
>
>From: "melamy(a)earthlink.net" <melamy(a)earthlink.net>
>
---snip---
>
>The image file that you are discussing is only good as a "how to" to write
>arbitrary data to arbitrary media. It is only good for creating a final
>media image because the format being contemplated (mixing data into the
>physical spec) will preclude the file from standing on its own.
No, that is not true. The description of the physical data in the archive
should be sufficient to recover the system side data. I can state
this because the system side data is just a subset of the physical
data.
I recommend that the archive should include enough information to
translate from the physical to the system but for archiving purposes,
that is not absolutely necessary. It is not that hard to describe as
part of the header what FM looks like. Still, if we restrict all archived
information to include this, some unknown formats may not be archived
because the person with the data doesn't know that format. It is
better to capture the data first.
It may also be that the only way that person has to capture the
data is the output of a controller chip. The archiving should allow
this as well ( more in the format the Steve would like it all to
be in ). This file could later be combined with the more physical
information when that was available, just as a archive file that
originally had only physical information could be appended with the
data as seen by the system.
What this means is that some of the archived files may not be directly
useable by Steve and his tools without some more in depth knowledge
about the encoding use. It doesn't make it impossible to use, just
difficult.
Dwight
>I was
>proposing identifying data blocks as files if that is what they were. The
>data blocks would have been sequential and trivial to retrieve. That is
>hardly what I would call defining a file system as part of the image file.
>
>best regards, Steve Thatcher
>
>From: "Roger Merchberger" <zmerch(a)30below.com>
>
>Rumor has it that Vintage Computer Festival may have mentioned these words:
>>On Wed, 11 Aug 2004, Steve Thatcher wrote:
>>
>> > If we create a physical description only and do not abstract the data
>> > then any emulator must understand the OS file structure in order to
>> > retrieve any internal file representation.
>
>I don't wanna sound like a doofus, but:
>
>So?
>
>> My idea would make the file
>> > re-creation simple in that the xml image file would be parsed for the
>> > actual file data that an emulator would need. This makes the emulator
>> > easier.
>>
>>You're describing an imagefile that contains a filesystem image, rather
>>than, say, a disk image. This can be accomodated in the spec, but it's a
>>different type of image than what we've been discussing.
>
>It could be accommodated, but why should it when it would be easier for
>someone to build a converter into an emulator image file?
>
>This spec (as far as I understand it) is going to be read-only anyway -
>this is for archiving after all. What happens when the emulator tries to
>rewrite the disk image (say, from saving a modified file)? What will happen
>to the original archive?
Hi
I don't see this as being an issue. Once one knew how to
decode the information for such applications that wanted to
extract the data, one could import information into the
image. It would of course require some knowledge of the
data structures used by that system. It would have to know
how to resolve things like logical interleaving and allocating
space by the allocation methods.
>
>>It's up to emulator writers to read the spec and adapt to it.
>
>Bam. That hit the thumbnail right on the head. ;-)
>
>> We'll make
>>the spec as flexible as possible, but first and foremost, the spec is
>>intended to archive images of data media, and not to serve as a universal
>>emulator image format (though it should be able to be used as such).
>
>I'd like to add that IMHO it should be "read-only" so the emulator doesn't
>destroy the archive when it rewrites certain sectors and/or files.
I don't see this as an issue. Once one has the archive file in hand,
using it as a method to import external data is surely a valid application.
The concept of read-only is just an OS flag that can be overwritten.
Surely, someone doing this would rename the file or otherwise indicate
that it has change. There is nothing anyone can do to restrict such
actions. Why specify something that can not be enforced? We can recommend
action. We can recommend that there be a field that indicated any change
in the data of an archive and if that change is an overwrite of data
or an enhancement of the data.
Dwight
>
>At least that's my take on the deal...
>Laterz,
>Roger "Merch" Merchberger
>
>--
>Roger "Merch" Merchberger --- sysadmin, Iceberg Computers
>Recycling is good, right??? Randomization is better!!!
>
>If at first you don't succeed, nuclear warhead
>disarmament should *not* be your first career choice.
>
>
the emails I received seem to be out of order...
I believe what you are saying does satisfy the requirement that I have been
proposing. There are different levels of physical to data mapping. I can
see target platform code running that creates an image file and might
actually send the XML directly out a serial port in zmodem for example. It
could send it out as files or strictly embedded data. My point has been to
make the file blocks available if the target has them just so the data can
be extracted easily.
best regards, Steve
Original Message:
-----------------
From: Vintage Computer Festival vcf(a)siconic.com
Date: Thu, 12 Aug 2004 09:39:18 -0700 (PDT)
To: cctalk(a)classiccmp.org
Subject: RE: Let's develop an open-source media archive standard
On Wed, 11 Aug 2004, Steve Thatcher wrote:
> I am not assuming anything about the data. The usual use is to have
> files... in the case of a paper tape emulator system used for CNC, the
> disk structure may not resemble a normal file structure. It still
> contains one or more blocks of data. you can apply whatever name to that
> you want to. The boot sector on a cp/m 8" disk doesn't have a name, but
> it is a block of data that is separate from everything else. Personally,
> I would want to be able to "read" the boot sector and potentially even
> write it back to an image file. It really doesn't make a difference
> whether you access track 0 and sector 1 or a data block inside the image
> file that contains the boot code.
I see what you're saying now. I suppose another potential format that can
be incorporated into the spec is a file-based format, where blocks or
"files" are represented inside the archive. I can see this being useful
for two purposes:
1) Imaging the filesystem from a medium (as opposed to the medium itself)
2) Incorporating code along the lines of what Dwight has been suggesting
that knows how to do special processing of the image it is contained in.
The code can be extracted and poked into memory using a very simple
algorithm that possibly can be described in comments in the image file.
I did suggest that the spec be able to image filesystems in a previous
message. I think this comes close enough to what you are describing. It
can be modified to also be able to indicate "blocks of data". These
blocks could be given an identifier--a name, number, whatever; it doesn't
matter, as long as it's an ASCII-representable string--and then extracted
>from the image file as needed.
Does this satisfy you, Steve? :)
--
Sellam Ismail Vintage Computer
Festival
----------------------------------------------------------------------------
--
International Man of Intrigue and Danger
http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers
]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org
]
--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .
All,
Just to let you know I followed up to Lyle's message, and
went to see those guys (with Lyle) to see what is there.
I am going in there tomorrow morning, and will save the
systems from the oven. Basically, we have:
- MicroVAX 3600 with RA82 (my new baby ;-)
- extra rack filled with 6 pcs RA92
- Cipher magtape unit, frontloader (not in rack)
- two VAX 4000(-500A) systems
- a stack of BA42 storage expanders with various drives
- a stack of MicroVAX 3100 -M38 and -M76 systems
- two what seem to be DECsystem 5500 systems
- one more VAX 3600
.. and some little things. If you have an interest in any
of this (cept the 3600- tis mine!) please contact me off-list
so we can work something out.
Warning: RA92 dont ship well. This means that it will probably
be pickup only, sorry.
--f
Just obtained an M4 Data tape drive, model 9914 (800/1600/3200/6250)... just pulled from working service...desktop enclosure... $50 bucks :)
I believe it's Differential, not single ended. I haven't kept up with PC technology (hooking this to a PC). Is this older Differential the same as LVD? I believe it had a centronics 50 pin connector labeled "differential". I'm wondering what SCSI card I can use with this... Adaptec 2940?
Jay
I am not assuming anything about the data. The usual use is to have files... in the case of a paper tape emulator system used for CNC, the disk structure may not resemble a normal file structure. It still contains one or more blocks of data. you can apply whatever name to that you want to. The boot sector on a cp/m 8" disk doesn't have a name, but it is a block of data that is separate from everything else. Personally, I would want to be able to "read" the boot sector and potentially even write it back to an image file. It really doesn't make a difference whether you access track 0 and sector 1 or a data block inside the image file that contains the boot code.
best regards, Steve Thatcher
-----Original Message-----
From: "Dwight K. Elvey" <dwight.elvey(a)amd.com>
Sent: Aug 11, 2004 1:39 PM
To: melamy(a)earthlink.net, cctalk(a)classiccmp.org
Subject: RE: Let's develop an open-source media archive standard
>From: "Steve Thatcher" <melamy(a)earthlink.net>
>
>I agree with Sellam on the point about using it both for media re-creation and
emulation. The trouble with the approach below of just using raw data on a
track sector basis is that now you have created a file that can only be used
with an emulator that understands the physical format and OS access for the
computer system you are emulating. My earlier point of separating the data and
the format information allows a single file (that would not be much bigger that
the one described below) to contain multiple platform specific files that can be
"read" by a simple utility that does not require any knowledge of the OS or the
platform.
>
>best regards, Steve Thatcher
Hi Steve
You seem to be assuming that the particular disk you are
archiving has a file structure. This is not always the case.
Dwight
comments embedded below...
best regards, Steve Thatcher
-----Original Message-----
From: Hans Franke <Hans.Franke(a)siemens.com>
Sent: Aug 12, 2004 10:06 AM
To: Steve Thatcher <melamy(a)earthlink.net>,
"General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
Subject: RE: Let's develop an open-source media archive standard
Am 11 Aug 2004 14:47 meinte Steve Thatcher:
> I realize that the idea is to create a format to make re-creation
> of media possible for a variety of platforms. We can certainly do
> that and have its only function be to maintain a physical data
> format. My added idea is that the data and the formatting be
> separated so that a simple utility on a non-target platform could
> extract data from the image file.
Well, physical format and data is not always seperable. In some
circumstances the physical format is part of the onformation an
application needs, and differs form media to media (e.g. copy
protection schemes)
*** the separated data that I am talking about is what would be accessed through normal OS channels. Copy
protection schemes, special destroyed sectors, etc are not accessible through the OS.
> If we create a physical description only and do not abstract
> the data then any emulator must understand the OS file
> structure in order to retrieve any internal file representation.
> My idea would make the file re-creation simple in that the xml
> image file would be parsed for the actual file data that an
> emulator would need. This makes the emulator easier.
But what if the emulator needs the physical format information?
*** I have not proposed that the physical format information be excluded...
> To retrieve a file from the physical layout that it at the end
> of this message, the emulator must know the actual disk format
> that is used on the target system (the one the image file was
> made for). I have seen cp/m systems where the actual physical
> sectors were sequential on disk and the OS file sector was
> actually virtual to increase speed. Not my idea of the way to
> do it. It is much easier to make the physical sectors slewed
> so that a physical sector is a file sector. These are the types
> of issues you will have to overcome if an emulator must totally
> understand each and every file system for a cp/m version for
> example.
At least within a CP/M system it usualy doesn't matter at all
how the files are stored on a disk. Except for some odd apps
who tried to implement system specific copy protection schemes,
all and every CP/M app accesses files just via BDOS which already
hides the real disk strukture.
*** talk to other people here and one of their arguements for keeping things all together was being able to
access track and sector directly. As for BDOS, that is fine, but keep in mind that BDOS was customized for nearly every cp/m platform
that would be fine. Your <archivetype> is what <media> was all about though i.e. rom definition etc. The archive really doesn't have a type, it does however contain media information and then the data to be accessed or put onto the defined media.
best regards, Steve Thatcher
-----Original Message-----
From: Jason McBrien <jbmcb(a)hotmail.com>
Sent: Aug 12, 2004 10:04 AM
To: Steve Thatcher <melamy(a)earthlink.net>,
"General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
Subject: Re: archive file format exmaple
How about breaking up the sections a bit, like HTML <head><body> sections?
Putting card-catalog information up top in it's own section would speed
indexing and searches. Like this:
<archive image file version="1.0">
<catalog>
<definitions>
a place to put whatever relevant information is needed about the format or
achive
or your favorite poem or joke
</defintions>
<archiver>Pete Smith</archiver>
<method>ROM Dumper V4.12</method>
<recdate>02004-02-02</recdate>
<title>Omega Race</title>
<author>Midway</author>
<platform>
<mfg>Commodore</mfg>
<system>VIC-20</system>
</platform>
<archivetype>
<fmtcategory>ROM</fmtcategory>
<fmttype>Game Cartridge</fmttype>
</archivetype>
</catalog>
<format>
<default-sector-data value="$AA"/>
....
</format>
<data>
....etc
I realize that the idea is to create a format to make re-creation of media possible for a variety of platforms. We can certainly do that and have its only function be to maintain a physical data format. My added idea is that the data and the formatting be separated so that a simple utility on a non-target platform could extract data from the image file.
If we create a physical description only and do not abstract the data then any emulator must understand the OS file structure in order to retrieve any internal file representation. My idea would make the file re-creation simple in that the xml image file would be parsed for the actual file data that an emulator would need. This makes the emulator easier.
To retrieve a file from the physical layout that it at the end of this message, the emulator must know the actual disk format that is used on the target system (the one the image file was made for). I have seen cp/m systems where the actual physical sectors were sequential on disk and the OS file sector was actually virtual to increase speed. Not my idea of the way to do it. It is much easier to make the physical sectors slewed so that a physical sector is a file sector. These are the types of issues you will have to overcome if an emulator must totally understand each and every file system for a cp/m version for example.
Sellam, let me know if you would like to discuss this via telephone so I can convery the idea that I am proposing.
best regards, Steve Thatcher
-----Original Message-----
From: Vintage Computer Festival <vcf(a)siconic.com>
Sent: Aug 11, 2004 3:00 PM
To: Steve Thatcher <melamy(a)earthlink.net>,
"General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
Subject: RE: Let's develop an open-source media archive standard
On Wed, 11 Aug 2004, Steve Thatcher wrote:
> I agree with Sellam on the point about using it both for media
> re-creation and emulation. The trouble with the approach below of just
> using raw data on a track sector basis is that now you have created a
> file that can only be used with an emulator that understands the
> physical format and OS access for the computer system you are emulating.
That is the point, really. What we are attempting to do is describe as
faithfully as possible a physical media with logical data in a purely
logical form. The goal would be that the physical media could be
re-created from the imagefile if need be. The parameters of the physcial
media are specified so that this can be possible.
> My earlier point of separating the data and the format information
> allows a single file (that would not be much bigger that the one
> described below) to contain multiple platform specific files that can be
> "read" by a simple utility that does not require any knowledge of the OS
> or the platform.
I'm not quite understanding you here. Or maybe I am. An image in the
format shown below could be read by any emulator. Making sense of the
data with respect to that emulator is a different issue altogether, but it
does make it possible for, say, a Northstar Horizon emulator to load up an
Apple ][ disk image and then try to access it.
Anyway, I don't think I am quite getting the point you are trying to make.
> <MEDIA TYPE=FLOPPY SIZE=5.25 SIDES=1 DENSITY=SINGLE FORMAT=GCR TRACKS=35
> SECTORS=16 SECTORSIZE=256>
>
> <VOLUME>Apple ][ System Disk</VOLUME>
>
> </MEDIA>
>
>
> <DATA>
> <TRACK 0><SECTOR 0>
>
> HERE WOULD BE THE ASCII HEX DATA FOR TRACK 0, SECTOR 0
>
> </SECTOR></TRACK>
>
> ...
>
> <TRACK 34><SECTOR 15>
>
> HERE WOULD BE THE ASCII HEX DATA FOR TRACK 34, SECTOR 15
>
> </SECTOR></TRACK>
> </DATA>
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
The following is a XML definition I did some 3+ years ago
during a discusion, here on classiccomp (back when there
was one list *G*), as an example what a XML storage good
for everything from punch card to CD could be.
This example shows two CCDD structures, one showing an
IBMish tape, the other a disk for a popular 8 Bit micro.
(Back then nobody came up with the systems name :).
Someone asked recently how to handle multiple XML within
one file ... well, that's exactly the way it works :)
Gruss
H.
CCDD stands for Classic Computer Device Data
-------------------------
<?xml version="1.0" standalone='yes' ?>
<!DOCTYPE CCDD [
<!ELEMENT CCDD (VORSPANN?, META?, (CHANNEL* | DEVICE* | MEDIA))>
<!ELEMENT VORSPANN (#PCDATA)>
<!ELEMENT META (#PCDATA | SYSTEM | OS)*>
<!ELEMENT SYSTEM (#PCDATA)>
<!ELEMENT OS (#PCDATA)>
<!ELEMENT CHANNEL (META?, DEVICE*)>
<!ELEMENT DEVICE (META?, MEDIA*)>
<!ELEMENT MEDIA (META?, (RAW | HEAD*) )>
<!ELEMENT HEAD (RAW | TRACK*)>
<!ELEMENT TRACK (RAW | BLOCK*)>
<!ELEMENT BLOCK (RAW | DATA*)>
<!ELEMENT DATA (#PCDATA)>
<!ELEMENT RAW (#PCDATA)>
<!ATTLIST CHANNEL
ID ID #IMPLIED>
<!ATTLIST DEVICE
ID ID #IMPLIED>
<!ATTLIST MEDIA
ID ID #IMPLIED
LFD CDATA #IMPLIED
SIZE CDATA #IMPLIED
FILLER CDATA #IMPLIED
FORMAT CDATA #IMPLIED>
<!ATTLIST HEAD
LFD CDATA #IMPLIED
SIZE CDATA #IMPLIED
FILLER CDATA #IMPLIED
FORMAT CDATA #IMPLIED>
<!ATTLIST TRACK
LFD CDATA #IMPLIED
SIZE CDATA #IMPLIED
FILLER CDATA #IMPLIED
FORMAT CDATA #IMPLIED>
<!ATTLIST BLOCK
LFD CDATA #IMPLIED
SIZE CDATA #IMPLIED
FILLER CDATA #IMPLIED
TYPE (DATA|HEADER|UNDEF) "DATA">
<!ATTLIST DATA
SIZE CDATA #IMPLIED
FILLER CDATA #IMPLIED
ENCODING (CHAR|BIN|SED|INTEL|MOT) "SED">
<!ATTLIST RAW
SIZE CDATA #IMPLIED
FILLER CDATA #IMPLIED
CONTENT (DATA|PYSICAL) "DATA"
ENCODING (CHAR|BIN|SED|INTEL|MOT) "SED">
]>
<CCDD>
<META>
Example for a tape mounted on Drive D0 on Channel 1.
</META>
<CHANNEL ID="C_1">
<META>
Standard type 1 channel
</META>
<DEVICE ID="D_D0">
<META>
T9G (6250bpi)
</META>
<MEDIA LFD="0" FORMAT="T6250">
<META>
First Tape in Device
</META>
<HEAD LFD="0" SIZE="36" FILLER="00">
<TRACK LFD="16" SIZE="16" FILLER="00">
<BLOCK TYPE="HEADER">
<DATA SIZE="80" ENCODING="CHAR" FILLER=" ">VOL1TAPE001 BS2000 TSOS 4</DATA>
</BLOCK>
<BLOCK TYPE="HEADER">
<DATA SIZE="80" ENCODING="CHAR" FILLER=" ">UVL1PRIVATE LABEL</DATA>
</BLOCK>
<BLOCK TYPE="HEADER">
<DATA SIZE="80" ENCODING="CHAR" FILLER=" ">HDR1FILE1 00010001000100000102000102 000000BS2000</DATA>
</BLOCK>
<BLOCK TYPE="HEADER">
<DATA SIZE="80" ENCODING="CHAR" FILLER=" ">HDR2U020480204841 00</DATA>
</BLOCK>
<BLOCK TYPE="HEADER">
<DATA SIZE="80" ENCODING="CHAR" FILLER=" ">HDR3TSOS COMPLETE.FILE.NAME.OF.FILE1 0</DATA>
</BLOCK>
<BLOCK TYPE="DATA">
<DATA SIZE="2048" ENCODING="CHAR" FILLER="�">NO REAL DATA INSIDE THIS BLOCK</DATA>
</BLOCK>
<BLOCK TYPE="HEADER">
<DATA SIZE="80" ENCODING="CHAR" FILLER=" ">EOF1FILE1 00010001000100000102000102 000001BS2000</DATA>
</BLOCK>
<BLOCK TYPE="HEADER">
<DATA SIZE="80" ENCODING="CHAR" FILLER=" ">EOF2U020480204841 00</DATA>
</BLOCK>
<BLOCK TYPE="HEADER">
<DATA SIZE="80" ENCODING="CHAR" FILLER=" ">EOF33TSOS COMPLETE.FILE.NAME.OF.FILE1 0</DATA>
</BLOCK>
</TRACK>
</HEAD>
</MEDIA>
</DEVICE>
</CHANNEL>
</CCDD>
<CCDD>
<META>
This is another CCDD File for a FD of
<SYSTEM>XXXXX</SYSTEM> running under <OS>yyyy</OS>.
</META>
<MEDIA LFD="0" SIZE="2" FORMAT="GCR">
<META>
Floppy disk for xxxxxx
</META>
<HEAD LFD="0" SIZE="36" FILLER="00">
<TRACK LFD="16" SIZE="16" FILLER="00">
<BLOCK LFD="14">
<DATA SIZE="256" ENCODING="SED" FILLER="00">
</DATA>
</BLOCK>
<BLOCK LFD="15">
<DATA SIZE="256" ENCODING="SED" FILLER="00">
000000000000000000000000100E
</DATA>
</BLOCK>
</TRACK>
<TRACK LFD="17" SIZE="16" FILLER="00">
<BLOCK LFD="0">
<DATA SIZE="256" ENCODING="SED" FILLER="00">
04110F030000FE000000000000000000
00000000000000000000000000000000
000000000000007A0000000000000000
23010000231001000000000000000000
00000000000000000000000000000000
00000000000000000000000000000000
00000000000000000000000000000000
000000000000000380010000
</DATA>
</BLOCK>
<BLOCK LFD="15">
<DATA SIZE="256" ENCODING="SED" FILLER="00">
0000000000000000000000100F02C8C5
CCCCCFA0A0A0A0A0A0A0A0A0A0A0A0A0
A0A0A0A0A0A0A0A0A0A0A00001
</DATA>
</BLOCK>
</TRACK>
</HEAD>
</MEDIA>
</CCDD>
--
VCF Europa 6.0 am 30.April und 01.Mai 2005 in Muenchen
http://www.vcfe.org/
the keyword arrangement was just a waking thought this morning and is not cast in codecrete.
Required fields is definitely a good thing as long as the info is really necessary.
The only data that was in the media section was data that was specific to to media such as fill bytes, address marks, etc.
Also, I kept a sub block arrangement because a single datablock was supposed to represent a complete data entity such as a file, boot routine, OS itself for example. In order to retrieve any block of data, one only had to find a "datablock" and then concatanate the dataitems.
best regards, Steve Thatcher
-----Original Message-----
From: Jules Richardson <julesrichardsonuk(a)yahoo.co.uk>
Sent: Aug 12, 2004 7:50 AM
To: cctalk(a)classiccmp.org
Subject: Re: archive file format exmaple
On Thu, 2004-08-12 at 11:11, Steve Thatcher wrote:
> here is a "crude" example of what I was talking about.
> [snip]
That looks good. How's about moving things like author into the
definitions section? I'm not sure that it belongs at the top level, but
more in the section containing archive info (description, creation date
etc.)
I'd suggest making certain fields mandatory in what you've called the
definitions section. People tend to be lazy, which means the temptation
is there to create an archive and not bother to include much of the
information, thinking that they'll know what it is a few years down the
line. We've all be caught out by that one! (author, date, description
are good candidates for mandatory fields; there'll be others too)
I'm not so sure about having the actual data within the archive element;
I'd rather have it afterwards. Data in your datamap section still points
to chunks of data, using whatever scheme is understood by the
storage/compression system used by the archive. But then the possibility
is there for using an encoding method that doesn't mix with XML data if
needs be. It also makes it easy to scan the archive section by eye as
it's not mixed up with huge chunks of encoded media data. Actually, you
could probably build collections of media archives (say for some common
platform) all within the same file that way, without breaking the format
- the data in the datamap section of the first archive section just
points to other archives within the same file. That's kinda neat.
eg. (simplifying a little for example purposes):
<archive image file version="1.0">
<definitions>
... as before ...
</defintions>
<media>
... as before ...
</media>
<datamap>
<head physical="0">
<track physical="0">
<sector logical="1" physical="4" id="DB1" />
<sector logical="2" physical="8" id="DB2" />
...
</track>
</head>
</datamap>
</archive>
<datablock id="DB1" type="boot">
....
</datablock>
<datablock id="DB2" type="boot">
....
</datablock>
.. that way anything following the archive section doesn't have to be
XML data even, providing the definition within the datamap section for
the encoding/compression scheme used by the archive can reach it. It
might be zipped data, other archive definitions, whatever.
cheers,
Jules
I haven't read the entire thread on this but I did read Steve Thatcher's
idea and it describes about where I was coming out on this myself.
I might have missed what the ultimate use of this archive would be. Will the
archive be used to (1) re-generate original media; (2) operate with
emualtors; (3) both?
To ensure integrity of the data I would propose recording the data in the
Intel Hex format -- it's text-based and has built-in CRC. Now, we'd have to
modify the standard format a bit to accommodate a larger address space and
to add some sort of standardized header (a "Hardware Descriptor"). This data
would be used by the de-archiver to interpret the stream of data read from
the data area (the "Hex Block").
I agree that a multi-layer approach offers the best combination of platform
neutrality and portability. I don't really know if we need two or three
layers as Steve described to describe the file in a standard fashion. Using
an Intel Hex-like format would increase the "de-archiving" time, but in my
view it's a fair trade-off. De-archiving software could translate the
platform-neutral file into another format better suited for use in
emulators.
I think that we should start compiling a list of the various media we want
represented and how that media is organized natively. I don't mean "well, it
has blocks and sectors" either. We should examine the exact format down to
the actual numbers (i.e., "2048 blocks of 256-bytes recorded twice"). Seeing
how the various data stores are organized should bring some clarity to how
we should represent it.
Just my $0.02.
Rich
Second try
------- Weitergeleitete Nachricht / Forwarded message -------
Von: Hans Franke <Hans.Franke(a)mch20.sbs.de>
An: "General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
Betreff: Re: Let's develop an open-source media archive standard
Datum: Wed, 11 Aug 2004 15:16:36 +0200
Am 11 Aug 2004 12:08 meinte Jules Richardson:
> On Wed, 2004-08-11 at 10:50, Steve Thatcher wrote:
> > Hi all, after reading all this morning's posts, I thought I would throw out some thoughts.
> > XML as a readable format is a great idea.
> I haven't done any serious playing with XML in the last couple of years,
> but back when I did, my experience was that XML is not a good format for
> mixing human-readable and binary data within the XML structure itself.
Only if you intend to keep it 100% human readable.
> To make matters worse, the XML spec (at least at the time) did not
> define whether it was possible to pass several XML documents down the
> same data stream (or, as we'd likely need for this, XML documents mixed
> with raw binary). Typically, parsers of the day expected to take control
> of the data stream and expected it to contain one XML document only -
> often closing the stream themselves afterwards.
Now, that's a feature of the reading application. XML does not
stat what happens next since this is outside the scope. It is
perfectly op to look for the next Document, or the next start
tag of the same document type, or for whatever.
> I did end up writing my own parser in a couple of KB of code which was a
> little more flexible in data stream handling (so XML's certainly not a
> heavyweight format, and could likely be handled on pretty much any
> machine), but it would be nice to make use of off-the-shelf parsers for
> platforms that have them where possible.
Right, but especialy when we're coming down to classic platforms,
such building blocks are not always usable, and in general way
oversized. On a 48k Apple (or a 64 K 4 MHz CP/M machine) we don't
have the space to just port a C-app that has 'only' 100k of code
size. So reader/writer applications for the original environment
have to be small and special to type.
> As you've also said, my initial thought for a data format was to keep
> human-readable config seperate from binary data. The human-readable
> config would contain a table of lengths/offsets for the binary data
> giving the actual definition. This does have the advantage that if the
> binary data happens to be a linear sequence of blocks (sectors in the
> case of a disk image) then the raw image can easily be extracted if
> needs be (say, to allow conversion to a different format)
Well, that is only true if you define binary data as 8 Bit and
all means of transport as 100% transparent. Just, this hasn't
worked that way in the past, and I doubt that we will be safe
>from changes in the future.
As for the character size: we had in the past everything from
6 to 12 Bit (ok, I can't remember 11 Bit characters/words) as
'binary' characters. Of course 6,7 and 8 Bit Bytes can be easy
stored in a 8 Bit Byte, but what about 9 Bit (Bull) or 12 (DEC)?
At that point you already have to incooperate speciual trans-
formation rules which are not necersary transparent.
Also for the requirement of a transparent transport: When
transfering files between different architectures we usualy
have code or even format conversions. Most notable code
conversion would be, for example, ISO 8859-1 <-> EBCDIC which
totally destroys the 'binary' part. Or take format conversions
as done on the way between Unix style files and (Win-)DOS, LF
vs CR/LF. Whenever you leave the A-Z and 0-9 range we are
likely to encounter such problems.
Shure, one could code an app capable to read ASCII/Binary on
a EBCDIC Machine and vice versa, but to my experience (doing
programming since 25 years in mixed environments) it's not
only a boring job, but also one of the most sensitive to
errors.
Any kind of standard format must be true machine independent.
Thus (at least when using the recommended representation) be
able to be transferred across all platforms thinkable of.
> > I looked at the CAPS format and in part that would be okay. I would like
> > to throw in an idea of whatever we create as a standard actually have
> > three sections to it.
> So, first section is all the 'fuzzy' data (author, date, version info,
> description etc.), second section describes the layout of the binary
> data (offsets, surfaces, etc.), and the third section is the raw binary
> data itself? If so, I'm certainly happy with that :-)
I would rather go for an anoted format, where more detailed
information can be added at any point, and not necersary
in certain sections. Especialy since the 'fuzzy' data is
usualy not needed for the job itself.
> One aside - what's the natural way of defining data on a GCR floppy? Do
> heads/sectors/tracks still make sense as an addressing mode, but it's
> just that the number of sectors per track varies according to the track
> number? Or isn't it that simple?
Well, that's already outside of what a standard definition
can define without doubt.
To my understanding interpretation of Data is always part
of a real application. As soon as it touches machine or
format specific implementation details a standard may only
give guidelines how to store them properly, but not how to
interprete. That's part of an actual reader implementation.
And each rader will of course only understand parts he's
made for - e.g. a Apple DOS 3.3 reader will have no idea
what a tape label for a IBM tape is not to mention be able
to differentiate between the various header types.
Reader/Writer apps will always be as specific as they are
right now, when handling a proprietary format. The big
advantage is that intermediate tools, like archiving,
indexing, etc.pp can be shared. Well, in fact it's the
only advantage, except the fact that one doesn't have to
figure out a new format each time, and the simple format
does allow the ad hoc inclusion of new machines/systems.
Gruss
H.
--- Ende der weitergeleiteten Nachricht / End of forwarded message ---
--
VCF Europa 6.0 am 30.April und 01.Mai 2005 in Muenchen
http://www.vcfe.org/
>From: "Fred Cisin" <cisin(a)xenosoft.com>
>
>truly demented idea:
>build a hardware device that can read the image file, connects
>via 34 or 50 pin cable to an FDC, and that produces pulses
>that look like disk data to the FDC.
>
>
>
Hi
It has already been done. I can look up the web
page if you like.
Dwight
Thanks for the very useful information!
>If you want the keyboard to work, you'll need to set up one of the 6520s. If
>you just want to see stuff on the screen, there's a table in the ROMs that is
>loaded into the 6545 CTRC. There are two sets of values, one for 4032 and one
>for 8032 (different ROMs). The other chips are used for IEEE-488, cassette
>control, and the user port. No initialization for your purposes required AFAIK.
I checked the funet archive and found some details, however the PET "IO" document
describes only the 6520's and 6522 - there is no mention of the video controller.
The only other references I found to I/O addresses were obviously a disassembley,
with such meaningful labels as:
AE810 DS 1
AE811 DS 1
AE812 DS 1
Can you tell me where (address) the 6545 is located?
I'm going to begin disassembling the Kernel ROM and see if I can figure out enough
to turn on the screen - don't need keyboard (yet) - just want to be able to display
some info.
>Once you extract the table and load the 6545, you should be able to sling
>bytes at the screen starting at $8000. Remember that "POKE codes" are not
>ASCII nor PETSCII, they are character codes from the chargen ROM (not
>accessible to 6502 memory space). There should be some docs on this on funet.
>As an example, though, while you might "PRINT CHR$(65)" to get an 'A' on the
>screen, you'd "POKE 32768,1" to get that 'A' to appear in the upper left
>corner of the screen.
I didn't see an obvious reference - can you give me a pointer - all I really
need right now is 0-9,A-F and SPACE - I can determine these by fooling with
the working machine if I have to.
Btw, do anyone have (or know of) a working stand-alone monitor program which
can be stuffed into the Kernel ROM position?
Regards,
Dave
--
dave04a (at) Dave Dunfield
dunfield (dot) Firmware development services & tools: www.dunfield.com
com Vintage computing equipment collector.
http://www.parse.com/~ddunfield/museum/index.html
Ok, instead of all the unproductive bickering over semantics (most of
which is illogical...come on, admit it) I'd rather we actually discuss
developing further an open-source, public domain, free (in every sense of
the word) standard for archiving data media of all types (magentic, paper,
or otherwise).
This is basically going to be a continuation of what we've already
discussed on the list, and what Hans and I have discussed in private.
First, let's start with the goals.
The format should be:
1) Well Documented (with such documentation actively preserved in all four
corners of the globe and beyond)
2) Not constrained to any particular hardware
3) Be inclusive of all physical (and logical?) manner of recording media
4) Be implementable on even the simplest architectures (because the
original media source will in many cases have to be read on the hardware
it is connected to)
5) Open source, public domain, etc. (although a copyright may be held if
it makes sense to do so)
6) Adaptable, expandable, revisable (for future extensions)
7) Text-based and storable in commonly accessible character formats (i.e.
a suitable subset of Unicode, i.e. ASCII)
8) Allow for the representation of media in either logical or physical
(raw bit stream) formats
This is a good start. Someone please continue adding to the definition.
I will establish a special session at the next VCF (November 6-7) where
we can commence a committee for formalizing this standard and getting it
recognized internationally in all the various relevant groups (i.e. ANSI
or ISO/IEC).
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
http://www.30below.com/~zmerch/classics/driveformats.html
It's new, it's rough, it's mainly to figure out what kinds of data we need
here -- right now, I can see that we need 2 spreadsheets; one for physical
drive formats, and one for OS layouts. (and yes, it's evil HTML ->
StarOffice 6.0 "saved as .html". I ain't goin' fer pretty yet... ;-) )
Believe me, I realize that it's not anywhere perfect... so I'm asking for
more input for whatever (many things) I've forgotten to include.
If you have info for different disk formats, please feel free to include
them, but please be aware that they may not be integrated until the format
is more solid.
Jay, I dunno if the addresses are munged in the archives... so I'm going to
err on the side of caution here. I have 1 mailing address that as yet gets
no spam (so has no spam-filter), and its:
{my Ham radio callsign} @30below.com.
(If you wanna go the hard way to find it, go here:
http://wireless2.fcc.gov/UlsApp/UlsSearch/searchLicense.jsp
and search for "Merchberger" and look for my active one.)
If not, I'll use the phonetic alphabet, hopefully spammers haven't figured
that one out yet for the spiders:
Alpha Bravo Ait Kilo Kilo
That address gets very little email (unlike my main address which gets
around 500 emails/day) so I'll see things quickly there... and also, the
main email is personal/work/mailer-daemon/root/catchall/whatever and also
has a spamfilter, so I might miss one email there. Rare, but it can (and
unforch, does) happen. This way, guaranteed not to be missed in any way. ;-)
Thanks,
Roger "Merch" Merchberger
--
Roger "Merch" Merchberger | A new truth in advertising slogan
sysadmin, Iceberg Computers | for MicroSoft: "We're not the oxy...
zmerch(a)30below.com | ...in oxymoron!"
The reason the subject is for "Software Users" is that ""Hardware Users"
will probably want to use ONLY standard DEC distributions of RT-11
which sufficiently exercise the real PDP-11 hardware. The goal of this
post is to reach and elicit comments from all of those hobby RT-11
Users who want the bugs fixed and / or want some enhancements.
I suppose that it might even be possible that there are some commercial
RT-11 users who might be interested, but at this point I am not aware
of any and doubt very much that any commercial RT-11 users would be
sufficiently interested. BUT, if you are still there, it might still be possible
to accommodate the commercial side.
The ISO file for a CD with 13 RT-11 distributions is available for download at:
http://www.classiccmp.org/PDP-11/RT-11/dists/
The file is: RT11DV10.ISO.zip
The CD contains 13 RT-11 distributions from V01-15 up to V05.03 under
both the ISO9660 file structure along with 4 RT-11 directories for the
4 RT-11 partitions on the CD. A rather unique aspect of the CD is that
none of the files in the ISO9660 file structure are duplicated. The RT-11
directory entries point to the same location on the CD for the same
file in each case. In addition, if SIMH is able to ATTACH the CD as
as single file, then it will be possible to actually BOOT the CD under
RT-11. Otherwise, just BOOT the container file RT11DV10.ISO, after
it is copied to the hard drive (UnZipped actually), under RT-11 as in:
ATTACH RQ0: RT11DV10.ISO
SET RQ0: LOCK
BOOT RQ0:
It is STRONGLY recommended that NO changes of any kind be
made to the ISO file under RT-11, even by experienced RT-11 users.
The reasons are explained in the 2 README files.
NOTE that all of the images for the RK05 and RL02 media have been
truncated - all of the contiguous blocks with zeros at the end of the
file image have been removed. The primary reason was so that more
of these file images would fit on each RT-11 partition. Specifically,
since all RL02 file images are 20,450 blocks, ONLY 3 would fit on
each RT-11 partition. In addition, even if the full image of 20,450
blocks were present, it would still not be possible to use each file as
a correct file image in all situations under SIMH since the system
area needed for the INITIALIZE command under RT-11 would
NOT be present. IF the RT-11 user wishes to copy the truncated
file image to a "fully functional" RL02 container file to be used under
SIMH, then copy the file ZEROSDL.DSK at the end of the CD as in:
C:\>COPY D:ZEROSDL.DSK C:RT11V5.03
C:\>PDP11
simh>ATTACH RL0: D:RTV5RL.03
simh>SET RL0: LOCK
simh>ATTACH RL1: C:RT11V5.03
simh>BOOT RL0:
COPY/DEVICE DL0: DL1:
NOTE that the file ZEROSDL.DSK was created by using PUTR to:
(C:\)>FORMAT ZEROSDL.DSK/RL02
A big thank you to John Wilson at: http://ww.dbit.com/pub/putr/
For those RT-11 users who have a legal license to run RT-11 under
E11, there are two alternatives. If the user has a DVD drive or CDROM
drive which allows the first 64 blocks (16 CD sectors) to be read under
E11, the it is possible to MOUNT the complete CD as an image as in:
E11>MOUNT DU0: CDROMD:/RONLY
Of course, it is also possible use the actual ISO image file of the CD as in:
E11>MOUNT DU0: C:RT11DV10.ISO/RONLY
OR, if the files are on a CD and the user prefers:
E11>MOUNT DL0: D:RTV5RL.03/RONLY
Depending on the option that the user prefers, the complete text would be:
C:\>COPY D:ZEROSDL.DSK C:RT11V5.03
C:\>E11
E11>MOUNT DU0: CDROMD:/RONLY
E11>MOUNT DL1: C:RT11V5.03
E11>BOOT DU0:
COPY/DEVICE/FILES DU0:RTV5RL.03 DL1:
If anyone requires any help in using V1-0 of the CD, please ask!!!!!!!!
MY BACKGROUND
--------------
I am an RT-11 software addict. I don't know why, but maybe a partial
answer is because I have OCB (Obsessive Compulsive Behaviour). In
any case, I have used RT-11 off and on for almost 30 years (mostly ON)
since I started with V02 of RT-11. I can't remember a single year, since,
when I did not use RT-11 some of the time and some years I probably
spent more than 4000 hours designing, writing and debugging RT-11
software. I am presently in the midst of fixing bugs and making some
of the enhancements mentioned below.
PURPOSE OF THIS POST
-----------------------
This point is to ask if there are any RT-11 users, either active or inactive,
who want to see any of the RT-11 bugs fixed along with enhancements
added. While I believe that these news groups are the best means of
making contact, perhaps others exist as well. PLEASE mention any
other method of contacting RT-11 users. INDEED, PLEASE forward
this message to them if you know of any other RT-11 users who may
still be interested.
LIST OF ITEMS TO BE CONSIDERED
-----------------------------------
(a) Bugs! There are a (large??) number of bugs, some of which are
able to crash RT-11. Identifying them (probably the biggest part of the
problem) as well as fixing them will be a large part of the effort.
(b) Enhancements are also very important. Just a few are:
- A Path Handler
- Enhancements to the MSCP device driver
- Enhancements to the SL: (Single Line Editor)
- Y2K for older versions followed by Y9K
- Enhancements to the BINCOM program
- Enhancements to the VBGEXE program
- Allow use of SET commands with ANY file
- Increase the precision of the TIME command to 1 tick followed by 1 msec
- Being able to BOOT an LD: file
While I am aware of a VERY small number of RT-11 users, the
more that are still interested, the more ideas that can be generated.
Please reply, either by private e-mail (first fix my munged e-mail
address since the original e-mail address is no longer active) or
else send a post and I will reply privately if so requested.
I am also sending this post to classiccmp.org, so if you are a member
of that list, you can reply either to that list or directly to me in a private
manner if you so choose.
Sincerely yours,
Jerome Fine
--
If you attempted to send a reply and the original e-mail
address has been discontinued due a high volume of junk
e-mail, then the semi-permanent e-mail address can be
obtained by replacing the four characters preceding the
'at' with the four digits of the current year.
Hi Sellam, I sent off another email with a comment with regards to what I was proposing. There is a misunderstanding with regards to what I was proposing. I was not looking at having two copies of the data. Only one is required and that is what the formatting information references when it needs sector data.
I think it would be better at this point for us all to try and understand all the facits of what is being done. My proposal for the data blocks are not a file specification per se. I would gather that a data block would consist of a descriptor that would indicate how many blocks comprise the data, a data type such as "file", internal data (boot sector), block ID for use with the formatting info. The formatting info would tell whether it is sequential, track/sector oriented, etc.
There is no need to create a bunch of different types. Even your description created two distinct formats for data. I was being more generic in that a engineering design for a image file could be created that was not difficult and could satisfy both EASY emulator access for non-physical media access and image file description for another tool to actually create the media required.
best regards, Steve Thatcher
-----Original Message-----
From: Vintage Computer Festival <vcf(a)siconic.com>
Sent: Aug 11, 2004 9:07 PM
To: "General Discussion: On-Topic and Off-Topic Posts" <cctalk(a)classiccmp.org>
Subject: RE: Let's develop an open-source media archive standard
On Wed, 11 Aug 2004, Steve Thatcher wrote:
> I realize that the idea is to create a format to make re-creation of
> media possible for a variety of platforms. We can certainly do that and
> have its only function be to maintain a physical data format. My added
> idea is that the data and the formatting be separated so that a simple
> utility on a non-target platform could extract data from the image file.
This can be done anyway. Parsing the tags should not be too difficult.
And unless someone can come up with an elegant way of doing it, what
you're proposing will require two sets of data in the imagefile: one in
the structured format and one in an unstructured format. I suppose tags
could be added to specify this is the case so that "lazy" programs don't
have to go through all the trouble of parsing the structured data.
> If we create a physical description only and do not abstract the data
> then any emulator must understand the OS file structure in order to
> retrieve any internal file representation. My idea would make the file
> re-creation simple in that the xml image file would be parsed for the
> actual file data that an emulator would need. This makes the emulator
> easier.
You're describing an imagefile that contains a filesystem image, rather
than, say, a disk image. This can be accomodated in the spec, but it's a
different type of image than what we've been discussing.
Remember, I envision the spec being able to handle:
1) A filesystem image (as you describe)
2) An image in "logical" format (i.e. blocks structured in tracks and
sectors for a floppy, or cards for a punch card deck)
3) An image in "raw" format (a bit stream, or the actual punched holes
>from a punch card deck)
And there's actually a:
2.5) Magnetic media at a level below the "logical" format (decoded tracks
and sectors) but above a bit-stream, which would be the raw sectors on a
disk or tape including address headers, data headers, prologue/epilogue
bytes, sync bytes, etc.
> To retrieve a file from the physical layout that it at the end of this
> message, the emulator must know the actual disk format that is used on
> the target system (the one the image file was made for). I have seen
Right. If you choose to store an image in the raw disk format.
> cp/m systems where the actual physical sectors were sequential on disk
> and the OS file sector was actually virtual to increase speed. Not my
> idea of the way to do it. It is much easier to make the physical sectors
> slewed so that a physical sector is a file sector. These are the types
> of issues you will have to overcome if an emulator must totally
> understand each and every file system for a cp/m version for example.
It's up to emulator writers to read the spec and adapt to it. We'll make
the spec as flexible as possible, but first and foremost, the spec is
intended to archive images of data media, and not to serve as a universal
emulator image format (though it should be able to be used as such).
> Sellam, let me know if you would like to discuss this via telephone so I
> can convery the idea that I am proposing.
I think I understand what you're saying. Let me know if I still don't get
it.
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]