13 Apr
2012
13 Apr
'12
4:28 p.m.
Is there any kind of sensible maximum limit for the number of directory
entries that a CP/M disk may contain? Or did every vendor just do their own
thing (probably!) and it's preferable to just keep parsing from the start
of the directory area until either the end of the media is reached, or a
location identified as file data by a previous entry is encountered?
And, related to that, if the directory area of a CP/M disk spans multiple
tracks, was there a standard for whether it iterated head-first (in the
event of media with multiple heads) or cylinder-first? Or does code have to
cope with both possibilities?
I'm just pulling some data off some QX-10 floppies (none of which seem to
take up more than a track for the directory area), but it seems like a good
idea to make it as generic as possible.
cheers
Jules
13 Apr
13 Apr
4:41 p.m.
On 04/13/2012 04:28 PM, Jules Richardson wrote:
Is there any kind of sensible maximum limit for the
number of directory
entries that a CP/M disk may contain? Or did every vendor just do their
own thing (probably!) and it's preferable to just keep parsing from the
start of the directory area until either the end of the media is
reached, or a location identified as file data by a previous entry is
encountered?
This is handled by the DISKDEF macro; one of the values you hand to it
when defining a disk format is the number of directory entries. It's
pretty arbitrary, but much like configuring the number of inodes in
older UNIX filesystems, one can strike a good balance. It's (usually!)
silly to have 255 directory entries, for example, on a 180KB disk format.
-Dave
--
Dave McGuire, AK4HZ
New Kensington, PA
5 p.m.
I wrote that the upper bound on directory entries was 65535. I
overlooked that there can only be a maximum of 16 allocation blocks used
for directory entries, so the maximum is 8192 directory entries, if the
allocation block is 16KB, and correspondingly fewer directory entries
for smaller allocation block sizes. Typically on a floppy disk the
allocation block size would be 1K or 2K, which results in an upper bound
of 512 or 1024 directory entries.
Eric
4:43 p.m.
Are you writing your own utilities?
When I wanted to pull files off of some CPM-86 formatted floppies from
a Tektronix 4170 system I used Cpmtools 2.9:
http://www.cpm8680.com/cpmtools/index.htm
These tools are reasonably generic as far as the disk format goes.
There is a disk definition file with some known formats. None of
those appeared to exactly match the Tektronix 4170 format so I just
added an appropriate definition and then extracted files from the disk
images I read from the physical disk using ImageDisk.
I actually had to write a simple utility to reorder the sectors as
written to the image file from ImageDisk before using Cpmtools on the
disk image. ImageDisk wrote the sectors from one side then the next
side before moving to the next track. For the Tek 4170 format I had
to reorder them from one side increasing tracks, then the next side
decreasing tracks. There doesn't seem to be an obvious way to get
ImageDisk to reorder the sectors that way when writing an image, and
there didn't seem to be an obvious way to get Cpmtools to work with
the sectors in ImageDisk order for the Tek 4170 format.
-Glen
7:36 p.m.
On 04/13/2012 03:43 PM, Glen Slick wrote:
Are you writing your own utilities?
When I wanted to pull files off of some CPM-86 formatted floppies from
a Tektronix 4170 system I used Cpmtools 2.9:
http://www.cpm8680.com/cpmtools/index.htm
These tools are reasonably generic as far as the disk format goes.
There is a disk definition file with some known formats. None of
those appeared to exactly match the Tektronix 4170 format so I just
added an appropriate definition and then extracted files from the disk
images I read from the physical disk using ImageDisk.
I actually had to write a simple utility to reorder the sectors as
written to the image file from ImageDisk before using Cpmtools on the
disk image. ImageDisk wrote the sectors from one side then the next
side before moving to the next track. For the Tek 4170 format I had
to reorder them from one side increasing tracks, then the next side
decreasing tracks. There doesn't seem to be an obvious way to get
ImageDisk to reorder the sectors that way when writing an image, and
there didn't seem to be an obvious way to get Cpmtools to work with
the sectors in ImageDisk order for the Tek 4170 format.
That pretty much resembles what I'm doing; I have a bunch of images in
Imagedisk format and needed to write a util on the Linux side to spit them
out in the correct order so that cpmtools could process them.
I've already got code I wrote to read Imagedisk-format files into a
sensible structure in memory, and so it wouldn't be a big step to dump the
data into a cpmtools-friendly format - but once I'd read the cpmtools man
page discussing the filesystem format, it didn't actually seem like it
would be a huge amount more effort to write code to extract the files
myself (with the benefit that I can quickly follow it and modify it if I
ever need to).
cheers
Jules
4:48 p.m.
Jules Richardson wrote:
Is there any kind of sensible maximum limit for the
number of
directory entries that a CP/M disk may contain?
As far as I can tell, the upper bound appears to be 65535, which back in
the day would only have been sensible on a hard disk, and probably not
even then. Unfortunately CP/M 2.x determines the number of directory
entries is determined by the DPB (Disk Parameter Block) in the BIOS, and
not by any on-disk metadata. It is possible that some BIOSes that
support multiple media formats on the same drive might dynamically
update the DPB based on on-disk metadata stored somewhere in the
reserved tracks, but AFAIK there wasn't any standard for that.
I haven't looked at CP/M 3, so I don't know whether they improved this.
Eric
5:43 p.m.
On Fri, 13 Apr 2012, Jules Richardson wrote:
Is there any kind of sensible maximum limit for the
number of directory
entries that a CP/M disk may contain? Or did every vendor just do their own
thing (probably!) and it's preferable to just keep parsing from the start
of the directory area until either the end of the media is reached, or a
location identified as file data by a previous entry is encountered?
Do you have access to a working machine?
STAT DSK:
will display that information.
If not, then look for sectors that are NOT DIRectory.
And, related to that, if the directory area of a CP/M
disk spans multiple
tracks, was there a standard for whether it iterated head-first (in the
event of media with multiple heads) or cylinder-first? Or does code have to
cope with both possibilities?
Yes, there were standards.
Standard 1: iterate head first, each side is a track (slightly faster)
Standard 2: iterate head first, but count both sides as one track (only
obviously different from #1 for some sequences of sectors)
Standard 3: iterate cylinder first, When you reach the last cylinder,
continue with the first cylinder on the second side
Standard 4: iterate cylinder first, When you reach the last cylinder,
continue with the first data cylinder on the second side (see "reserved
tracks")
Standard 5: iterate cylinder first, When you reach the last cylinder,
continue with the last cylinder on the second side, and count down
(saves a seek to 0 over #3 or #4) (Note that "last" may not match
physical last of the disk (see HP), just as "last sector" on a track may
not match the number that were formatted on the track)
Each person could have their own unique standard.
And then, there were a few "Non-standard" ones.
I'm just pulling some data off some QX-10 floppies
"Valdocs"?
(none of which seem to
take up more than a track for the directory area), but it seems like a good
idea to make it as generic as possible.
If you are dealing with multiple different formats (and QX-10 had
several), number-of-blocks of DIRectory is one of your variables.
Are you planning to make your code general for formats besides Epson QX10?
"Side Pattern" (see above "standards") and "sector sequence
within
track" are some that can NOT be determined from STAT DSK: display.
--
Grumpy Ol' Fred cisin at xenosoft.com
7:26 p.m.
On 13 Apr 2012 at 14:43, Fred Cisin wrote:
Yes, there were standards.
Standard 6: (National BLC 86/128)
Start in the middle of the disk with the directory on side 1,
decrement the cylinder, alternating sides until cylinder 0. Continue
from cylinder 40, side 0, incrementing the cylinder,
alternating
sides until cylinder 79, side 1.
Standard 7: (Cifer 2684)
Start at the beginning (cylinder 0, side 0), increment alternating
sides, but skew each new cylinder sufficiently to accommodate seek
time and head settling time. Thus, each successive side and cylinder
starts with a different sector.
(those are a couple off the top of my head that you failed to
mention).
--Chuck
10:40 p.m.
On Fri, 13 Apr 2012, Chuck Guzis wrote:
Standard 6: (National BLC 86/128)
Start in the middle of the disk with the directory on side 1,
decrement the cylinder, alternating sides until cylinder 0. Continue
from cylinder 40, side 0, incrementing the cylinder, alternating
sides until cylinder 79, side 1.
THAT is what I meant when I said "non-standard".
I still think that the presence of a track 0 switch and track 0 stop, on
the drives, with a "seek to 0", made low tracks faster to get to than
center of the disk.
I never implemented any of the formats that spread tracks, or even just
single disk file-system, accross multiple disks.
Would there be any benefit to doing all of the prime numbered cylinders
first, then the Fibonacci numbers, etc.?
Standard 7: (Cifer 2684)
Start at the beginning (cylinder 0, side 0), increment alternating
sides, but skew each new cylinder sufficiently to accommodate seek
time and head settling time. Thus, each successive side and cylinder
starts with a different sector.
I consider THAT to be an issue of sector sequence within the track issue,
rather than "side-pattern". But, I was a beginner working in a vacuum
when I wrote XenoCopy, so I came up with my own names for all of the
variables.
(those are a couple off the top of my head that you
failed to
mention).
There are indubitably MANY other possibilities.
But, of the 400+ formats of XenoCopy, the majority are CP/M or CP/M like
structures, and I needed to make less than 100 special case exceptions
within the "CP/M like" formats. And even fewer exceptions among the
P-system, MS-DOS, and MS Stand-Alone BASIC formats.
--
Fred Cisin cisin at xenosoft.com
XenoSoft http://www.xenosoft.com
PO Box 1236 (510) 558-9366
Berkeley, CA 94701-1236
11:28 p.m.
On 13 Apr 2012 at 19:40, Fred Cisin wrote:
I still think that the presence of a track 0 switch
and track 0 stop,
on the drives, with a "seek to 0", made low tracks faster to get to
than center of the disk.
I'm not certain that either way is faster in practice--some BIOS
versions use the recalibrate command stepping at the lowest rate, so
they're actually slower than a seek.
But since opening a file extent involves reading the directory entry,
putting the directory in the center means that you half to seek a
maximum of half the disk after an open.
However, higher-numbered cylinders exhibit higher packing densities,
so in theory the center tracks are less reliable than the outer ones.
On the other hand, for many controllers (e.g. NEC 765) a reset
involves a recalibrate, so cylinder 0 at power-on is the somewhat
likely to get clobbered by a hardware glitch.
I recall working on a hard disk file system where the allocation
information for a cylinder was kept on the first sector of the
cylinder. The master allocation bitmap recorded the cylinders that
contained unallocated sectors. I don't know if it turned out to be
faster.
Floppies were always nasty with allocation maps getting clobbered
leaving one to wonder how to put things back together again. The
first allocation scheme for floppy that I ever wrote used contiguous
allocation (create, write, truncate a la RT11) so losing the
directory wasn't a serious issue and consecutive file access was
always fast. Of course, the application had to have some sort of
idea of how large a file was going to be when creating the file.
Defragmenting was an obvious burden, but for many applications it was
less necessary than you'd think.
--Chuck
10:08 p.m.
On 04/13/2012 04:43 PM, Fred Cisin wrote:
I suspected that might be the case, although I didn't realize there was
quite *that* much variety...
Yes, several different releases, but also various other tools, utilities,
public domain disks, and a couple of games. The games disks seem to have
some form of copy protection such that Imagedisk chokes on them (they work
OK on the system itself), but everything else read OK[1]. I just want to
check that they are what they claim to be (and haven't been overwritten
with e.g. previous owner's financial records!) before I upload them somewhere.
[1] Although I have a neat little pile of hub rings sitting on the desk
next to me; they were falling off every single one of the PD disks. I don't
know how common others have found that, but it's been a *long* time since
I've seen a hub ring failure (sadly there's no vendor information on the
disks, so I don't know who made them).
Well, it was just a thought that it it was virtually no effort to make
things quite generic, I may as well. I think cpmtools claims a directory
entry limit of 128 for the QX-10, which is comfortably within the 512x10
track format of all the disks that I seem to have, but it wouldn't be much
work to iterate across multiple tracks for a more flexible reader.
cheers
Jules
On Fri, 13 Apr 2012, Jules Richardson wrote:
Thanks - yes, I do, so will give that a try...
Is there any kind of sensible maximum limit for
the number of directory
entries that a CP/M disk may contain? Or did every vendor just do their own
thing (probably!) and it's preferable to just keep parsing from the start
of the directory area until either the end of the media is reached, or a
location identified as file data by a previous entry is encountered?
Do you have access to a working machine?
STAT DSK:
will display that information. And, related
to that, if the directory area of a CP/M disk spans multiple
tracks, was there a standard for whether it iterated head-first (in the
event of media with multiple heads) or cylinder-first? Or does code have to
cope with both possibilities?
Yes, there were standards.
Standard 1: iterate head first, each side is a track (slightly faster)
... I'm just
pulling some data off some QX-10 floppies
"Valdocs"? (none of which
seem to
take up more than a track for the directory area), but it seems like a good
idea to make it as generic as possible.
If you are dealing with multiple different formats (and QX-10 had
several), number-of-blocks of DIRectory is one of your variables.
Are you planning to make your code general for formats besides Epson QX10? 
8:13 p.m.
On 04/13/2012 04:28 PM, Jules Richardson wrote:
Is there any kind of sensible maximum limit for the number of
directory entries that a CP/M disk may contain? Or did every vendor
just do their own thing (probably!) and it's preferable to just keep
parsing from the start of the directory area until either the end of
the media is reached, or a location identified as file data by a
previous entry is encountered?
And, related to that, if the directory area of a CP/M disk spans
multiple tracks, was there a standard for whether it iterated
head-first (in the event of media with multiple heads) or
cylinder-first? Or does code have to cope with both possibilities?
I'm just pulling some data off some QX-10 floppies (none of which seem
to take up more than a track for the directory area), but it seems
like a good idea to make it as generic as possible.
cheers
Jules
For practical purposes there are limits. those are the amount of buffer
to directory entries,
and the time it take to do a linear search.
there was only one standard 8"SSSD with resulted in 64 directory
entries, 1kbyte allocation size, and
a whopping 241allocation block (241K of storage). that averages to 4
allocation blocks or 4K per
file and either the disk is full or the directory fills.
The other limit is a logical device maximum is 65535 128 byte logical
sectors or 8191.875 Kilobytes.
since the smallest allocation size allowed for disks over 255Kbytes is
2K that means the largest directory
will be 4096 entries (at 32bytes per entry or 128Kbytes of the disk).
That represents the wall as far as
how high can you go.
The lower end is a 82kb (NS* SSSD, and others) where 32 entries eating
1KB of disk and 81K left for
storage.
If there is a pattern that emerged its that many CP/M files were in the
4-16K size and for the larger disks
allocation sizes of 2K or 4K were common with directory sizes based on a
range of 4K to 16K per file.
So I'd see 5.25" DSQD (720-800k) disks set up to have directories
accommodating 128 or 256 entries
and the larger hard disks usually set for 512, 1024 or maybe 2048 entries.
There may have been some ill conceived few that had way to few entries
way to many for the
media. I did see on 8MB disk with 16K allocation size (kinda large for
practical use) and as a result
only 512 entries were possible but the vendor did 1024 (it would break
at the 512th file added).
NOTE: That was for CP/M-80 though V2.x, cpm 3 fixed the math so logical
devices could go to
32mb. At 32MB a 8192 entry directory might make sense but to get a
32MB disk you also
used the larger allocation sizes like 8 or 16K and that again limited
the number of possible files
to less.
Some of the CP/M clones P2dos, Suprbdos, and ZPR could allocate
devices as large
as 2GB but then file sorting and directory times could be painfully
slow. That and with
a relatively flat directory more than 1024 entries tend to be more
annoying than useful.
The largest I'd done was a CP/M clone using Suprbdos and a 2GB CF
partitioned as
four 128MB, and two 512mb logical devices. Those proved to be not
manageable
as the 512MB partitions with 16K allocation block size the best plan was
32,768
entries. I gave up on that and found a 32MB CF and used that as one 16MB
(1024 entries)
and four 8MB (512 entries) which has more reasonable proportions. FYI:
even with the
Suprbdos hash doing a DIR of the 16mb partition was a minute of waiting
thing
even with a 10mhz Z80 and banked ram. It took 32KBytes of data shuffling
(read and parse) for that 1024 directory entry.
I didn't get into the fact that CP/M maintains ALLOC Bit map in ram and
its size is
1 bit per allocation block so a 8MB disk has 8192 ALLOC bits and need a
1K space
(per logical drive) in ram to keep track of them! With 64K of
addressable space
that alone is painful. There is also a directory buffer but usually
thats limited to
128bytes. and if the device is not self cached (IDE,CF are) then the
BIOS also
has a host sector cache for deblocking and usually that 512or 1025 bytes.
So an old style 8mb hard disk could easily hit the system for 1.5k of
scarce ram.
Allison
4459
days inactive
4460
days old
11 comments
7 participants
participants (7)
-
ajp166＠verizon.net -
cclist＠sydex.com -
cisin＠xenosoft.com -
eric＠brouhaha.com -
glen.slick＠gmail.com -
jules.richardson99＠gmail.com -
mcguire＠neurotica.com