11 Aug
2004
11 Aug
'04
9:29 a.m.
This example represents the block data using metatags...I guess along the
"XML" part of the thread.
I was thinking similarly to you but not using XML metadata:
;Hardware descriptor
MFGR
MACHINE
SUBTYPE
DRIVETYPE (this of course defines what follows)
;for floppy
DRIVESIZE
ENCODING
TRACKS
SECTORS
SECTSIZE
;HexData
; Each record or group of records contains the related media data. The
address record would be used for encoding the metadata
00TTSSHH: (00-track-sector-head)
I looked to Intel Hex (or Motorola) because it had built-in CRC facilities
and it was human-readable ASCII. The drive and machine description could be
encoded in special MOT records probably.
XML is more a more "current" technology but I was trying to keep with the
platform neutrality by sticking to text-only and not assuming the use of any
other technology like XML.
Rich
-----Original Message-----
From: cctalk-bounces(a)classiccmp.org
[mailto:cctalk-bounces@classiccmp.org]On Behalf Of Vintage Computer
Festival
Sent: Wednesday, August 11, 2004 1:56 PM
To: General Discussion: On-Topic and Off-Topic Posts
Subject: RE: Let's develop an open-source media archive standard
On Wed, 11 Aug 2004, Cini, Richard wrote:
I might have missed what the ultimate use of this
archive would be. Will
the
archive be used to (1) re-generate original media; (2)
operate with
emualtors; (3) both?
Both. Emulators will certainly be able to make use of the archive by
having parsers built-in that can translate the archive data into
something the emulator can use. So instead of point the emulator to a
binary disk image, you would point it to an archive file and it would
translate the file back into tracks/sectors, or punch cards, or whatever.
To ensure integrity of the data I would propose
recording the data in the
Intel Hex format -- it's text-based and has built-in CRC. Now, we'd have
to
modify the standard format a bit to accommodate a
larger address space and
to add some sort of standardized header (a "Hardware Descriptor"). This
data
would be used by the de-archiver to interpret the
stream of data read from
the data area (the "Hex Block").
I think you're thinking of this in terms of a large binary file encoded as
ASCII hex. If so, this is not what's being proposed. What is being
discussed is a format which actually describes the physical medium. For
example, on floppy:
<MEDIA TYPE=FLOPPY SIZE=5.25 SIDES=1 DENSITY=SINGLE FORMAT=GCR TRACKS=35
SECTORS=16 SECTORSIZE=256>
<VOLUME>Apple ][ System Disk</VOLUME>
</MEDIA>
<DATA>
<TRACK 0><SECTOR 0>
HERE WOULD BE THE ASCII HEX DATA FOR TRACK 0, SECTOR 0
</SECTOR></TRACK>
...
<TRACK 34><SECTOR 15>
HERE WOULD BE THE ASCII HEX DATA FOR TRACK 34, SECTOR 15
</SECTOR></TRACK>
</DATA>
I think that we should start compiling a list of the
various media we want
represented and how that media is organized natively. I don't mean "well,
it
has blocks and sectors" either. We should examine
the exact format down to
the actual numbers (i.e., "2048 blocks of 256-bytes recorded twice").
Seeing
how the various data stores are organized should bring
some clarity to how
we should represent it.
I agree. This would be useful. Does someone want to volunteer to do
this?
--
Sellam Ismail Vintage Computer
Festival
----------------------------------------------------------------------------
--
International Man of Intrigue and Danger
http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers
]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org
]
11 Aug
11 Aug
10:23 a.m.
On Wed, 11 Aug 2004, Cini, Richard wrote:
This example represents the block data using
metatags...I guess along the
"XML" part of the thread.
I was thinking similarly to you but not using XML metadata:
;Hardware descriptor
MFGR
MACHINE
SUBTYPE
DRIVETYPE (this of course defines what follows)
;for floppy
DRIVESIZE
ENCODING
TRACKS
SECTORS
SECTSIZE
;HexData
; Each record or group of records contains the related media data. The
address record would be used for encoding the metadata
00TTSSHH: (00-track-sector-head)
I looked to Intel Hex (or Motorola) because it had built-in CRC facilities
and it was human-readable ASCII. The drive and machine description could be
encoded in special MOT records probably.
I like the XML style because it's more explicit; more human-readable.
XML is more a more "current" technology but
I was trying to keep with the
platform neutrality by sticking to text-only and not assuming the use of any
other technology like XML.
XML is platform neutral because it's basically ASCII, right?
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
10:46 a.m.
----- Original Message -----
From: "Vintage Computer Festival" <vcf(a)siconic.com>
To: "General Discussion: On-Topic and Off-Topic Posts"
<cctalk(a)classiccmp.org>
Sent: Wednesday, August 11, 2004 3:23 PM
Subject: RE: Let's develop an open-source media archive standard
It usually is. But the beauty of eXtensible Markup Language is that it's...
extensible! :)
http://www.w3.org/TR/2003/NOTE-unicode-xml-20030613/
> XML is more a more "current" technology
but I was trying to keep with
the
> platform neutrality by sticking to text-only and
not assuming the use of
any
other
technology like XML.
XML is platform neutral because it's basically ASCII, right? 
12:43 p.m.
It was thus said that the Great Vintage Computer Festival once stated:
Nope. XML files can be represented in multiple character sets, possibly
including (but certainly not limited to):
437, 500, 500V1, 850, 851, 852, 855, 856, 857, 860, 861, 862, 863, 864, 865,
866, 866NAV, 869, 874, 904, 1026, 1046, 1047, 8859_1, 8859_2, 8859_3, 8859_4,
8859_5, 8859_6, 8859_7, 8859_8, 8859_9, 10646-1:1993, 10646-1:1993/UCS4,
ANSI_X3.4-1968, ANSI_X3.4-1986, ANSI_X3.4, ANSI_X3.110-1983, ANSI_X3.110,
ARABIC, ARABIC7, ARMSCII-8, ASCII, ASMO-708, ASMO_449, BALTIC, BIG-5,
BIG-FIVE, BIG5-HKSCS, BIG5, BIG5HKSCS, BIGFIVE, BS_4730, CA, CN-BIG5, CN-GB,
CN, CP-AR, CP-GR, CP-HU, CP037, CP038, CP273, CP274, CP275, CP278, CP280,
CP281, CP282, CP284, CP285, CP290, CP297, CP367, CP420, CP423, CP424, CP437,
CP500, CP737, CP775, CP813, CP819, CP850, CP851, CP852, CP855, CP856, CP857,
CP860, CP861, CP862, CP863, CP864, CP865, CP866, CP866NAV, CP868, CP869,
CP870, CP871, CP874, CP875, CP880, CP891, CP903, CP904, CP905, CP912, CP915,
CP916, CP918, CP920, CP922, CP930, CP932, CP933, CP935, CP936, CP937, CP939,
CP949, CP950, CP1004, CP1026, CP1046, CP1047, CP1070, CP1079, CP1081, CP1084,
CP1089, CP1124, CP1125, CP1129, CP1132, CP1133, CP1160, CP1161, CP1162,
CP1163, CP1164, CP1250, CP1251, CP1252, CP1253, CP1254, CP1255, CP1256,
CP1257, CP1258, CP1361, CP10007, CPIBM861, CSA7-1, CSA7-2, CSASCII,
CSA_T500-1983, CSA_T500, CSA_Z243.4-1985-1, CSA_Z243.4-1985-2,
CSA_Z243.419851, CSA_Z243.419852, CSDECMCS, CSEBCDICATDE, CSEBCDICATDEA,
CSEBCDICCAFR, CSEBCDICDKNO, CSEBCDICDKNOA, CSEBCDICES, CSEBCDICESA,
CSEBCDICESS, CSEBCDICFISE, CSEBCDICFISEA, CSEBCDICFR, CSEBCDICIT, CSEBCDICPT,
CSEBCDICUK, CSEBCDICUS, CSEUCKR, CSEUCPKDFMTJAPANESE, CSGB2312, CSHPROMAN8,
CSIBM037, CSIBM038, CSIBM273, CSIBM274, CSIBM275, CSIBM277, CSIBM278,
CSIBM280, CSIBM281, CSIBM284, CSIBM285, CSIBM290, CSIBM297, CSIBM420,
CSIBM423, CSIBM424, CSIBM500, CSIBM851, CSIBM855, CSIBM856, CSIBM857,
CSIBM860, CSIBM863, CSIBM864, CSIBM865, CSIBM866, CSIBM868, CSIBM869,
CSIBM870, CSIBM871, CSIBM880, CSIBM891, CSIBM903, CSIBM904, CSIBM905,
CSIBM918, CSIBM922, CSIBM930, CSIBM932, CSIBM933, CSIBM935, CSIBM937,
CSIBM939, CSIBM943, CSIBM1026, CSIBM1124, CSIBM1129, CSIBM1132, CSIBM1133,
CSIBM1160, CSIBM1161, CSIBM1163, CSIBM1164, CSIBM11621162,
CSISO4UNITEDKINGDOM, CSISO10SWEDISH, CSISO11SWEDISHFORNAMES,
CSISO14JISC6220RO, CSISO15ITALIAN, CSISO16PORTUGESE, CSISO17SPANISH,
CSISO18GREEK7OLD, CSISO19LATINGREEK, CSISO21GERMAN, CSISO25FRENCH,
CSISO27LATINGREEK1, CSISO49INIS, CSISO50INIS8, CSISO51INISCYRILLIC,
CSISO58GB1988, CSISO60DANISHNORWEGIAN, CSISO60NORWEGIAN1, CSISO61NORWEGIAN2,
CSISO69FRENCH, CSISO84PORTUGUESE2, CSISO85SPANISH2, CSISO86HUNGARIAN,
CSISO88GREEK7, CSISO89ASMO449, CSISO90, CSISO92JISC62991984B, CSISO99NAPLPS,
CSISO103T618BIT, CSISO111ECMACYRILLIC, CSISO121CANADIAN1, CSISO122CANADIAN2,
CSISO139CSN369103, CSISO141JUSIB1002, CSISO143IECP271, CSISO150,
CSISO150GREEKCCITT, CSISO151CUBA, CSISO153GOST1976874, CSISO646DANISH,
CSISO2022CN, CSISO2022JP, CSISO2022JP2, CSISO2022KR, CSISO2033,
CSISO5427CYRILLIC, CSISO5427CYRILLIC1981, CSISO5428GREEK, CSISO10367BOX,
CSISOLATIN1, CSISOLATIN2, CSISOLATIN3, CSISOLATIN4, CSISOLATIN5, CSISOLATIN6,
CSISOLATINARABIC, CSISOLATINCYRILLIC, CSISOLATINGREEK, CSISOLATINHEBREW,
CSKOI8R, CSKSC5636, CSMACINTOSH, CSNATSDANO, CSNATSSEFI, CSN_369103,
CSPC8CODEPAGE437, CSPC775BALTIC, CSPC850MULTILINGUAL, CSPC862LATINHEBREW,
CSPCP852, CSSHIFTJIS, CSUCS4, CSUNICODE, CUBA, CWI-2, CWI, CYRILLIC, DE,
DEC-MCS, DEC, DECMCS, DIN_66003, DK, DS2089, DS_2089, E13B, EBCDIC-AT-DE-A,
EBCDIC-AT-DE, EBCDIC-BE, EBCDIC-BR, EBCDIC-CA-FR, EBCDIC-CP-AR1,
EBCDIC-CP-AR2, EBCDIC-CP-BE, EBCDIC-CP-CA, EBCDIC-CP-CH, EBCDIC-CP-DK,
EBCDIC-CP-ES, EBCDIC-CP-FI, EBCDIC-CP-FR, EBCDIC-CP-GB, EBCDIC-CP-GR,
EBCDIC-CP-HE, EBCDIC-CP-IS, EBCDIC-CP-IT, EBCDIC-CP-NL, EBCDIC-CP-NO,
EBCDIC-CP-ROECE, EBCDIC-CP-SE, EBCDIC-CP-TR, EBCDIC-CP-US, EBCDIC-CP-WT,
EBCDIC-CP-YU, EBCDIC-CYRILLIC, EBCDIC-DK-NO-A, EBCDIC-DK-NO, EBCDIC-ES-A,
EBCDIC-ES-S, EBCDIC-ES, EBCDIC-FI-SE-A, EBCDIC-FI-SE, EBCDIC-FR,
EBCDIC-GREEK, EBCDIC-INT, EBCDIC-INT1, EBCDIC-IS-FRISS, EBCDIC-IT,
EBCDIC-JP-E, EBCDIC-JP-KANA, EBCDIC-PT, EBCDIC-UK, EBCDIC-US, EBCDICATDE,
EBCDICATDEA, EBCDICCAFR, EBCDICDKNO, EBCDICDKNOA, EBCDICES, EBCDICESA,
EBCDICESS, EBCDICFISE, EBCDICFISEA, EBCDICFR, EBCDICISFRISS, EBCDICIT,
EBCDICPT, EBCDICUK, EBCDICUS, ECMA-114, ECMA-118, ECMA-128, ECMA-CYRILLIC,
ECMACYRILLIC, ELOT_928, ES, ES2, EUC-CN, EUC-JISX0213, EUC-JP, EUC-KR,
EUC-TW, EUCCN, EUCJP, EUCKR, EUCTW, FI, FR, GB, GB2312, GB13000, GB18030,
GBK, GB_1988-80, GB_198880, GEORGIAN-ACADEMY, GEORGIAN-PS, GOST_19768-74,
GOST_19768, GOST_1976874, GREEK-CCITT, GREEK, GREEK7-OLD, GREEK7, GREEK7OLD,
GREEK8, GREEKCCITT, HEBREW, HP-ROMAN8, HPROMAN8, HU, IBM-856, IBM-922,
IBM-930, IBM-932, IBM-933, IBM-935, IBM-937, IBM-939, IBM-943, IBM-1046,
IBM-1124, IBM-1129, IBM-1132, IBM-1133, IBM-1160, IBM-1161, IBM-1162,
IBM-1163, IBM-1164, IBM037, IBM038, IBM256, IBM273, IBM274, IBM275, IBM277,
IBM278, IBM280, IBM281, IBM284, IBM285, IBM290, IBM297, IBM367, IBM420,
IBM423, IBM424, IBM437, IBM500, IBM775, IBM813, IBM819, IBM848, IBM850,
IBM851, IBM852, IBM855, IBM856, IBM857, IBM860, IBM861, IBM862, IBM863,
IBM864, IBM865, IBM866, IBM866NAV, IBM868, IBM869, IBM870, IBM871, IBM874,
IBM875, IBM880, IBM891, IBM903, IBM904, IBM905, IBM912, IBM915, IBM916,
IBM918, IBM920, IBM922, IBM930, IBM932, IBM933, IBM935, IBM937, IBM939,
IBM943, IBM1004, IBM1026, IBM1046, IBM1047, IBM1089, IBM1124, IBM1129,
IBM1132, IBM1133, IBM1160, IBM1161, IBM1162, IBM1163, IBM1164, IEC_P27-1,
IEC_P271, INIS-8, INIS-CYRILLIC, INIS, INIS8, INISCYRILLIC, ISIRI-3342,
ISIRI3342, ISO-2022-CN-EXT, ISO-2022-CN, ISO-2022-JP-2, ISO-2022-JP-3,
ISO-2022-JP, ISO-2022-KR, ISO-8859-1, ISO-8859-2, ISO-8859-3, ISO-8859-4,
ISO-8859-5, ISO-8859-6, ISO-8859-7, ISO-8859-8, ISO-8859-9, ISO-8859-10,
ISO-8859-11, ISO-8859-13, ISO-8859-14, ISO-8859-15, ISO-8859-16, ISO-10646,
ISO-10646/UCS2, ISO-10646/UCS4, ISO-10646/UTF-8, ISO-10646/UTF8, ISO-CELTIC,
ISO-IR-4, ISO-IR-6, ISO-IR-8-1, ISO-IR-9-1, ISO-IR-10, ISO-IR-11, ISO-IR-14,
ISO-IR-15, ISO-IR-16, ISO-IR-17, ISO-IR-18, ISO-IR-19, ISO-IR-21, ISO-IR-25,
ISO-IR-27, ISO-IR-37, ISO-IR-49, ISO-IR-50, ISO-IR-51, ISO-IR-54, ISO-IR-55,
ISO-IR-57, ISO-IR-60, ISO-IR-61, ISO-IR-69, ISO-IR-84, ISO-IR-85, ISO-IR-86,
ISO-IR-88, ISO-IR-89, ISO-IR-90, ISO-IR-92, ISO-IR-98, ISO-IR-99, ISO-IR-100,
ISO-IR-101, ISO-IR-103, ISO-IR-109, ISO-IR-110, ISO-IR-111, ISO-IR-121,
ISO-IR-122, ISO-IR-126, ISO-IR-127, ISO-IR-138, ISO-IR-139, ISO-IR-141,
ISO-IR-143, ISO-IR-144, ISO-IR-148, ISO-IR-150, ISO-IR-151, ISO-IR-153,
ISO-IR-155, ISO-IR-156, ISO-IR-157, ISO-IR-166, ISO-IR-179, ISO-IR-193,
ISO-IR-197, ISO-IR-199, ISO-IR-203, ISO-IR-209, ISO-IR-226, ISO646-CA,
ISO646-CA2, ISO646-CN, ISO646-CU, ISO646-DE, ISO646-DK, ISO646-ES,
ISO646-ES2, ISO646-FI, ISO646-FR, ISO646-FR1, ISO646-GB, ISO646-HU,
ISO646-IT, ISO646-JP-OCR-B, ISO646-JP, ISO646-KR, ISO646-NO, ISO646-NO2,
ISO646-PT, ISO646-PT2, ISO646-SE, ISO646-SE2, ISO646-US, ISO646-YU,
ISO2022CN, ISO2022CNEXT, ISO2022JP, ISO2022JP2, ISO2022KR, ISO6937,
ISO8859-1, ISO8859-2, ISO8859-3, ISO8859-4, ISO8859-5, ISO8859-6, ISO8859-7,
ISO8859-8, ISO8859-9, ISO8859-10, ISO8859-11, ISO8859-13, ISO8859-14,
ISO8859-15, ISO8859-16, ISO88591, ISO88592, ISO88593, ISO88594, ISO88595,
ISO88596, ISO88597, ISO88598, ISO88599, ISO885910, ISO885911, ISO885913,
ISO885914, ISO885915, ISO885916, ISO_646.IRV:1991, ISO_2033-1983, ISO_2033,
ISO_5427-EXT, ISO_5427, ISO_5427:1981, ISO_5427EXT, ISO_5428, ISO_5428:1980,
ISO_6937-2, ISO_6937-2:1983, ISO_6937, ISO_6937:1992, ISO_8859-1,
ISO_8859-1:1987, ISO_8859-2, ISO_8859-2:1987, ISO_8859-3, ISO_8859-3:1988,
ISO_8859-4, ISO_8859-4:1988, ISO_8859-5, ISO_8859-5:1988, ISO_8859-6,
ISO_8859-6:1987, ISO_8859-7, ISO_8859-7:1987, ISO_8859-8, ISO_8859-8:1988,
ISO_8859-9, ISO_8859-9:1989, ISO_8859-10, ISO_8859-10:1992, ISO_8859-14,
ISO_8859-14:1998, ISO_8859-15:1998, ISO_9036, ISO_10367-BOX, ISO_10367BOX,
ISO_69372, IT, JIS_C6220-1969-RO, JIS_C6229-1984-B, JIS_C62201969RO,
JIS_C62291984B, JOHAB, JP-OCR-B, JP, JS, JUS_I.B1.002, KOI-7, KOI-8, KOI8-R,
KOI8-T, KOI8-U, KOI8, KOI8R, KOI8U, KSC5636, L1, L2, L3, L4, L5, L6, L7, L8,
L10, LATIN-GREEK-1, LATIN-GREEK, LATIN1, LATIN2, LATIN3, LATIN4, LATIN5,
LATIN6, LATIN7, LATIN8, LATIN10, LATINGREEK, LATINGREEK1, MAC-CYRILLIC,
MAC-IS, MAC-SAMI, MAC-UK, MAC, MACCYRILLIC, MACINTOSH, MACIS, MACUK,
MACUKRAINIAN, MS-ANSI, MS-ARAB, MS-CYRL, MS-EE, MS-GREEK, MS-HEBR,
MS-MAC-CYRILLIC, MS-TURK, MSCP949, MSCP1361, MSMACCYRILLIC, MSZ_7795.3,
MS_KANJI, NAPLPS, NATS-DANO, NATS-SEFI, NATSDANO, NATSSEFI, NC_NC0010,
NC_NC00-10, NC_NC00-10:81, NF_Z_62-010, NF_Z_62-010_(1973), NF_Z_62-010_1973,
NF_Z_62010, NF_Z_62010_1973, NO, NO2, NS_4551-1, NS_4551-2, NS_45511,
NS_45512, OS2LATIN1, OSF00010001, OSF00010002, OSF00010003, OSF00010004,
OSF00010005, OSF00010006, OSF00010007, OSF00010008, OSF00010009, OSF0001000A,
OSF00010020, OSF00010100, OSF00010101, OSF00010102, OSF00010104, OSF00010105,
OSF00010106, OSF00030010, OSF0004000A, OSF0005000A, OSF05010001, OSF100201A4,
OSF100201A8, OSF100201B5, OSF100201F4, OSF100203B5, OSF1002011C, OSF1002011D,
OSF1002035D, OSF1002035E, OSF1002035F, OSF1002036B, OSF1002037B, OSF10010001,
OSF10020025, OSF10020111, OSF10020115, OSF10020116, OSF10020118, OSF10020122,
OSF10020129, OSF10020352, OSF10020354, OSF10020357, OSF10020359, OSF10020360,
OSF10020364, OSF10020365, OSF10020366, OSF10020367, OSF10020370, OSF10020387,
OSF10020388, OSF10020396, OSF10020402, OSF10020417, PT, PT2, R8, ROMAN8,
RUSCII, SE, SE2, SEN_850200_B, SEN_850200_C, SHIFT-JIS, SHIFT_JIS,
SHIFT_JISX0213, SJIS, SS636127, ST_SEV_358-88, T.61-8BIT, T.61, T.618BIT,
TCVN-5712, TCVN, TCVN5712-1, TCVN5712-1:1993, TIS-620, TIS620-0,
TIS620.2529-1, TIS620.2533-0, TIS620, TS-5881, TSCII, UCS-2, UCS-2BE,
UCS-2LE, UCS-4, UCS-4BE, UCS-4LE, UCS2, UCS4, UHC, UJIS, UK, UNICODE,
UNICODEBIG, UNICODELITTLE, US-ASCII, US, UTF-7, UTF-8, UTF-16, UTF-16BE,
UTF-16LE, UTF-32, UTF-32BE, UTF-32LE, UTF7, UTF8, UTF16, UTF16BE, UTF16LE,
UTF32, UTF32BE, UTF32LE, VISCII, WCHAR_T, WIN-SAMI-2, WINBALTRIM,
WINDOWS-1250, WINDOWS-1251, WINDOWS-1252, WINDOWS-1253, WINDOWS-1254,
WINDOWS-1255, WINDOWS-1256, WINDOWS-1257, WINDOWS-1258, WINSAMI2, WS2, YU
Best decide this now. You might also want to investigate the work on Atom
(a blogging archive/API format in XML) where they are having to deal with an
explostion of character encoding schemes (for a sample of what they're
doing, read http://intertwingly.net/stories/2004/04/14/i18n.html).
My recomendation if you use XML? Mandate the following:
<?xml version="1.0" encoding="US-ASCII"?>
and be done with the mess that is character encoding. Also, the more
explicit you are in labling the data, the better. So, for instance:
<sector bit-encoding="mfm" encoding="text/base64"> ...
</sector>
where "bit-encoding" is the format used to encode the data on the physical
disk, and "encoding" is the format used *in the archive* of the data. Just
an example, but the more metadata you include, the better.
-spc (It really is quite a mess)
XML is more a more "current" technology
but I was trying to keep with the
platform neutrality by sticking to text-only and not assuming the use of any
other technology like XML.
XML is platform neutral because it's basically ASCII, right?
3:41 p.m.
On Wed, 11 Aug 2004, Sean 'Captain Napalm' Conner wrote:
It was thus said that the Great Vintage Computer
Festival once stated:
Nope. XML files can be represented in multiple character sets, possibly
including (but certainly not limited to):
<snip!>
XML is more a more "current" technology
but I was trying to keep with the
platform neutrality by sticking to text-only and not assuming the use of any
other technology like XML.
XML is platform neutral because it's basically ASCII, right?
Best decide this now.
Ok, I choose US-ASCII. This will be up for debate I'm sure, but surely
US-ASCII is the most widely deployed character set in the world currently?
You might also want to investigate the work on Atom
(a blogging archive/API format in XML) where they are having to deal with an
explostion of character encoding schemes (for a sample of what they're
doing, read http://intertwingly.net/stories/2004/04/14/i18n.html).
You might be missing the point of what we're doing here. We're not
encapsulating textfiles. We're encapsulating binary data inside
textfiles.
My recomendation if you use XML? Mandate the
following:
<?xml version="1.0" encoding="US-ASCII"?>
and be done with the mess that is character encoding.
I never considered anything else, really.
Also, the more
explicit you are in labling the data, the better. So, for instance:
<sector bit-encoding="mfm" encoding="text/base64"> ...
</sector>
where "bit-encoding" is the format used to encode the data on the physical
disk, and "encoding" is the format used *in the archive* of the data. Just
an example, but the more metadata you include, the better.
Well, those are confusing tags. Which is used for the disk encoding and
which is used for the imagefile encoding? Sure, if you have the spec you
can refresh yourself if you forget, but it's not absolutely obvious to
someone who doesn't have the spec.
Anyway, point taken.
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
12 Aug
12 Aug
1:32 a.m.
On Thu, 2004-08-12 at 00:41, Vintage Computer Festival wrote:
On Wed, 11 Aug 2004, Sean 'Captain Napalm'
Conner wrote:
*if* we've decided that it's sensible to use XML for this over some
mechanism, then does it matter? I thought that to be compliant with the
XML spec, the XML document should say what version of the spec and what
character encoding it uses?
In other words, who cares what charset is used - people can use whatever
charset makes sense for them. It just needs to be spelled out that it's
mandatory to say which charset is in use for the archive to be valid.
Someone in Japan, say, may well want to fill in data fields in the
archive (such as description) using their native language. We shouldn't
stop them from doing this and force them to use a single-byte character
set such as ASCII.
I'd rather future generations stumble across an archive in Japanese and
have to translate it if necessary at the time, rather than say someone
in Japan who wasn't too good at English be forced to fill in data in
English and end up with ambiguous information.
cheers
Jules
It was thus said that the Great Vintage Computer
Festival once stated:
Nope. XML files can be represented in multiple character sets, possibly
including (but certainly not limited to):
<snip!>
XML is more a more "current" technology
but I was trying to keep with the
platform neutrality by sticking to text-only and not assuming the use of any
other technology like XML.
XML is platform neutral because it's basically ASCII, right?
Best decide this now.
Ok, I choose US-ASCII. This will be up for debate I'm sure, but surely
US-ASCII is the most widely deployed character set in the world currently? 
5:06 a.m.
New subject: Charset/CCDD (was: Let's develop an open-source media archive standard)
Am 11 Aug 2004 17:41 meinte Vintage Computer Festival:
On Wed, 11 Aug 2004, Sean 'Captain Napalm'
Conner wrote:
> It was thus said that the Great Vintage Computer Festival once stated:
> > > XML is more a more "current" technology but I was trying to keep
with the
> > > platform neutrality by sticking to text-only and not assuming the use of
any
> > > other technology like XML.
> > XML is platform neutral because it's basically ASCII, right?
Nope. XML
files can be represented in multiple character sets, possibly
including (but certainly not limited to):
<snip!>
> Best decide this now. Ok, I choose US-ASCII. This will be up for debate
I'm sure, but surely
US-ASCII is the most widely deployed character set in the world currently?
Well, yes and no. Shure, a lot of the most common codes are
to some extend US-ASCII compatible, but different, thus even
the use of a stric 7 Bit ASCII doesn't save us from complications.
But beside that, it's not the point. Especialy when allowing
the inclusion of binary data, and/or when switching between
systems, ASCII can become incompatible.
Further more, such a definition (US-ASCII only) would make it
quite hard to use the format in a mainframe environment, wher
EBCDIC is still the language of choice.
Already on a PET we have differences in the 7 Bit encodeing.
Not to mention the even bigger problems to handle these files
on real old systems.
Rather then restricting the encodeing of the XML file to a
specific charset, we need to restrict the USAGE within the
standard to certain characters, regardless of the encodeing.
I suggest to restrict the caracters used in tags, attribute
names and attributes to 'A-Z' (uppercase), '0-9' and '-'.
these are to be found in all character sets I've seen up to
date, and thus able to tunnel whatever needed. For various
data encodeing schemes additional characters may be needed.
By restricting our definition to this, we become independant
from the charset (as long as it includes our
characters) and
so transparent to all code conversions that may happen on
the way between machines.
Gruss
H.
--
VCF Europa 6.0 am 30.April und 01.Mai 2005 in Muenchen
http://www.vcfe.org/
11:10 a.m.
New subject: Charset/CCDD (was: Let's develop an open-source media archive
It was thus said that the Great Hans Franke once stated:
Rather then restricting the encodeing of the XML file to a
specific charset, we need to restrict the USAGE within the
standard to certain characters, regardless of the encodeing.
Unless otherwise noted, XML files are assumed to be encoded in UTF-8,
*but* an XML parser is required to abort at the first error in the XML file.
If a parser is reading an XML file without an explicit character set
encoding scheme (which means it's assuming UTF-8) and it reads a character
that is illegal (say the file was encoded in ISO-8859-3) it gives up
(usually with an "illegal character at such-n-such position" error).
Right now, this is a real problem with XML deployment (it gets even
wierder when XML files are transported via HTTP but I'm getting ahead of
myself) so when I suggested that (if we are using XML) that each *must*
start with:
<?xml version="1.0" encoding="US-ASCII"?>
It was a way of self-defense. Perhaps it can be relaxed some and require:
<?xml version="1.0" encoding="some XML defined character encoding
scheme"?>
and if the encoding scheme isn't defined, it's an error and further
processing of the archive should stop.
I suggest to restrict the caracters used in tags,
attribute
names and attributes to 'A-Z' (uppercase), '0-9' and '-'.
Unfortunately, XML is defined with lowercase (or it may be case
sensitive---I do know that all XML I've seen is with lowercase tags, and
it's pretty much a standard).
-spc (hmmm ... )
11:14 a.m.
New subject: Charset/CCDD (was: Let's develop an open-source media archive
Sean 'Captain Napalm' Conner wrote:
It was thus said that the Great Hans Franke once
stated:
What about a boot-strapable subset and revised error handeling?
Ben.
Rather then restricting the encodeing of the XML
file to a
specific charset, we need to restrict the USAGE within the
standard to certain characters, regardless of the encodeing.
Unless otherwise noted, XML files are assumed to be encoded in UTF-8,
*but* an XML parser is required to abort at the first error in the XML file.
If a parser is reading an XML file without an explicit character set
encoding scheme (which means it's assuming UTF-8) and it reads a character
that is illegal (say the file was encoded in ISO-8859-3) it gives up
(usually with an "illegal character at such-n-such position" error).
13 Aug
13 Aug
6:05 a.m.
New subject: Charset/CCDD (was: Let's develop an open-source media archive
Am 12 Aug 2004 16:10 meinte Sean 'Captain Napalm' Conner:
It was thus said that the Great Hans Franke once
stated:
> Rather then restricting the encodeing of the XML file to a
> specific charset, we need to restrict the USAGE within the
> standard to certain characters, regardless of the encodeing.
Unless otherwise noted, XML files are assumed to be
encoded in UTF-8,
Except that this is not realy followed by any parser - all I
found assume ISO 8859-1 or some other stuff, but produce UTF-8.
*but* an XML parser is required to abort at the first
error in the XML file.
If a parser is reading an XML file without an explicit character set
encoding scheme (which means it's assuming UTF-8) and it reads a character
that is illegal (say the file was encoded in ISO-8859-3) it gives up
(usually with an "illegal character at such-n-such position" error).
Not necersarry, if the parser accepts 8 Bit input, it just takes
the charactes as is - a X'C3' ist just seen as that, no matter if
it's 8859-1 or -3 or whatever. Now if he expects UTF-8, then still
X'C3' is a valid first character of a two character sequence. Only
if the next doesn't start with B'10' (read X'80'..X'BF'), an
error
will occure. Quite likely if you look at some langiuage encodeings.
I had whole 8 Bit russian files sliping thru UTF-8 parsers without
illegal character errors.
Right now, this is a real problem with XML
deployment (it gets even
wierder when XML files are transported via HTTP but I'm getting ahead of
myself) so when I suggested that (if we are using XML) that each *must*
start with:
<?xml version="1.0"
encoding="US-ASCII"?>
It was a way of self-defense. Perhaps it can be
relaxed some and require:
<?xml version="1.0" encoding="some
XML defined character encoding scheme"?>
and if the encoding scheme isn't defined, it's
an error and further
processing of the archive should stop.
Basicly I fully support your motion here ... if we would live in
a binary world. A world where for one thing any Parser can parse
any character set, and where all files are always transported in
binary mode. But we don't. If I generate a file on a EBCDIC system,
and transfer it to a Unix system, the file transfer will convert
the characterset from the EBCDIC variation I use to the ASCII
variant on the target system - if I included an 'encodeing="EBCDIC"'
in the ?XML line, it suddenly would be 100% wrong. Now what?
Includeing the encodeing _within_ the file as part of the data
is in my opinion the only real flaw in the basic XML definition
(if we ignore all that rubish of namespaces etc.). The carset
of a text file is always a meta information which should never
(only) be part of the application specific data within the file.
The whole XML part works only by assumeing that the encodeing
is at least done with something that is compatible to the systems
default code, otherwise the parser couldn't even find the
<?xml line.
> I suggest to restrict the caracters used in tags,
attribute
> names and attributes to 'A-Z' (uppercase), '0-9' and '-'.
Unfortunately, XML is defined with lowercase (or it
may be case
sensitive---I do know that all XML I've seen is with lowercase tags, and
it's pretty much a standard).
No, XML is case sensitive, not lower case or whatever. In fact,
you can use all characters you may find to define tags or atributes.
One could for example use German Umlauts within tags, or even so
all tags in Kanji or arabic. So for our definition we can prety
much stay with all upercase tags, attribute names and attributes.
I strongly recommend this to allow the usage of our format on
systems/charsets that offer only uppercase latin letters. This
is not only true for old Russian or Japaneese 8 Bit systems,
even the good old Apple ][+ had only uppercase !
Beside that, uppercase is, to me, way more readable as keywords.
decideing for lowercase was one of the many bad ideas in XHTML.
Oh, and at least that part the XML people did get right, since
it is defined, that the leading ?XML may be upper or lower case.
Gruss
H.
--
VCF Europa 6.0 am 30.April und 01.Mai 2005 in Muenchen
http://www.vcfe.org/
12 Aug
12 Aug
1:01 p.m.
New subject: Charset/CCDD (was: Let's develop an open-source media archive standard)
On Thu, 12 Aug 2004, Hans Franke wrote:
I suggest to restrict the caracters used in tags,
attribute
names and attributes to 'A-Z' (uppercase), '0-9' and '-'.
these are to be found in all character sets I've seen up to
date, and thus able to tunnel whatever needed. For various
data encodeing schemes additional characters may be needed.
Agreed.
--
Sellam Ismail Vintage Computer Festival
------------------------------------------------------------------------------
International Man of Intrigue and Danger http://www.vintage.org
[ Old computing resources for business || Buy/Sell/Trade Vintage Computers ]
[ and academia at www.VintageTech.com || at http://marketplace.vintage.org ]
7265
days inactive
7267
days old
10 comments
7 participants
participants (7)
-
bfranchuk＠jetnet.ab.ca -
Hans.Franke＠siemens.com -
jbmcb＠hotmail.com -
julesrichardsonuk＠yahoo.co.uk -
RCini＠congressfinancial.com -
spc＠conman.org -
vcf＠siconic.com