Text encoding Babel. Was Re: George Keremedjiev

At 10:52 PM 25/11/2018 -0700, you wrote: I was speaking poetically. Perhaps "the mail software he uses was written by morons" is clearer. Oh yes, tell me about the html 'there is no such thing as hard formatting and you can't have any even when you want it' concept. Thank you Tim Berners Lee. http://everist.org/NobLog/20130904_Retarded_ideas_in_comp_sci.htm http://everist.org/NobLog/20140427_p-term_is_retarded.htm Except that 'non-breaking space' is mostly about inhibiting line wrap at that word gap. But anyway, there's little point trying to psychoanalyze the writers of that software. Probably involved pointy-headed bosses. Of course not. It was for American English only. This is one of the major points of failure in the history of information processing. Well now he's flaunting it in his latest posts. Never mind. :) Containing extended Unicode character sets via UTF-8, doesn't make it a non-hard-formatted medium. In ASCII a space is a space, and multi-spaces DON'T collapse. White space collapse is a feature of html, and whether an email is html or not is determined by the sending utility. As you see, this IS NOT HTML, since those extra spaces and your diagram below would have collapsed if it was html. Also saving it as text and opening in a plain text ed or hex editor absolutely reveals what it is. Hmm... the problem is it's intended to be serious, but is still far from exposure-ready. So if I talk about it now, I risk having specific terms I've coined in the doco (including the project name) getting meme-jammed or trademarked by others. The plan is to release it all in one go, eventually. Definitely will be years before that happens, if ever. However, here's a cut-n-paste (in plain text) of a section of the Introduction (html with diags.) ---------- Almost always, a first attempt at some unfamiliar, complex task produces a less than optimal result. Only with the knowledge gained from actually doing a new thing, can one look back and see the mistakes made. It usually takes at least one more cycle of doing it over from scratch to produce something that is optimal for the needs of the situation. Sometimes, especially where deep and subtle conceptual innovations are involved, it takes many iterations. Human development of computing science (including information coding schemes) has been effectively a 'first time effort', since we kept on developing new stuff built on top of earlier work. We almost never went back to the roots and rebuilt everything, applying insights gained from the many mistakes made. In reviewing the evolution of information coding schemes since very early stages such as the Morse code, telegraph signal recording, typewriters, etc, through early computing systems, mass data storage and file systems, computer languages from Assembler through Compilers and Interpreters, and so on, several points can be identified at which early (inadequate) concepts became embedded then used as foundations for further developments. This made the original concepts seem like fundamentals, difficult to question (because they are underlying principles for much more complex later work), and virtually impossible to alter (due to the vast amounts of code dependent on them.) And yet, when viewed in hindsight many of the early concepts are seriously flawed. They effectively hobbled all later work dependent on them. Examples of these pivotal conceptual errors: Defects in the ASCII code table. This was a great improvement at the time, but fails to implement several utterly essential concepts. The lack of these concepts in the character coding scheme underlying virtually all information processing since the 1960s, was unfortunate. Just one (of many) bad consequences has been the proliferation of 'patch-up' text coding schemes such as proprietry document formats (MS Word for eg), postscript, pdf, html (and its even more nutty academia-gone-mad variants like XML), UTF-8, unicode and so on. [pic] This is a scan from the 'Recommended USA Standard Code for Information Interchange (USASCII) X3.4 - 1967' The Hex A-F on rows 10-15, added here. Hexadecimal notation was not commonly in use in the 1960s. Fig. ___ The original ASCII definition table. ASCII's limitations were so severe that even the text (ie ASCII) program code source files used by programmers to develop literally everything else in computing science, had major shortcomings and inconveniences. A few specific examples of ASCII's flaws: Missing concept of control vs data channel separation. And so we needed the "< >" syntax of html, etc. Inability to embed meta-data about the text in standard programatically accessible form. Absense of anything related to text adornments, ie italics, underline and bold. The most basic essentials of expressive text, completely ignored. Absense of any provision for creative typography. No awareness of fonts, type sizes, kerning, etc. Lack of logical 'new line', 'new paragraph' and 'new page' codes. Inadequate support of basic formatting elements such as tabular columns, text blocks, etc. Even the extremely fundamental and essential concept of 'tab columns' is impropperly implemented in ASCII, hence almost completely dysfunctional. No concept of general extensible-typed functional blocks within text, with the necessary opening and closing delimiters. Missing symmetry of quote characters. (A consequence of the absense of typed functional blocks.) No provision for code commenting. Hence the gaggle of comment delimiting styles in every coding language since. (Another consequence of the absense of typed functional blocks.) No awareness of programatic operations such as Inclusion, Variable substitution, Macros, Indirection, Introspection, Linking, Selection, etc. No facility for embedding of multi-byte character and binary code sequences. Missing an informational equivalent to the pure 'zero' symbol of number systems. A specific "There is no information here" symbol. (The NUL symbol has other meanings.) This lack has very profound implications. No facility to embed multiple data object types within text streams. No facility to correlate coded text elements to associated visual typographical elements within digital images, AV files, and other representational constructs. This has crippled efforts to digitize the cultural heritage of humankind. Non-configurable geometry of text flow, when representing the text in 2D planes. (Or 3D space for that matter.) Many of the 32 'control codes' (characters 0x00 to 0x1F) were allocated to hardware-specific uses that have since become obsolete and fallen into disuse. Leaving those codes as a wasted resource. ASCII defined only a 7-bit (128 codes) space, rather than the full 8-bit (256 codes) space available with byte sized architectures. This left the 'upper' 128 code page open to multiple chaotic, conflicting usage interpretations. For example the IBM PC code page symbol sets (multiple languages and graphics symbols, in pre-Unicode days) and the UTF-8 character bit-size extensions. Inability to create files which encapsulate the entirety of the visual appearance of the physical object or text which the file represents, without dependence on any external information. Even plain ASCII text files depend on the external definition of the character glyphs that the character codes represent. This can be a problem if files are intended to serve as long term historical records, potentially for geological timescales. This problem became much worse with the advent of the vast Unicode glyph set, and typset formats such as PDF. The PDF 'archival' format (in which all referenced fonts must be defined in the file) is a step in the right direction ? except that format standard is still proprietary and not available for free. ---------- Sorry to be a tease. Soon I'd like to have a discussion about the functional evolution of the various ASCII control codes, and how they are used (or disused) now. But am a bit too busy atm to give it adequate attention. Guy