On Thu, Sep 24, 2015 at 05:35:54AM -0700, Al Kossow wrote: I think the oldest I have seen from Sun is from 1987 or 88. Is that old enough? /P

There seems to be some interest in filling in the different versions of these SUN computer manuals. I though that perhaps there might also be some interest in the DOC set of manuals for various versions of RT-11 and TSX-Plus. While I understand that a number of individuals would appreciate their own dead tree versions, before they arrive at that good home of an interested individual, would there be any interest in adding them the bitsavers archive? From memory, I suspect that there is: RT-11: (a) Various pieces of V2 and V3, most of the pages from V04.00 (b) Probably complete DOC sets for V05.02, V05.04G, V05.05 TSX-Plus: (a) V6.31 in a binder (b) V6.5 as ONLY loose sheets There has already been interest expressed in having the dead tree version of the V05.05 RT-11 DOC set, but I just wanted to be sure that other individuals who may be interested in having PDF files for V05.05 and older versions of RT-11 know about the possibility. As for TSX-Plus, I don't see a PDF of V6.5 being available. I would like to keep the dead tree version of the V6.5 TSX-Plus DOC set manuals available for myself until I no longer need them, but perhaps arrangements might be made to have them scanned into PDF files and returned. I have other RT-11 manuals as well and some PDP-11 hardware manuals, but probably most are already in PDF files at bitsavers. Jerome Fine

On 2015-09-25 00:04, Al Kossow wrote: Several RSX manuals have important color coding. Some manuals have both black, red and blue text. Others have sections with a light red or gray background. I have paper versions of them. Unfortunately, all scanned manuals I've seen have been plain b/w. Also, bitmaps, and not OCRed. :-( Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt at softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol

On 2015-09-25 16:15, Paul Koning wrote: Agreed that it always needs checking/cleanup. But I seriously would like to OCR the manuals, instead of dealing bit bitmaps. Johnny

On 2015-09-25 22:35, Al Kossow wrote: All of them. Aha. I wasn't aware of that. I've downloaded copies many years ago that I've been keeping locally. I'll check out the current versions on bitsavers then. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt at softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol

On 2015-09-26 12:16, Johnny Billquist wrote: Al, exactly how have they been OCRed? Looking at them, it would appear that what you see is still the bitmaps of all the pages, but then you have the basic text also available for selection/searching. My issue with that is that the documents are huge, and the experience just scrolling through them is pretty bad. Sadly I don't even remember what software I used for OCR about 10 years ago, but I had something for Windows back then, which actually figured out fonts and all, and created a plain Word document from the OCR process. That was a really nice piece of software, which preserved formatting, fonts and all. I have a short example of the results at http://www.update.uu.se/~bqt/Clarkson.pdf, which was just a scan of two pages from a book. I created the pdf from Word. A process like that is what I'd like, except for figures, which needs to be kept as bitmaps, I suspect. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt at softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol

On 2015-09-26 23:35, Al Kossow wrote: Understandable. But you'll have to live with me wishing (and complaining Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt at softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol

On 2015-09-26 23:42, Toby Thain wrote: Look at the results in the link I posted. I was more than happy with that result. But sure, for those who like bitmaps, I'm certainly not going to take them away. But for me, I'm actually interested in the content, and not the pixels. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt at softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol

On 2015-09-27 03:41, Toby Thain wrote: I suppose it is the eternal argument between preservation and use. I use these documents every day. I don't care about the pixels, but the content. Museums and the like are obviously more interested in the preservation. I get the feeling you didn't actually check the text that I OCRed from a book. That text is an example what I'm looking for. I will not prevent people who want pixel preservation from continuing to have that. But for me, it is a problem. The experience in actually using these documents are pretty poor. And, as have also been noted, information have been lost in these scans, as they have not preserved color codings. Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt at softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol

On 2015-09-27 15:38, Dave G4UGM wrote: That's because it looked exactly that way in the original as well. Blame the author/typesetter. Like I said, I don't even remember what software I used. :-( I got it with a scanner I bought in the 90s, and running on Windows ME. Otherwise I would love to apply it to something bigger/more technical. Errors are always bad. Agreed. That is not something we're discussing here. I don't have problems reading the current scans, as such. But when having ten of these open at the same time, and scrolling through them, it becomes obvious that the bitmaps are heavy. It can take a while for the screen to be updated. Not to mention the problems you sometimes hits with searching... Johnny -- Johnny Billquist || "I'm on a bus || on a psychedelic trip email: bqt at softjar.se || Reading murder books pdp is alive! || tryin' to stay hip" - B. Idol

On Sun, Sep 27, 2015 at 04:08:07PM +0200, Johnny Billquist wrote: It seems to me that a better tool could solve the issue. One that could display the OCR:ed content only and the scanned content only when desired, for instance when you suspect an error. Is there such a reader? Is the content organised to make it possible. /P

On 2015-09-27 12:22 PM, Pontus Pihlgren wrote: Right, if the bitmaps aren't available, then it's not an acceptable archive. Personally I never, ever, want to see the OCR'd version. But that may be coloured by a career as typographer and finished artist. No software can apply the judgment that humans did in the print edition; it's only more or less degrading steps from that point on. And to be clear I'm not at loggerheads with Johnny because I am indeed talking about acceptable archiving practice, not some conversion of a particular text which might be useful for a particular person on a particular day. --Toby

On 2015-09-27 2:33 PM, Fred Cisin wrote: DJVU compression is somewhat analogous to this process, because, font-like, it builds a set of master glyphs then uses them as a compression dictionary (if everyone will forgive my simplistic explanation). Being lossy, like OCR, it inherently adds the risk of picking the wrong (but visually almost indistinguishable) glyph -- the WORST kind of typo for being so insidious. There was a somewhat scary case study on the web a few years ago (not sure if it's still out there, haven't been able to find it) where the DJVU compression in a Xerox copier was quietly changing digits on scanned schematics to different digits. Close enough for DJVU -- but wrong. The risks are obvious(*). --Toby * - Hat tip to PGN. comp.risks digest.

True. But that's not the biggest problem with OCR. The biggest problem is that even professional grade OCR programs have rather low accuracy. Maybe they do acceptably well on really high grade scans of very clean new documents, but on books, typewritten documents, etc., even after you use the "train" feature you need to spend a long time cleaning up. It may be faster than retyping things, if you're lucky. Not if you're not; two of us recently retyped 300 pages of line printer listing because that was faster and more accurate than OCR on that particular printout. Given that OCR can only do, at best, a just barely acceptable recognition of the letters of the alphabet, it follows that accurately recognizing the actual font used will be vastly less accurate. And indeed you can see that clearly. I wonder if there are OCR programs that can be told to choose among 2 or 3 fonts, as opposed to guess from the entire inventory of the machine. If so, and if they are sufficiently distinct, then maybe you'd stand a chance. Especially if it also added heuristics like "never change fonts in mid-word" -- an obvious rule but not one I have seen implemented. paul

On Sun, 27 Sep 2015, Johnny Billquist wrote: While I have no way of knowing what you were using back in the day, something else to keep in mind about books of text V manuals, . . . with a book of text, there is significantly more context for every item. As a trivial example, in English language text, a 'Q' is virtually never followed by anything other than 'u', space, or punctuation. Therefore, if there is a letter following a 'Q', it can be assumed to be 'u' unless proven otherwise. Not so for part numbers, variable names, etc. Applying "spell-checking" to a document gives a very high initial set of probabilities for letters that might otherwise be unclear. If a given font has a very stylistic 'e', then its likelihood can be checked just with letter frequency, and if extremely common and surrounded by other letters, it is quite unlikely to be a slashed '0'. In general, '0', 'O', '1', 'l', 'I' can generally be differentiated by context, such as whether surrounded by numbers or letters, but NOT as reliably based on shape, or even pixel matching. Therefore, some OCR programs that make use of some of those kinds of techniques might do great on text, but be bordering on unusable for tech documents. My idea was to make human assisted OCR, by displaying the OCR in progress, with color coding of characters based on their probability of accuracy. Then, cheap labor could manually enter characters, starting with those that had lowest probability of accuracy. Minor heuristic algorithms could then use the incoming data of additional character/pixel pattern pairs to improve the guesses of subsequent characters. The cumulative data pairs would learn additional fonts. The cheap labor could be neighborhood kids, off-shore out-sourcing, or even grad students, depending on how much you care about their quality of life and cost of living. For premium quality, use workers who even have some knowledge of the material. BUT, one must never pick a worker who was brought up to interchange '0' and 'O', '1' and 'l', etc. (Remember when some typewriters didn't HAVE both characters?)

On 9/24/15 11:16 PM, Pontus Pihlgren wrote: Sounds good. Shipping would be to Mountain View, CA 94043 Email me to work out the details.