31 Dec
2018
31 Dec
'18
4:20 a.m.
I used the libtiff-tools (Debian 8.x - 32 Bit) to extract all 61 .TIF's
from the
Multipage .tif file. While the
.tif's look descent, and RasterVect shows
the
.tif properties to be Group 4 Fax (1bpp) with 5100 x 6600 pixels - 300 DPI,
I can't get tesseract 3.x, TextBridge Classic 2.0, or Irfanview with KADMOS
Plugin to OCR any of the .tif files, with descent results. I'd expect an
OCR
of 85 to 90 % correct conversion to ASCII text.
Typically, one of the three above Software packages will do a descent job
of OCRing .tif's of such scans. (Most PDF's end up at 72 x 72 DPI, and
converting them to 300 DPI, allows them to be properly OCR'd.)
If anyone else has had better luck, I'd like to know what your process is.
Thanks.
Larry
31 Dec
31 Dec
6:35 a.m.
On 2018-12-31 7:20 AM, Larry Kraemer via cctalk wrote:
I used the libtiff-tools (Debian 8.x - 32 Bit) to
extract all 61 .TIF's
from the
Multipage .tif file. While the .tif's look descent, and RasterVect shows
the
.tif properties to be Group 4 Fax (1bpp) with 5100 x 6600 pixels - 300 DPI,
I can't get tesseract 3.x, TextBridge Classic 2.0, or Irfanview with KADMOS
Plugin to OCR any of the .tif files, with descent results. I'd expect an
OCR
of 85 to 90 % correct conversion to ASCII text.
Typically, one of the three above Software packages will do a descent job
of OCRing .tif's of such scans. (Most PDF's end up at 72 x 72 DPI, and
converting them to 300 DPI, allows them to be properly OCR'd.)
If anyone else has had better luck, I'd like to know what your process is.
I don't know if OCR software is sensitive to having correct resolution
(I've practically zero experience with it), but 300 dpi seems wrong for
Mattis' scans.
Seems they should be 600 dpi (21.7 cm x 28 cm).
--Toby
Thanks.
Larry
9:46 a.m.
On Mon, 31 Dec 2018, Larry Kraemer via cctalk wrote:
I used the libtiff-tools (Debian 8.x - 32 Bit) to
extract all 61 .TIF's
from the Multipage .tif file. While the .tif's look descent, and
RasterVect shows the .tif properties to be Group 4 Fax (1bpp) with 5100
x 6600 pixels - 300 DPI, I can't get tesseract 3.x, TextBridge Classic
2.0, or Irfanview with KADMOS Plugin to OCR any of the .tif files, with
descent results. I'd expect an OCR of 85 to 90 % correct conversion to
ASCII text.
Software listings need more accuraacy than that.
How many wrong characters does it take for a program not to work?
"desCent" isn't good enough.
85 to 90 % correct is a character wrong in every 6 to 10 characters.
How many errors is that PER LINE?
"But, you can start with that, and just fix the errors, without retyping
the rest." Doing it that way is a desCent into madness.
BTDT. wore out the T-shirts.
A competent typist can retype the whole thing faster than fixing an error
in every six to ten characters.
Only if there is less than one error for every several hundred characters
does "patching it" save time for a competent typist.
In general, for a competent typist, the fastest way to reposition the
cursor to the next error in the line is to simply hit the keys of the
intervening letters.
It is NOT to move the cursor with the mouse, then put your hand back on
the keys to type a character.
Using cursor motion keys is no faster for a competent typist than hitting
the keys of the letters toskip over.
TIP: display the OCR'ed text that is to be corrected in a font that
exaggerates the difference between zero and the letter 'O', and between
one and lower case 'l'. There are some programs that will attempt to
select those based on context.
--
Grumpy Ol' Fred cisin at xenosoft.com
4:13 p.m.
Fred is right, OCR is only worth it if the document is in perfect condition. I just finish
getting an old 4004 listing working. I made only two mistakes on the 4K of code that were
not the fault of the poorness of the listing. Twice I put LDM instead of LD. LDM was the
most commonly used.
There were still some 15 or so other errors do to the printing. It looked to be done on a
ASR33 with poor registration of the print drum. Cs and 0s were often missing the right
1/3. Expecting an OCR to do much would have been a folly. Even though some 85% to 90%
could be read properly. It took be about 3 weeks of evenings to make heads or tails of the
code. I've finally got it running correctly.
If it had been done with an OCR, many cases it would have simply put a C instead of a 0.
I'd have had to go through the listing, checking each C to make sure it was right. It
is easier in many cases to have analysed what I could see and make a judgement, based on
what I could see and the general context as I was typing it in.
Dwight
________________________________
From: cctalk <cctalk-bounces at classiccmp.org> on behalf of Fred Cisin via cctalk
<cctalk at classiccmp.org>
Sent: Monday, December 31, 2018 9:46 AM
To: General Discussion: On-Topic and Off-Topic Posts
Subject: Re: OCR old software listing
On Mon, 31 Dec 2018, Larry Kraemer via cctalk wrote:
I used the libtiff-tools (Debian 8.x - 32 Bit) to
extract all 61 .TIF's
from the Multipage .tif file. While the .tif's look descent, and
RasterVect shows the .tif properties to be Group 4 Fax (1bpp) with 5100
x 6600 pixels - 300 DPI, I can't get tesseract 3.x, TextBridge Classic
2.0, or Irfanview with KADMOS Plugin to OCR any of the .tif files, with
descent results. I'd expect an OCR of 85 to 90 % correct conversion to
ASCII text.
Software listings need more accuraacy than that.
How many wrong characters does it take for a program not to work?
"desCent" isn't good enough.
85 to 90 % correct is a character wrong in every 6 to 10 characters.
How many errors is that PER LINE?
"But, you can start with that, and just fix the errors, without retyping
the rest." Doing it that way is a desCent into madness.
BTDT. wore out the T-shirts.
A competent typist can retype the whole thing faster than fixing an error
in every six to ten characters.
Only if there is less than one error for every several hundred characters
does "patching it" save time for a competent typist.
In general, for a competent typist, the fastest way to reposition the
cursor to the next error in the line is to simply hit the keys of the
intervening letters.
It is NOT to move the cursor with the mouse, then put your hand back on
the keys to type a character.
Using cursor motion keys is no faster for a competent typist than hitting
the keys of the letters toskip over.
TIP: display the OCR'ed text that is to be corrected in a font that
exaggerates the difference between zero and the letter 'O', and between
one and lower case 'l'. There are some programs that will attempt to
select those based on context.
--
Grumpy Ol' Fred cisin at xenosoft.com
5:18 p.m.
On Dec 31, 2018, at 7:13 PM, dwight via cctalk
<cctalk at classiccmp.org> wrote:
Fred is right, OCR is only worth it if the document is in perfect condition. I just
finish getting an old 4004 listing working. I made only two mistakes on the 4K of code
that were not the fault of the poorness of the listing. Twice I put LDM instead of LD. LDM
was the most commonly used.
I wouldn't put it quite so strongly. OCR even if not perfect can help a lot. You can
often OCR + test assembly + proofread faster than retyping, especially since that requires
fixing typos and proofreading also. Many OCR errors are caught by the assembler, though
not all of them of course. I've done both in an ongoing software preservation
project; my conclusion still is to use OCR when it works "well enough". A
couple of errors per page is definitely "well enough".
The program used matters. I looked at Tesseract a bit but its quality was vastly inferior
to commercial products in the examples I tried. I now use Abbyy FineReader, which handles
a lot of line printer and typewriter material quite well.
paul
5:41 p.m.
I've had a lot of success using Adobe's Clearscan for OCR'ing old stuff.
Admittedly it's not perfect but it can improve the quality of an old
document a lot.
Kevin Parker
0418 815 527
-----Original Message-----
From: cctalk <cctalk-bounces at classiccmp.org> On Behalf Of Paul Koning via
cctalk
Sent: Tuesday, 1 January 2019 12:18 PM
To: dwight <dkelvey at hotmail.com>; General Discussion: On-Topic and Off-Topic
Posts <cctalk at classiccmp.org>
Subject: Re: OCR old software listing
On Dec 31, 2018, at 7:13 PM, dwight via cctalk
<cctalk at classiccmp.org>
wrote:
Fred is right, OCR is only worth it if the document is in perfect
condition. I just
finish getting an old 4004 listing working. I made only
two mistakes on the 4K of code that were not the fault of the poorness of
the listing. Twice I put LDM instead of LD. LDM was the most commonly used.
I wouldn't put it quite so strongly. OCR even if not perfect can help a
lot. You can often OCR + test assembly + proofread faster than retyping,
especially since that requires fixing typos and proofreading also. Many OCR
errors are caught by the assembler, though not all of them of course. I've
done both in an ongoing software preservation project; my conclusion still
is to use OCR when it works "well enough". A couple of errors per page is
definitely "well enough".
The program used matters. I looked at Tesseract a bit but its quality was
vastly inferior to commercial products in the examples I tried. I now use
Abbyy FineReader, which handles a lot of line printer and typewriter
material quite well.
paul
5:33 p.m.
On the other hand, just for the FUN of it,
can you write some software to find and fix (or simply flag) the most
common errors?
When I had to terminate my publisher, I was s'posed to receive a copy of
their customer database.
They deleted all delimiters (spaces, commas, periods, and other
punctuation), and then printed it out on greenbar with a font that used
the same character for zero and letter 'O'; and same character for
one, lower case 'l', and upper cse 'I'. Surprisingly, they did NOT use a
bad ribbon and printhead!
An acquaintance OCR'ed it. They were able to get what they thought was "80
to 90%" from the originals, but not from a xerox copy that visually
seemed to be just as good.
I spent a little time writing some simple code to parse and fix most of
it.
Mostly simple context, such as a zero between two letters is likely an
'O', or an 'O' between two numerals is likely a zero. Similarly with one,
lower case 'L' and upper case 'I'. Some OCR software now pays attention
to context.
Five consecutive numerals following two capital letters is likely to be a
zip code, and end of the record. USUALLY. Comparison of those digits
with the two letters in a zipcode database provided partial confirmation.
. . . and so forth . . .
Not a practical use of time, but a fun exercise in parsing.
Another time, the .SRT file that I found for "Company Man" used upper case
'I' instead of lower case 'L'! (AND had a three minute offset for the
start time) Did not take very long to fix.
--
Grumpy Ol' Fred cisin at xenosoft.com
On Tue, 1 Jan 2019, dwight wrote:
Fred is right, OCR is only worth it if the document is
in perfect condition. I just finish getting an old 4004 listing working. I made only two
mistakes on the 4K of code that were not the fault of the poorness of the listing. Twice I
put LDM instead of LD. LDM was the most commonly used.
There were still some 15 or so other errors do to the printing. It looked to be done on a
ASR33 with poor registration of the print drum. Cs and 0s were often missing the right
1/3. Expecting an OCR to do much would have been a folly. Even though some 85% to 90%
could be read properly. It took be about 3 weeks of evenings to make heads or tails of the
code. I've finally got it running correctly.
If it had been done with an OCR, many cases it would have simply put a C instead of a 0.
I'd have had to go through the listing, checking each C to make sure it was right. It
is easier in many cases to have analysed what I could see and make a judgement, based on
what I could see and the general context as I was typing it in.
Dwight
________________________________
From: cctalk <cctalk-bounces at classiccmp.org> on behalf of Fred Cisin via cctalk
<cctalk at classiccmp.org>
Sent: Monday, December 31, 2018 9:46 AM
To: General Discussion: On-Topic and Off-Topic Posts
Subject: Re: OCR old software listing
On Mon, 31 Dec 2018, Larry Kraemer via cctalk wrote:
I used the libtiff-tools (Debian 8.x - 32 Bit) to
extract all 61 .TIF's
from the Multipage .tif file. While the .tif's look descent, and
RasterVect shows the .tif properties to be Group 4 Fax (1bpp) with 5100
x 6600 pixels - 300 DPI, I can't get tesseract 3.x, TextBridge Classic
2.0, or Irfanview with KADMOS Plugin to OCR any of the .tif files, with
descent results. I'd expect an OCR of 85 to 90 % correct conversion to
ASCII text.
Software listings need more accuraacy than that.
How many wrong characters does it take for a program not to work?
"desCent" isn't good enough.
85 to 90 % correct is a character wrong in every 6 to 10 characters.
How many errors is that PER LINE?
"But, you can start with that, and just fix the errors, without retyping
the rest." Doing it that way is a desCent into madness.
BTDT. wore out the T-shirts.
A competent typist can retype the whole thing faster than fixing an error
in every six to ten characters.
Only if there is less than one error for every several hundred characters
does "patching it" save time for a competent typist.
In general, for a competent typist, the fastest way to reposition the
cursor to the next error in the line is to simply hit the keys of the
intervening letters.
It is NOT to move the cursor with the mouse, then put your hand back on
the keys to type a character.
Using cursor motion keys is no faster for a competent typist than hitting
the keys of the letters toskip over.
TIP: display the OCR'ed text that is to be corrected in a font that
exaggerates the difference between zero and the letter 'O', and between
one and lower case 'l'. There are some programs that will attempt to
select those based on context.
--
Grumpy Ol' Fred cisin at xenosoft.com
1951
days inactive
1952
days old
6 comments
6 participants
participants (6)
-
cisin＠xenosoft.com -
dkelvey＠hotmail.com -
ldkraemer＠gmail.com -
paulkoning＠comcast.net -
toby＠telegraphics.com.au -
trash80＠internode.on.net