26 Dec
2018
26 Dec
'18
1:29 p.m.
Finally I got hold of the sources for the PDP-11 SPACE WAR that was
submitted to DECUS by Bill Seiler.
The format is scans of the PAL-11S listing output. It is easy to crop the
image to only contain actual source. Then running OCR on it. Tried a few
online versions and tesseract.
The problem is that the paper that the listing is printed on has lines.
Very black lines. It makes the OCR go completely crazy. Source lines
without black lines OCR ok. The others do not. The files need massive
amount of manual intervention.
Does anyone have an idea how to process files like this?
A good way to remove the black lines?
There are only 19 source files with three or four pages each so I don't
think it makes sense to try to train tesseract to do it (training tesseract
seems to be a huge undertaking).
https://i.imgur.com/dvY973s.png
/Mattis
26 Dec
26 Dec
2 p.m.
On December 26, 2018 at 4:29 PM Mattis Lind via cctech
<cctech at classiccmp.org> wrote:
Finally I got hold of the sources for the PDP-11 SPACE WAR that was
submitted to DECUS by Bill Seiler.
The format is scans of the PAL-11S listing output. It is easy to crop the
image to only contain actual source. Then running OCR on it. Tried a few
online versions and tesseract.
The problem is that the paper that the listing is printed on has lines.
Very black lines. It makes the OCR go completely crazy. Source lines
without black lines OCR ok. The others do not. The files need massive
amount of manual intervention.
Does anyone have an idea how to process files like this?
A good way to remove the black lines?
There are only 19 source files with three or four pages each so I don't
think it makes sense to try to train tesseract to do it (training tesseract
seems to be a huge undertaking).
https://i.imgur.com/dvY973s.png
/Mattis
One thing you might try is to pull the scan images into matlab/gnu octave
and do a 2d FFT, remove the frequency band of the lines, inverse fft, and save. I've
had good luck removing regular patterns of noise from images that way.
Will
"He may look dumb but that's just a disguise." -- Charlie Daniels
"The names of global variables should start with // " -- https://isocpp.org
2:15 p.m.
On 2018-12-26 4:29 PM, Mattis Lind via cctalk wrote:
Finally I got hold of the sources for the PDP-11 SPACE
WAR that was
submitted to DECUS by Bill Seiler.
The format is scans of the PAL-11S listing output. It is easy to crop the
image to only contain actual source. Then running OCR on it. Tried a few
online versions and tesseract.
The problem is that the paper that the listing is printed on has lines.
Very black lines. It makes the OCR go completely crazy. Source lines
without black lines OCR ok. The others do not. The files need massive
amount of manual intervention.
Does anyone have an idea how to process files like this?
A good way to remove the black lines?
Hi Mattis
I have some ideas. Can you give me access to the original scans?
--Toby
There are only 19 source files with three or four
pages each so I don't
think it makes sense to try to train tesseract to do it (training tesseract
seems to be a huge undertaking).
https://i.imgur.com/dvY973s.png
/Mattis
2:55 p.m.
Mattis said
Finally I got hold of the sources for the PDP-11 SPACE
WAR that was
submitted to DECUS by Bill Seiler.
The format is scans of the PAL-11S listing output. It is easy to crop the
image to only contain actual source. Then running OCR on it. Tried a few
online versions and tesseract.
The problem is that the paper that the listing is printed on has lines.
Very black lines. It makes the OCR go completely crazy. Source lines
without black lines OCR ok. The others do not. The files need massive
amount of manual intervention.
Does anyone have an idea how to process files like this?
A good way to remove the black lines?
There are only 19 source files with three or four pages each so I don't
think it makes sense to try to train tesseract to do it (training tesseract
seems to be a huge undertaking).
https://i.imgur.com/dvY973s.png
/Mattis
Scan them all as-is, put them up and 'crowd source' this list to divvy up a bunch
of pages per person
who could then use their favourite graphics program to paint out the lines and send you
back the
cleaned-up images? I could do a few.
Steve.
4:15 p.m.
On 12/26/18 3:17 PM, Al Kossow via cctalk wrote:
On 12/26/18 2:55 PM, Steve Malikoff via cctalk wrote:
I've found that it's often the best course of action and consumes the
least time overall. You also have a better chance of understanding the
code.
--Chuck
Scan them all as-is, put them up and 'crowd
source' this list
And TYPE the programs in again 
4:37 p.m.
On Wed, Dec 26, 2018 at 6:15 PM Chuck Guzis via cctalk <
cctalk at classiccmp.org> wrote:
On 12/26/18 3:17 PM, Al Kossow via cctalk wrote:
I'm doing this right now for DECUS 8-152. Yes, it certainly helps you
understand what the code is actually doing.
Kyle
And TYPE the programs in again
I've found that it's often the best course of action and consumes the
least time overall. You also have a better chance of understanding the
code.
6:19 p.m.
On Wed, Dec 26, 2018, 17:15 Chuck Guzis via cctalk <cctalk at classiccmp.org>
wrote:
On 12/26/18 3:17 PM, Al Kossow via cctalk wrote:
I typed in 710 pages of listing of the main processor code of HP
2000C'/2000F Time-Shared BASIC. I typed in the full listing format,
including the line numbers, addresses, and object code. I used a simple awk
script to strip it back down to source form. I wrote an HP 21xx
cross-assembler in Perl, assembled the sources, and diffed the generated
assembly listing against the typed-in listing, in order to find the
inevitable errors. I believe the results contain no errors introduced by me
other than typos in the comments.
This did take a fair bit of spare time.
On 12/26/18 2:55 PM, Steve Malikoff via cctalk
wrote:
I've found that it's often the best course of action and consumes the
least time overall. You also have a better chance of understanding the
code.
Scan them all as-is, put them up and 'crowd
source' this list
And TYPE the programs in again 
7:30 p.m.
On 12/26/2018 03:29 PM, Mattis Lind via cctalk wrote:
A good way to remove the black lines?
https://i.imgur.com/dvY973s.png
Oh, boy! The printer was not properly aligned, so the lines
actually overlay the dot-matrix printed text! This is going
to make OCR very difficult! I don't think you can just get
rid of the lines, that will drop dots from the characters,
too. A bad situation.
Jon
27 Dec
27 Dec
7:29 a.m.
On Dec 26, 2018, at 10:30 PM, Jon Elson via cctalk
<cctalk at classiccmp.org> wrote:
On 12/26/2018 03:29 PM, Mattis Lind via cctalk wrote:
At some point the simplest answer is to type it all in again. I've been doing work on
old software using old listings. Some are nice and clean and OCR just fine. Some are so
muddy that they are hard to read for humans, and utterly hopeless for OCR. It's no
fun to type in 300 pages of assembly code, but sometimes that's the only way.
paul
A good way to remove the black lines?
https://i.imgur.com/dvY973s.png
Oh, boy! The printer was not properly aligned, so the lines actually overlay the
dot-matrix printed text! This is going to make OCR very difficult! I don't think you
can just get rid of the lines, that will drop dots from the characters, too. A bad
situation.
28 Dec
28 Dec
9:47 p.m.
On 2018-12-26 4:29 PM, Mattis Lind via cctalk wrote:
Finally I got hold of the sources for the PDP-11 SPACE
WAR that was
submitted to DECUS by Bill Seiler.
The format is scans of the PAL-11S listing output. It is easy to crop the
image to only contain actual source. Then running OCR on it. Tried a few
online versions and tesseract.
The problem is that the paper that the listing is printed on has lines.
Very black lines. It makes the OCR go completely crazy. Source lines
without black lines OCR ok. The others do not. The files need massive
amount of manual intervention.
Does anyone have an idea how to process files like this?
A good way to remove the black lines?
Hi Mattis
Here's a first cut. Can probably be improved slightly. Let me know how
much this still confuses Tesseract.
https://docs.telegraphics.com.au/mattis/spcwar_pdp11_edit.tif
--Toby
There are only 19 source files with three or four pages each so I don't
think it makes sense to try to train tesseract to do it (training tesseract
seems to be a huge undertaking).
https://i.imgur.com/dvY973s.png
/Mattis
10:32 p.m.
On 2018-12-29 12:47 AM, Toby Thain via cctalk wrote:
On 2018-12-26 4:29 PM, Mattis Lind via cctalk wrote:
That is a multipage TIF, and the page order key is listed below.
I just noticed that a handful of pages seem to be missing, so I'll look
into that.
CHAR--0000
CHAR--0001
CHAR--0002
CHRTAB--0000
CHRTAB--0001
CHRTAB--0002
COMPAR--0000
COMPAR--0001
COMPAR--0002
COMPAR--0003
EXPLOD--0000
EXPLOD--0001
EXPLOD--0002
GRAVTY--0000
GRAVTY--0001
GRAVTY--0002
GRAVTY--0003
MULPLY--0000
MULPLY--0001
MULPLY--0002
PARM--0000
PARM--0001
PARM--0002
PARM--0003
PARM--0005
PARM--0006
PARM--0007
PARM--0008
PARM--0009
PWRUP--0000
PWRUP--0001
RESET--0000
RESET--0001
RKT1--0000
RKT1--0001
RKT2--0000
RKT2--0001
SCORE--0000
SCORE--0001
SINCOS--0000
SINCOS--0001
SINCOS--0002
SLINE--0000
SLINE--0001
SPCWAR--0000
SPCWAR--0001
SPCWAR--0002
SUN--0000
SUN--0001
SUN--0002
UPDAT1--0000
UPDAT1--0001
UPDAT1--0002
UPDAT2--0000
UPDAT2--0002
point--0000
point--0001
Finally I got hold of the sources for the PDP-11
SPACE WAR that was
submitted to DECUS by Bill Seiler.
The format is scans of the PAL-11S listing output. It is easy to crop the
image to only contain actual source. Then running OCR on it. Tried a few
online versions and tesseract.
The problem is that the paper that the listing is printed on has lines.
Very black lines. It makes the OCR go completely crazy. Source lines
without black lines OCR ok. The others do not. The files need massive
amount of manual intervention.
Does anyone have an idea how to process files like this?
A good way to remove the black lines?
Hi Mattis
Here's a first cut. Can probably be improved slightly. Let me know how
much this still confuses Tesseract.
https://docs.telegraphics.com.au/mattis/spcwar_pdp11_edit.tif
--Toby
There are only 19 source files with three or four pages each so I don't
think it makes sense to try to train tesseract to do it (training tesseract
seems to be a huge undertaking).
https://i.imgur.com/dvY973s.png
/Mattis
29 Dec
29 Dec
11:16 a.m.
On 2018-12-29 1:32 AM, Toby Thain via cctalk wrote:
On 2018-12-29 12:47 AM, Toby Thain via cctalk wrote:
Fixed that. I was also able to improve the quality. Same link.
The full page manifest is:
CHAR--0000
CHAR--0001
CHAR--0002
CHRTAB--0000
CHRTAB--0001
CHRTAB--0002
CHRTAB--0003
COMPAR--0000
COMPAR--0001
COMPAR--0002
COMPAR--0003
EXPLOD--0000
EXPLOD--0001
EXPLOD--0002
GRAVTY--0000
GRAVTY--0001
GRAVTY--0002
GRAVTY--0003
MULPLY--0000
MULPLY--0001
MULPLY--0002
PARM--0000
PARM--0001
PARM--0002
PARM--0003
PARM--0004
PARM--0005
PARM--0006
PARM--0007
PARM--0008
PARM--0009
PWRUP--0000
PWRUP--0001
RESET--0000
RESET--0001
RKT1--0000
RKT1--0001
RKT2--0000
RKT2--0001
SCORE--0000
SCORE--0001
SINCOS--0000
SINCOS--0001
SINCOS--0002
SINCOS--0003
SLINE--0000
SLINE--0001
SPCWAR--0000
SPCWAR--0001
SPCWAR--0002
SUN--0000
SUN--0001
SUN--0002
UPDAT1--0000
UPDAT1--0001
UPDAT1--0002
UPDAT2--0000
UPDAT2--0001
UPDAT2--0002
point--0000
point--0001
On 2018-12-26 4:29 PM, Mattis Lind via cctalk
wrote:
That is a multipage TIF, and the page order key is listed below.
I just noticed that a handful of pages seem to be missing, so I'll look
into that.
Finally I got hold of the sources for the PDP-11
SPACE WAR that was
submitted to DECUS by Bill Seiler.
The format is scans of the PAL-11S listing output. It is easy to crop the
image to only contain actual source. Then running OCR on it. Tried a few
online versions and tesseract.
The problem is that the paper that the listing is printed on has lines.
Very black lines. It makes the OCR go completely crazy. Source lines
without black lines OCR ok. The others do not. The files need massive
amount of manual intervention.
Does anyone have an idea how to process files like this?
A good way to remove the black lines?
Hi Mattis
Here's a first cut. Can probably be improved slightly. Let me know how
much this still confuses Tesseract.
https://docs.telegraphics.com.au/mattis/spcwar_pdp11_edit.tif
--Toby
There are only 19 source files with three or four pages each so I don't
think it makes sense to try to train tesseract to do it (training tesseract
seems to be a huge undertaking).
https://i.imgur.com/dvY973s.png
/Mattis
2184
days inactive
2187
days old
12 comments
10 participants
participants (10)
-
aek＠bitsavers.org -
cclist＠sydex.com -
elson＠pico-systems.com -
kylevowen＠gmail.com -
mattislind＠gmail.com -
paulkoning＠comcast.net -
spacewar＠gmail.com -
steven＠malikoff.com -
toby＠telegraphics.com.au -
wrcooke＠wrcooke.net