26 Jun
2004
26 Jun
'04
4:13 a.m.
I scan at 400dpi 1bpp and save the pages using group4 compressed TIFFs
which is the maximum res the IS520 scans at.
Acrobat does a really poor job of saving JPEGs (it makes them a LOT
bigger than the originals) and I would NOT use a lossy format on any
page with only text or line art. There is a noticable loss of edge
contrast on text pages compressed as JPEGs.
26 Jun
26 Jun
9:30 a.m.
On Saturday, June 26, 2004, Al Kossow, wrote:
I scan at 400dpi 1bpp and save the pages using group4
compressed TIFFs
which is the maximum res the IS520 scans at.
I was doing 150dpi greyscale jpegs. I've changed to 400dpi 1bpp tiffs,
and now I'm ending up with each page about 700K... (scan as greyscale
tiff (unknown compression), convert to 1-bit with lzw compression, save
as pdf in Photoshop).
Using tiffcp to change the compression from lzw to g4 more than doubles
the size to 1.6 MB. That's for a single page.
Surely this is too large?
Tom
Applefritter
www.applefritter.com
10:05 a.m.
Tom Owad wrote:
I was doing 150dpi greyscale jpegs. I've changed to 400dpi 1bpp tiffs,
and now I'm ending up with each page about 700K... (scan as greyscale
tiff (unknown compression), convert to 1-bit with lzw compression, save
as pdf in Photoshop).
You need to scan at 1bpp, not grayscale. When you are converting from
grayscale to 1bpp, you are probably getting dithering (a pseudo-random
scattering of black and white pixels) in order to approximate the
original grays at the edges of characters. This dithering will ruin the
achievable compression in G4, which is a fax algorithm optimised for
runs of white and black pixels.
12:15 p.m.
On Saturday, June 26, 2004, Paul Williams, wrote:
You need to scan at 1bpp, not grayscale. When you are
converting from
grayscale to 1bpp, you are probably getting dithering (a pseudo-random
scattering of black and white pixels) in order to approximate the
original grays at the edges of characters. This dithering will ruin the
achievable compression in G4, which is a fax algorithm optimised for
runs of white and black pixels.
Ah, ok - I was using dithering.
Now I've gotten the images down to 28 KB G4 tiffs, which become 40 KB pdfs.
Here's the first three pages, a 152 KB pdf file: <www.applefritter.com/
temp/sample.pdf>. Any reservations about the quality of the file, or
anything else, before I start doing a whole bunch more just like this?
Thanks!
Tom
Applefritter
www.applefritter.com
12:24 p.m.
Here's the first three pages, a 152 KB pdf file:
<www.applefritter.com/temp/sample.pdf>. Any reservations about the
quality of the file, or anything else, before I start doing a whole
bunch more just like this?
Well, I'd rather have the scans, especially in greyscale form. (If you
don't want to take the space, I'll happily host them myself, for the
sake of preserving the information. For archival purposes like this, I
almost always come down on the side of preserving information.)
/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML mouse(a)rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
2:42 p.m.
www.applefritter.com/temp/sample.pdf
Any reservations about the quality of the file, or anything
else, before I start doing a whole bunch more just like this?
The bleed between the letters is too great.
I pull down the B/W cut-off under such circumstances.
The other limit, to stay above in this case, is the point where
continuous lines break, E.G "m" starts to look like "ni",
"k" like "lc",
etc.
If you're still using the 150dpi it's a problem.
John A.
7:45 p.m.
Tom Owad wrote:
Here's the first three pages, a 152 KB pdf file: <www.applefritter.com/
temp/sample.pdf>. Any reservations about the quality of the file, or
anything else, before I start doing a whole bunch more just like this?
These scans are still the original 150dpi grayscale scans,
down-converted to 1bpp, aren't they? I haven't tried OCRing these pages,
but they look as if the bold text especially would upset the program.
The best technique is to scan at the *final* colour depth and
resolution, so I would suggest rescanning these pages at 400dpi, 1bpp. I
used to scan at 600dpi, but for text pages it doesn't produce noticeably
better results than 400dpi, and the files are significantly larger.
Al has already mentioned Eric Smith's tumble
<http://tumble.brouhaha.com>, which does a great job of concatenating a
bunch of TIFFs and JPEGs into one PDF file, converting the TIFFs to the
best Group 4 compression on the way. It will even allow you to add
bookmarks (with some limitations).
I used to use tumble, but now I am using the Perl PDF::API2 module,
which allows me to take a more hybrid (read: time-consuming!) approach
to each page. I now scan text pages at 400dpi, 1bpp and then rescan any
pages with photographs at 150dpi or 200dpi grayscale, saved as JPEG. I
then crop the JPEG to just the photograph and put that over the top of
the TIFF page (having removed the TIFF version of the photograph). That
improves the overall page size, because the TIFF compression doesn't
have to cope with the inevitable dithering of the photograph.
You can see an example of this approach in the ADM-3A Maintenance Manual
at <http://vt100.net/lsi/adm3a-mm/adm3a-mm.pdf> (124pp, 7.8 MiB). This
manual has a bookmarked Table of Contents, photographs and text on the
same page, and a colour schematic of the double-sided PCB at the back.
--
Paul
28 Jun
28 Jun
2:35 a.m.
>>>> "Paul" == Paul Williams
<paul(a)frixxon.co.uk> writes:
Paul> Tom Owad wrote:
> I was doing 150dpi greyscale jpegs. I've
changed to 400dpi 1bpp
> tiffs, and now I'm ending up with each page about 700K... (scan
> as greyscale tiff (unknown compression), convert to 1-bit with lzw
> compression, save as pdf in Photoshop).
Paul> You need to scan at 1bpp, not grayscale. When you are
Paul> converting from grayscale to 1bpp, you are probably getting
Paul> dithering (a pseudo-random scattering of black and white
Paul> pixels) in order to approximate the original grays at the edges
Paul> of characters. This dithering will ruin the achievable
Paul> compression in G4, which is a fax algorithm optimised for runs
Paul> of white and black pixels.
That certainly would do it, and it will also wreck the appearance of
the resulting files. For black and white material, the conversion
method you need is "Threshold" -- you do NOT want "dither".
paul
26 Jun
26 Jun
10:38 p.m.
On Sat, 2004-06-26 at 21:30, Tom Owad wrote:
I was doing 150dpi greyscale jpegs. I've changed
to 400dpi 1bpp tiffs,
Personally I do greyscale TIFF images at 8bpp - I wouldn't want to risk
something not being able to process the data at a later date, and
storage is cheap.
I'm not set up for messing around with images on the system I'm typing
this on in order to test, but if using 4bpp results in a file size
reduction then that would probably be sufficient (I can't remember how
TIFF handles 'odd' palette sizes when encoding the pixel data)
Personally I find just looking at stuff encoded at 1bpp can be hard on
the eyes due to harsh pixellated edges on text. Plus if you're dealing
with source media that's dirty, faded, printouts, or photocopies of
originals then encoding at 1bpp could rule out any later OCR process.
Storage is cheap these days - go for maximum possible quality (and
archive the data elsewhere, keeping your own lower quality copies if you
don't have the storage space).
Of course if you're just doing this for your own personal benefit then
it's less of an issue - but if you're doing it to preserve data for
others then it makes sense to only have to scan stuff once!
cheers,
Jules
28 Jun
28 Jun
2:33 a.m.
>>>> "Tom" == Tom Owad
<owad(a)applefritter.com> writes:
Tom> On Saturday, June 26, 2004, Al Kossow, wrote:
> I scan at 400dpi 1bpp and save the pages using
group4 compressed
> TIFFs which is the maximum res the IS520 scans at.
Tom> I was doing 150dpi greyscale jpegs. I've changed to 400dpi 1bpp
Tom> tiffs, and now I'm ending up with each page about 700K... (scan
Tom> as greyscale tiff (unknown compression), convert to 1-bit with
Tom> lzw compression, save as pdf in Photoshop).
Tom> Using tiffcp to change the compression from lzw to g4 more than
Tom> doubles the size to 1.6 MB. That's for a single page.
Tom> Surely this is too large?
Quite possibly the scan is "dirty". Ideally you'd want the letters to
be black and the rest of the page white. More typically, the scan
process will put a scattering of black pixels elsewhere on the page,
for example due to "bleed through" from the text on the reverse side.
If the pages are pretty consistent in quality and color, you can tweak
the scan settings to help this, or you can achieve the same result by
post-processing in Photoshop or a similar tool. If it has
"automation" features (as Photoshop does) you can process lots of
pages quickly.
The trick is to look for a threshold setting for the black vs. white
threshold that results in minimal pixels on the page, but not so high
that the letters lose their shape. This is a compromise -- the edge
of a printed letter is not really sharp in a scan, so as you raise the
threshold some of the outer pixels change from black to white -- your
letter gets "thinner". If you can't get both a clean page and an
acceptable letter shape, then the source material isn't good enough to
support bitonal scanning. If so, you'll need a grayscale scan and
you'll have to put up with the larger file sizes that result.
paul
4:43 a.m.
On Mon, 2004-06-28 at 14:33, Paul Koning wrote:
The trick is to look for a threshold setting for the
black vs. white
threshold that results in minimal pixels on the page, but not so high
that the letters lose their shape. This is a compromise -- the edge
of a printed letter is not really sharp in a scan, so as you raise the
threshold some of the outer pixels change from black to white -- your
letter gets "thinner". If you can't get both a clean page and an
acceptable letter shape, then the source material isn't good enough to
support bitonal scanning. If so, you'll need a grayscale scan and
you'll have to put up with the larger file sizes that result.
Personally I'd make sure I had copies of the *original* non-processed
scans archived, though. It's really easy to lose quality when messing
around with threshold settings for 1bpp scans or tweaking
brightness/contrast for greyscale scans. Typically diagrams tend to
suffer more than text, and it's very hard to "proof read" those after
processing to make sure they're spot-on - it's all to easy to miss
something.
Remember that on old documents the original page quality and contrast
can vary quite a bit, plus some pages may have aged differently to
others - so it's not like you can pick a setting that works for one page
and apply it to all.
Personally I'd only want to be doing the scanning once as it's such a
time-consuming job (and OCRing is even worse!). Plus (as I'm sure is the
case with others on the list) I have some original documents where the
number of surviving copies worldwide is probably in single figures. I
wouldn't want a scan to exist where information is missing, and the
original source document is impossible to track down. For more common
documents it's less of an issue - but still a pain to re-scan anything
(and it means there's a fixed and a broken version then floating around
out there too!)
I just do everything as greyscale, save and archive the scans with no
processing whatsoever, and only tweak
colour/brightness/contrast/threshold/whatever settings just prior to
feeding into OCR software. Storage really isn't an issue these days (I
keep data on both tape and hard disk)
cheers,
Jules
7433
days inactive
7435
days old
10 comments
7 participants
participants (7)
-
aek＠spies.com -
allain＠panix.com -
julesrichardsonuk＠yahoo.co.uk -
mouse＠Rodents.Montreal.QC.CA -
owad＠applefritter.com -
paul＠frixxon.co.uk -
pkoning＠equallogic.com