Comparing PDFs, Round 2

I've found an even better application for the idea mentioned in my last post. It can be very hard to see all the copy editor's corrections. Well, PDF comparison to the rescue! This script is slightly more complex, since I have a number of different files from them, one for each chapter. But not a problem. First, explode the PDFs into images using pdftocairo, as before. Now:

#!/bin/bash

#DEBUG=echo;
# basename of the images for this
# chapter from copy editor
TYP=$1;
# basename for 'original' pages
BASE=SenseBook-19-IX-2023-Revised
# first page number of 'original' pages
XPAGE=$2;

for PG in $TYP*tif; do
    NEXT=$XPAGE;
    if (($XPAGE < 10)); then NEXT=0$NEXT; fi
    if (($XPAGE < 100)); then NEXT=0$NEXT; fi
    $DEBUG compare $PG $BASE-$NEXT.tif Comp-$NEXT.tif;
    XPAGE=$(($XPAGE + 1));
done

When that's done, it gives me the comparison images for that chapter. So now just:

convert Comp*tif Edited-ChN.pdf

And now it's easy to see where the corrections are:

Comparing PDFs

I'm working on final corrections for my book Modes of Presentation, which I'm again typesetting myself via LyX and LaTeX (as I did Frege's Theorem and Reading Frege's Grundgesetze). I'm paranoid about something weird creeping into the book and have been comparing the new and old pages as I go. I figured there had to be a better way to do that than flipping back and forth between the two PDFs. All the more so given that doing so feels like one of those change-blindness experiments.

Well, Linux and the command line to the rescue. The ImageMagick suite contains a 'compare' command that takes two images and produces a new one that shows the differences between them. So all I have to do is explode the PDF into a bunch of page images and run the compare command on them. Here's a generic script to do it:

#!/bin/bash

# Uncomment to test with just a few pages
#D2="-l 10";
# Image resolution
RES=100;

pdftocairo $D2 -gray -tiff -r $RES NEWPDF.pdf New;
pdftocairo $D2 -gray -tiff -r $RES OLDPDF.pdf Old;
for NEW in New*tif; do
    BASE=${NEW#New};
    OLD=Old$BASE;
    compare $NEW $OLD Comp$BASE;
done

Here's an example of what you get:

It's not readable, but you can easily see where the changes have been made and, if need be, check the actual page. Mostly, I want to make sure that nothing dramatic has changed with the page breaks, etc, anyway.

Of course, you can also do this on other operating systems, but they do not encourage you, as Linux does, to use the command line.

Creating PDFs for EBrary Reader

Turns out that Brown has a subscription to at least part of the Ebrary archive of online books. I found this out because I was looking for a book in our library, using the online catalog, and it turned out that said book was available online. Very cool!

Well, kind of cool, until I found out what this involved. By default, you basically have to read the document online. You can download the "EBrary Reader", which is a Java application, and read documents using it. But it's kind of clunky, to say the least. What I wanted was a PDF that I could then use as I wanted. How to get one?

I noticed that the EBrary Reader would allow me to print, so I thought maybe I could print the file as a PDF, since the default print dialog for Fedora lets you do that. Unfortunately, the Java application was not using the system dialog, but a Java dialog, so that didn't work.

A little googling led me to the cups-pdf package, which installs a system-wide PDF printer for the Common Unix Printing System. A quick "sudo yum install cups-pdf" was enough to give me access to that.

The next step was to convert this file to DjVu, which tends to be much smaller than the corresponding PDF. I've done this a million times before, so figured it would be pretty easy. Unfortunately, it was not.

The first step was to run the pdfimages command (from the poppler-utils package):

pdfimages -p file.pdf p

to extract the page images. Imagine my surprise when I got 420 images from a 20 page paper! It turned out that each page was constructed from 21 different images, stacked on top of each other. (To help with download times?)

Fortunately, I've had enough experience with ImageMagick to know this was not a problem that could not be solved. It took a little more googling, and a little experimentation, but eventually I found out that:

for i in 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20; do
  montage p-0${i}*.ppm -geometry +0+0 -background none -tile 1x21 page-$i.tiff;
done

would stack all the images back on top of each other.

So now I had 20 page images, all as TIFFs, and those could then be fed to ScanTailor for processing on the way to creating a DjVu.