Extracting table contents from a collection of PDF files

Question

I have a stack of PDFs - potentially hundreds or thousands. They are not all formatted the same, but any of them MAY have one or more tables with interesting information that I would like to collect into a separate database.

Of course, I know I have to write something to do this. Perl is an option for me - or perhaps Java. I don't really care what language so long as it's free (or cheap with a free trial period to ensure it suits my purposes).

I'm looking at CAM::Parse (using strawberry Perl), but I'm not sure how to use it to locate and extract tables from the files. I guess I do have a preference for Perl, but really I want something that works dependably and is reasonably easy to do string manipulations with.

What is a good approach for something like this? I'm at square one, so if java (or python etc.) have better hooks, now is a good time to know about it. General pointers good; starter code would be strongly preferred.

Answer 1

1. The PDF format from its inception (more than 20 years ago) never was intended to be host of extractable, meaningfully structured data.

2. Its purpose was to be a reliable visual representation of text, images and diagrams in a document -- a kind of digital paper (that would also reliably be transferred to real paper via printing). Only later in its development more features were added, which should help in extracting data again (google for Tagged PDF).

3. For some examples of problems which are posed when data scraping tables from PDFs, see this article:

   • Why Updating Dollars for Docs Was So Difficult
      

4. Contradicting my point '1.' above, now I say this: for an amazing family of tools that gets better and better from week to week for extracting tabular data from PDFs (unless they are scanned pages), see these links:

   • Introducing Tabula: Upload a PDF, get back tabular CSV data. Poof!
   • Tabula-Extractor: A Command Line Interface to Tabula
   • Tabula source code repository
   • Tabula API (upcoming, not ready yet)

So: go look for Tabula. If any tools can do what you want, at this time Tabula is probably amongst the best for the job!

---

Update

I've recently created an ASCiinema screencast demonstrating the use of the Tabula command line interface to extract a big table from a PDF as CSV:

(Click on image above to see it running. If it runs too fast for you to read all text, make use of the "Pause" button (||-symbol).)

It is hosted here:

• https://asciinema.org/a/22761