Why PDFs give AI a headache

Every charity sits on a mountain of PDFs. Board papers, annual reports, policy documents, survey forms, evaluation reports, case files. When people talk about using AI with your existing documents, they usually mean PDFs. And the pitch sounds simple: upload the file, ask your question, get an answer.

It works, sometimes. Upload a cleanly formatted report and ask for a summary, and you'll get something useful. But that's not the problem most charities actually have. The problem is: we have 500 survey forms from six different NHS trusts across three years, each with slightly different layouts, and we need structured data out of them. Or: we have a decade of board papers and we need to trace how a policy position evolved. Or: we need to extract financial data from 200 annual reports that each use different table formats.

That's where the headache starts.

What's actually happening under the hood

A PDF was designed to look identical on any screen and any printer. It was deliberately built to be human-readable, not machine-readable. It tells the screen where to put marks. It doesn't know what a table is, or that the text in column two relates to the header in row one, or that a question continues from page three to page four. It just knows where the ink goes.

When you upload a PDF to ChatGPT or Claude, one of two things happens. Either the PDF gets converted to an image and the AI figures out the hierarchy visually, or the text gets extracted first and the AI has to guess where content belongs based on whatever the extraction preserved. Neither is as clean as it looks. The AI model itself is often capable enough. The problem is the scaffolding around it: how the document gets broken down before the AI ever sees it.

For simple documents this works well enough. For tables, it frequently doesn't. Merged cells, spanning headers, nested sub-rows: the more complex the table, the more the extraction scrambles things. We've seen financial tables where AI confidently reported numbers from the wrong column because the PDF's internal structure didn't match what it looked like on screen.

Multi-column layouts cause similar problems. Scanned documents are another category entirely, needing OCR that's good but not perfect, especially with handwriting or faded print. Every extraction error compounds into the analysis.

There are off-the-shelf document processing services from Microsoft, AWS, and Google that handle some of this. For certain tasks they're tolerable. But for many, you need a more sophisticated pipeline. The advantage of using an AI model within that pipeline is that it can handle the variance you inevitably get from human-created PDFs. Helping deconstruct what's on the page for the model, rather than just throwing a raw file at it, is where the real improvement comes from.

The gap between demo and solution

"AI can read a PDF" and "AI can reliably extract structured data from your actual documents" are very different claims. Treating them as equivalent is where projects come unstuck.

We built a system for Breast Cancer Now to process patient and staff feedback surveys from their Service Pledge Programme across the NHS. Over 20 different form structures, each with approximately 400 data elements: single-choice questions, multi-choice, complex matrices, free text, hybrid questions combining multiple choice with open responses. Questions spanning multiple pages. Forms that had evolved differently at each NHS trust because the collaborative relationships with hospitals mattered more than format consistency, and rightly so.

A recipe that says "upload the PDF and ask AI to extract the data" would have produced unusable results. The actual solution involved building schema systems for each form variation, creating mapping files to reconcile different question wordings, dual processing pipelines for page-level and question-level extraction, and extensive mock testing before touching real patient data. The system works - the AI works - because of there's context and scaffolding placed around it. Getting there required understanding both the documents and the organisational context behind them.

That's an extreme example. But the principle scales down. Any charity wanting to reliably process documents at volume, not just read them one at a time, will hit some version of this problem.

The trade-offs that actually matter

For one-off tasks, uploading a PDF to an AI chat is fine. Summarise this report. Find the key recommendations. Pull out the budget figures. Check the output carefully and you'll get useful results most of the time.

For anything beyond that, the question isn't really "recipe or build?" It's about four trade-offs.

How many documents are you processing? One annual report is a different problem from 500 survey forms. Volume changes everything about what approach makes sense.

How fast do you need them processed? If speed matters, manual upload-and-check won't scale. If you can take your time, a simpler approach might be good enough.

What can you spend on processing? AI tokens cost money. A sophisticated pipeline that processes each page multiple times to cross-check results costs more per document than a single pass. For some tasks, the extra cost is justified. For others, a cheaper approach with a slightly higher error rate is the right call.

What error rate can you tolerate? This is the big one. If you're summarising board papers for convenience, a few mistakes don't matter much. If you're extracting patient feedback data that feeds into NHS reporting then accuracy is important. Your tolerance for error should drive every other decision.

Some PDFs will always be challenging because of their layout. But in most cases, you can improve performance by helping the AI understand what's on the page rather than just throwing the raw file at it. If you're sitting on a collection of PDFs and wondering what AI could do with them, start with those four questions to understand what pipeline you should be considering investing in.