The problem
Tablefire extracts tables from financial PDFs — invoices, bank statements, trial balances — and turns them into structured data. The core use case isn’t processing an isolated document; it’s processing the same document type hundreds of times: the monthly statement from the same bank, the recurring invoices from the same vendor.
That splits the problem into two separate questions, and conflating them is the easiest early mistake to make:
- What kind of document is this? (detection)
- Where is the data inside it? (extraction)
The second one is tractable: if I know I’m looking at a Bank X statement, I know where the table is and what each column means. The first one determines whether the system is useful at all, because a detection error doesn’t produce bad data — it produces plausible but wrong data, mapped with the wrong rules. That’s exactly the kind of silent failure an accounting team can’t afford.
What makes the problem non-trivial is that documents from the same issuer aren’t identical to each other. They shift month over month: row counts vary, promotional blocks appear and disappear, the logo gets redesigned, the table moves 15 pixels, the scan comes in skewed. Any method that assumes positional stability breaks in production.
The options I considered
ML classification (visual or text-based). The “correct” path by modern instinct: train a classifier that takes the document and returns its type.
The problem is the cold start, not the ceiling. A classifier needs labeled data, and a new Tablefire user shows up with one document of that type, not two hundred. If the system demands a corpus before it’s useful, the product works for nobody on day one. A generic pretrained model doesn’t solve it either: it can’t distinguish “invoices” from “invoices from vendor Acme, with Acme’s normalization rules” — which is the granularity that actually matters here.
There’s a second problem, less obvious and more serious in this market: a classifier can’t explain its decision in an auditable way. “The model assigned this template with 0.94 confidence” is not an acceptable answer to an auditor asking why an entry was booked the way it was.
Visual template matching. Comparing the document’s visual structure against stored templates.
Fragile for the reasons already noted: skewed scans, resolution changes, minor redesigns. It also penalizes the exact case that matters most — low-quality scanned documents — and inherits the same opacity problem.
Fixed coordinates. Storing absolute positions for each field.
The simplest approach and the most rigid. It survives documents generated by the same system without changes, and fails on any layout variation. Ruled out as the primary mechanism, though part of the idea survives in the extraction stage.
What I chose: user-defined text anchors
A Tablefire template isn’t a saved layout. It’s a set of rules covering the full pipeline — detection, extraction, normalization, and export — and the detection stage relies on text anchors: text fragments the user marks as identifiers for that document type.
While building the template from a real document, the user flags the fragments that characterize the issuer and the format. When a new document arrives, the system extracts its text layer (running OCR first if it’s a scan), searches for those anchors, and scores the match by combining coverage — how many anchors are present — with positional coherence — whether they appear in the region of the document where they were expected. The result is a composite score against an empirically calibrated threshold. Below the threshold, the document isn’t processed blind: it’s flagged for human review.
Three reasons behind the decision:
It works from a single example document. There’s no training phase. The user builds the template and the system can already recognize similar documents. Startup cost is measured in minutes, not in a labeled corpus.
It’s auditable by construction. The answer to “why was this template assigned?” is a concrete list: these anchors matched, these didn’t, at these positions. An auditor can read it and verify it against the original PDF. This isn’t a secondary product detail — it’s the entire value proposition. The alternative already on the market is fast black boxes that nobody fully trusts.
It degrades predictably. When an issuer changes their format, the system doesn’t quietly start producing wrong data: the score drops below threshold and the document comes out flagged. A false negative costs one manual review; a false positive costs a misbooked accounting entry that nobody catches until reconciliation. The cost asymmetry is enormous, and the design respects it explicitly.
What went wrong along the way
I treated confidence as a single value, and that was wrong. The first version produced one number per document. It turned out to be useless in practice: a document can have an excellent template match and still contain three cells where OCR misread a digit. Aggregating everything into one metric hid exactly the information the user needed. Confidence ended up being per field, propagated from the OCR’s word level, with low-confidence cells flagged individually for review.
I overestimated automatic table-line detection. It worked very well on native PDFs with defined borders, and poorly on borderless tables aligned only by whitespace. Trying to solve the general case algorithmically was a bottomless pit. The fix was a product decision, not an engineering one: automatic detection stayed as assistance, and the user draws or corrects the grid when needed. It happens once per template, not once per document, so the cost amortizes immediately. Accepting that a human solves in five seconds what the algorithm wasn’t solving in weeks was the best decision of that phase.
I underestimated normalization. I assumed it was a trivial cleanup step at the end of the pipeline. It isn’t. Ambiguous date formats, decimal separators that change by region, negative amounts expressed in parentheses, vendor names that vary across documents from the same issuer — normalization has as many edge cases as extraction does. It ended up as a first-class stage, with explicit rules defined in the template and visible in the audit log.
What I’d do differently
Design the audit model before the pipeline. I built extraction first and traceability afterward, which forced a rewrite of the data flow so every value would carry its origin. Traceability doesn’t get bolted on: it’s inherited from the first step or it doesn’t exist.
Instrument calibration from the start. The detection threshold was tuned more by hand than I’d like. With a test document bank and automated precision/recall metrics from day one, calibration would have been a measurement instead of a judgment call.
Consider a hybrid sooner. Text anchors are still the right foundation, for auditability and for startup cost. But once a user accumulates processed volume, that history is a labeled corpus — labeled by them, with their corrections included — and it can feed a suggestion layer that proposes anchors or detects format drift before it breaks anything. The original constraint dissolves with usage; the architecture decision should have anticipated that transition from the beginning.