Agent P-DF

Agent P-DF finds tables in scanned documents and PDFs, rebuilds their rows, columns and merged cells, reads the text, and opens the result in a split-screen editor. You fix what the models got wrong, then export JSON, CSV, HTML or Excel.

I built it for STP MACHATHON 7.0, a table extraction competition, where it took first place. The rule behind it: the models will sometimes be wrong, so the editor has to be exact. Models propose geometry, plain code turns geometry and OCR text into cells, and the studio handles what is left.

A page goes through two Table Transformer (TATR) models. One finds the tables; the other finds the rows, columns, headers and merged cells inside each one. PaddleOCR v5 models, run through ONNX Runtime, find and read the text. Python code then assigns every piece of text to a cell and builds the table.

Keeping the parts separate meant I could measure each one and replace the one that failed. I fine-tuned the two TATR models; I used the PaddleOCR models as released, because retraining them was not needed.

Public table datasets are clean, and real scans are not. Before training anything, I built a dataset generator. It wrote tables and table-like pages from pools of scientific and financial words, many fonts, colours, sizes and table types, compiled them with LaTeX and rasterised them with PyMuPDF. It produced 16,000 pixel-precise images, with exact labels for both detection and structure.

They were too perfect: crisp, high-DPI renders that no real scan looks like, so I lowered the DPI to match real pages. The part that lasted is the adversarial synthesizer. It generates pages that look like tables but are not, in four tiers from a table of contents to a grid of sub-figures, and saves each one with an empty label file. A model trained only on real tables learns to say yes to every grid; these pages teach it what plain background is. In the final detection run, 100 of them went into the mix beside the public sets and the noisy Phase 3 pages.

My first detector was YOLO. It drew boxes around repeated words instead of following the page layout, and its GPL licence did not fit a product I wanted to ship. I moved to the Table Transformer family.

Trained on clean public tables, the TATR detector scored AP 0.93 but also drew tables on flowcharts and multi-column text. I added Phase 3, a noisy set of 422 real pages, and generated hard negatives: pages that look like tables but are not, such as tables of contents, figure grids and ruled text. The score dropped to AP 0.89. I kept that model because the deck's comparison rates its false positives on real pages as low instead of high. I did not measure a false-positive rate.

The final run trained for 40 epochs on 5,272 images and validated on 932. Loss fell from 0.61 to 0.16, and the last epoch scored AP 0.896 and AP50 0.971. I froze most of the ResNet backbone and trained its last stage at a lower learning rate than the transformer, so the pretrained model was adjusted, not overwritten.

The structure model started at AP 0.477 (AP50 0.658) on my training data. I looked at the pages. On more than 60 images of the Phase 3 set, the "projected row header" label sat on almost every cell. Column headers were missing and merged cells were inconsistent.

A quick test confirmed it: dropping that one class from the training config lifted AP50 from about 0.75 to about 0.91 at once. Then I wrote an editor for the annotations and bulk-deleted 5,353 wrong boxes in one pass, on pages 82 to 154. With the class dropped and the wrong boxes gone, the same TATR model reached AP50 of about 0.95 and AP75 of about 0.90.

I did not change the architecture. I changed the data.

The TATR decoder predicts at most 125 objects per table. My diagnosis was that large tables ran out of those queries when the model also had to guess which rows were projected headers. I removed that class so the model only outputs a dense grid of rows and columns, then added a rule: if OCR finds one text block that spans the table's full width, that row is a projected header.

The pattern repeats through the project. The model finds geometry, and plain code handles the part where the answer has to be predictable.

The studio shows the page and the table side by side. The video above is a walkthrough. It is a single-process FastAPI service with a React front end, and on a CPU it runs two jobs at a time. A health page shows uptime, success rate and p50 and p95 latency.