sheet 02.2 · Client project · name withheld
Structured extraction from historical registers
Two centuries of scanned printed registers turned into validated datasets, with a fine-tuned vision model that runs locally.
- role
- Main developer. Pipeline, fine-tunes and evaluation.
- period
- Aug 2025 — now
- stack
Printed registers from the 18th to the early 20th century: dense tables, abbreviations, changing layouts across periods, and roughly half of all fields legitimately blank. The goal was clean, structured records that a researcher can trust.
Two stages. OCR turns a page image into a Markdown table. Structuring turns that table into canonical records, which are validated, normalised against reference lists (places, vocabulary) and exported as the delivered dataset.
Everything that differs between register periods (prompts, JSON schema, column count, per-book exceptions) lives in a profile file per period. No code enumerates the periods, so adding a period means adding a folder.
The first fine-tunes ran on Vertex AI. Later I trained a LoRA on Qwen3-VL 8B, exported it to GGUF and serve it with Ollama, so the extraction can run on local hardware. Gendox is the model service in the middle, so models and settings are switched from its UI.
I wrote the evaluation harness. Every report has the same layout: what was measured, a fixed headline, the worst pages first, then a per-field breakdown.
Blank fields are scored separately from content fields. If they are not, a sparse register looks better than a better model. Blanks the model fills anyway are reported as a hallucination rate.
On ten hand-corrected pages the model had never seen, the delivered pipeline scored 98.9% on the primary metric, 95.1% exact on content fields, and filled 13 of 4,254 blank fields. No records missed, none invented.