*A multilingual extraction pipeline, an evidence-first review dashboard, and three passes of agentic review to close a five-figure citation backlog.*
**Summary:** I built the pipeline and PBMP Citation Studio, then used Codex computer use with three agent review passes per flagged citation to reduce roughly 10,000–13,000 unresolved flags to zero in the completed run.
**Status:** active · **Year:** 2023 – present **Tags:** AI Systems, Digital Humanities, Data Engineering, OCR, Multilingual
---
I completed PBMP Citation Studio so the review queue, exact page evidence, and editable citation record could sit in one workspace. Then I ran agentic loops with Codex computer use across three layers of review for each flagged citation. A backlog of roughly 10,000–13,000 flagged citations reached zero unresolved flags in the completed review run.
**Pipeline:**
1. **Review queue** (input) — Find flagged citations and review reasons. 2. **Exact-page evidence** (process) — Inspect the mapped source image and column. 3. **Citation record** (process) — Check and correct structured fields. 4. **Agent review loop** (rule) — Three passes using Codex computer use. 5. **Completed review queue** (output) — Zero unresolved flags in the finished run.
The review system joins source evidence, editing, and repeated agent review in one workflow.
Zero unresolved flags describes the completed workflow, not a proof of zero extraction errors. The public studio currently loads an earlier dataset snapshot with 840 review items; it demonstrates the interface rather than the final review state.
The sections below document the earlier extraction and evaluation stages. Their row counts, review queues, and evaluation samples are historical snapshots and should not be read as the final state of the completed review run.
The Nova Bibliografia Pompeiana is a huge bibliography of work on Pompeii and Herculaneum. It was valuable, but it was still basically trapped as a book: pages, columns, headings, and citations that humans could read but software could not reliably sort. To turn it into data, the pipeline had to understand the words alongside the layout, author inheritance, cross-references, and source location. That is why the problem stayed stuck for so long.
 *SOURCE — A raw page from the Nova Bibliografia Pompeiana.*
 *OVERVIEW — The same bibliography varies by language, authors, places, dates, series, and level of detail. Color overlay maps each axis of variation onto a real two-column page.*
German title in an Italian-language bibliography. Title-case rules differ.
RM is Mitteilungen des Deutschen Archäologischen Instituts. No universal registry maps it.
"v. anche" = "see also." A pointer, never a standalone record. Has a number, still isn’t one.
Same number, different volumes, different authors, different years. The number alone fails as an ID.
Model matches the page exactly. Strict scoring marks it wrong because it doesn’t match the normalized gold.
The breakthrough was GLM OCR. It preserved enough of the page structure, columns, headings, and citation numbers to make the rest of the pipeline possible. From there, the system could parse rows, classify fragments, send real standalone citations to Qwen 3.6 Plus, and keep risky rows out of the clean import path. The final pipeline is not one prompt, it is a full production system with parsing, routing, extraction, cleanup, IDs, and review tiers.
**Pipeline:**
1. **Source PDF** (input) — Nova Bibliografia Pompeiana, scanned pages and columns. 2. **GLM OCR** (process) — Layout-preserving read of pages, columns and citation numbers. 3. **OCR parser** (process) — Segments rows and tracks each row back to its page and column. 4. **Row classifier** (rule) — Decides if a row is a standalone citation or a fragment like 'v. anche'. 5. **Qwen 3.6 Plus** (process) — Structured field extraction on standalone citations only. 6. **Fragment rules** (rule) — Keeps the raw row, marks it for review, never invents fields. 7. **Postprocess** (process) — Normalizes whitespace, dates, ranges and inherited authors. 8. **Deep-sweep audit** (process) — Fixed 12,503 cells, every change logged with a reason. 9. **NBP_UI_UID** (process) — Collision-safe ID built from source, page, column and hash. 10. **Final split** (output) — Zotero-ready rows on one side, need-review rows on the other.
Each stage has one job. Rows retain their source locations and review reasons so uncertain cases can be inspected and corrected.
*Historical model-error estimate: ~1.5%. These results use different scoring definitions, not successive accuracy improvements or the error rate of the completed review run.*
I compared candidate models against hand-labeled citation examples for accuracy, speed, and cost per record. The pipeline also needed to preserve page layout, distinguish standalone citations from fragments, and retain the source location behind each field.
I separated those responsibilities: OCR reads the page, deterministic rules segment and route rows, an LLM extracts structured fields, and validation identifies cases requiring review. PBMP Citation Studio brings the queue, exact source image, and editable record into one workspace for repeated review.
This earlier export contained 40,628 structured rows: 30,526 labeled Zotero-acceptable and 10,102 requiring review. Structural checks reported no duplicate UIDs or blank values in the listed required fields. These counts describe the earlier extraction export; the completed agentic review run and the public demo use separate snapshots. Structural completeness does not measure bibliographic correctness.