A general-purpose VLM document parser: a DP-Bench TEDS and TEDS-S leaderboard candidate
One locked run of the Gemini 3.6 Flash-based Consilience document-parsing pipeline scored 96.34 TEDS and 98.41 TEDS-S on the latest public 200-document DP-Bench, 0.28 and 0.79 points above the highest listed values. Official review has been requested.
- TEDS / TEDS-S, one locked run, n=200, listing requested in Discussion #6
- 96.34 / 98.41
- documents completed; no per-document selection or manual correction
- 200 / 200
Result: A single locked run of the Consilience document-parsing pipeline scored 96.34 TEDS and 98.41 TEDS-S on all 200 documents in the latest public DP-Bench revision. Both scores are above the highest values currently listed in the benchmark table.
Status: Submitted to the official DP-Bench repository for maintainer review. Leaderboard listing is pending.
On 31 August 2026, I submitted the result to DP-Bench Discussion #6.
Result
The evaluation used the latest public scorer revision and all 200 released documents. This was one full-corpus run, not the best result selected from several attempts. All 200 documents completed successfully, and no document-level outputs were selected or combined.
| Metric | Consilience | Highest currently listed | Difference |
|---|---|---|---|
| TEDS, structure + text | 96.34 | 96.06, Upstage standard | +0.28 |
| TEDS-S, structure only | 98.41 | 97.62, Upstage enhanced | +0.79 |
The currently listed systems include services from Upstage, AWS, Google and Microsoft, as well as LlamaParse and Unstructured. The comparison here is specifically about TEDS and TEDS-S. DP-Bench does not publish a single overall score, so this result should not be read as an overall ranking across every metric.
What I built
The foundation model was Google’s general-purpose Gemini 3.6 Flash. I did not train or claim ownership of that model. My work was the Consilience system around it: PDF ingestion, inference orchestration, structured document representation, benchmark-schema conversion and a reproducible evaluation harness.
That distinction matters. Model capability is only the starting point in document parsing. The quality of the surrounding system determines whether content, reading order and table structure survive the path from a PDF to usable structured data.
Validation and claim boundary
The candidate was evaluated with a fixed configuration across the complete dataset. Ground truth was not used during inference, outputs were not selected per document, and predictions were not manually corrected after the run. The measurement records and evaluation artifacts have been retained for maintainer verification.
The precise claim supported by this result is:
On the latest public DP-Bench revision, the submitted Consilience candidate scored 96.34 TEDS and 98.41 TEDS-S, above every value in the currently listed rows. Official listing is under review.
DP-Bench is a public document-parsing benchmark, not a hidden-test OCR competition. This result demonstrates performance on its current public evaluation contract; by itself, it is not proof of the world’s best OCR model or of generalization to every unseen document domain.
Why it matters
A pipeline built independently around a general-purpose VLM exceeded the currently listed specialist document-processing services on both table-recognition metrics. For me, the important result is not only the two scores. It is evidence that careful engineering across the full document pipeline can be as decisive as the choice of model.
What this measures
- Consilience
A general-purpose AI agent that uses a knowledge graph for coding, design and answering questions. It builds and maintains ontologies from documents without a resident engineer.