Research

A general-purpose VLM document parser: a DP-Bench TEDS and TEDS-S leaderboard candidate

One locked run of the Gemini 3.6 Flash-based Consilience document-parsing pipeline scored 96.34 TEDS and 98.41 TEDS-S on the latest public 200-document DP-Bench, 0.28 and 0.79 points above the highest listed values. Official review has been requested.

• 96.34 / 98.41 TEDS / TEDS-S, one locked run, n=200, listing requested in Discussion #6
• 200 / 200 documents completed; no per-document selection or manual correction

Consilience EngramRAG: document-grounded ontology memory for long-horizon human–AI work

A persistent memory lifecycle joining human-editable Markdown with a machine-navigable ontology. Model-fixed ablations, real-vault evaluations, failed replications, and explicit limits isolate the contribution of structure outside the model.

• 14/21 → 18/21 code localization after adding graph structure with model, tasks and agent held fixed
• +30.7 ~ +38.7pp recall@10 gain over keyword-only across two real vaults

Veridion open-set calibration: scale-aware thresholds and measured abstention

Recasts character retrieval as 1:N open-set identification rather than a fixed cosine cut-off. It derives a false-positive budget from gallery and region count, abstains beyond empirical resolution, calibrates whole-frame and crop cohorts separately, and merges unequal galleries fairly.

Wrtn Technologies R&D · sole contributor for requirements, architecture and implementation

• 1:N · FPIR the cut-point is a function of gallery size, region count and review budget, not a model constant
• ABSTAIN withhold the verdict instead of extrapolating beyond the measured impostor tail

Counterfactual rewind validation: preventing out-of-scope graph deletion

A read-only replay of the old algorithm against the oldest live checkpoint calculated that 102,020 of 159,506 quads would be removed, along with graphs for 1,057 documents untouched by the anchor turn. This was a counterfactual, not an incident; the fix scopes replacement to touched documents.

• 102,020 / 159,506 quads the old algorithm would remove, live-state counterfactual, not observed deletion
• 0 / 1,057 affected documents actually touched by the anchor turn

A defect audit built on controls and evidence grades

Bug and control conditions were paired on a green baseline, with evidence graded as UI, executed logic, deductive trace, or live forensics. Fourteen reproduced findings are enumerated, but the full 45-candidate ledger and individual refutation logs were not retained.

• 14 findings enumerated in the audit snapshot; the full 45-candidate ledger was not retained
• 0 failures pre-audit baseline, 982 frontend and 1,010 backend tests

A default-model swap exposed an undefined document-language policy; a later harness audit narrowed the claim

A July synthetic harness recorded different language choices across two models and motivated separate document and conversational language policies. An August 10 audit then found a missing runtime prompt tail, unmeasured BARE branch, and wrong transport, withdrawing production-effect rates.

• 3 fidelity breaks runtime tail omitted, BARE unmeasured, transport unlike production
• Superseded production-causal and post-fix rates from the historical synthetic harness withdrawn; faithful rerun required

Exhaustive extraction added relations, but no retrieval gain was detected

Only selected subgraphs changed: 120 prose and 59 table notes inside a 2,810-document graph. Relations increased by 10–17%, while note recall@10 moved +0.7pp for prose and +0.1/+0.7pp for tables, with no gain detected beyond harness variability. This is not proof of a global zero.

• +0.7pp observed prose note-recall@10 difference; 120/2,810 subgraphs changed
• +0.1 / +0.7pp table note-recall@10 in two five-replicate runs; relations +17%

Post-ship validation: why a judge-source change and tolerant parser were reverted

With 121 relation-verdict pairs held fixed, all three source-text disagreements favored the old implementation under a same-model oracle. A tolerant parser added zero recoveries in 60 direct calls. Both changes were reverted; a table exemption was deferred after a 9/3,215 scope check.

• OLD 3 · NEW 0 121 pairs, 3 disagreements, same-model three-call oracle, sign-test p=.25
• 0 / 60 additional content recoveries by tolerant parsing, direct /responses, 95% upper bound about 5%

MuSiQue pooled-corpus ablation: the PPR graph leg and the limits of fixed fusion weights

In an internal 100-question, 1,730-passage retrieval pilot, strict support-set ALL@10 was 63% with full fusion, 25% without PPR, and 23% with keyword-only retrieval. The four-hop 23.5% versus 0/17 result is scoped to this setup, alongside a corpus-dependent BM25 reversal.

• 63.0 vs 25.0 ALL@10, full fusion vs no PPR, n=100
• 23.5 vs 0.0 four-hop ALL@10, n=17, no per-hop significance test

Adapting CSLS for retrieval seeding: a 100k-label cutoff regression and raw-cosine floor

Dense-seeding recall@10 rose from 77.5 to 97.0 across six fixtures and three runs. At up to 100k labels and 350 balanced queries, plain CSLS improved @10 while regressing @40. A raw-top1 floor narrowed the loss, but a significant −4.9pp residual remained at 30k and 60k.

• 77.5 → 97.0 dense-seeding recall@10, six fixtures × three runs
• 46.9 / 78.9 100k floor @10 / @40; raw 40.3 / 81.4; n=350

A full SWE-bench Verified run: 394/500 resolved, with attribution limits

An integrated Opus 4.8 and full-scaffold system resolved 394 of 500 tasks under the official scoring code. With no same-model scaffold-off control, attribution is kept separate: a Sonnet 4.6 tool A/B measured code graph at 70.1% versus grep at 65.2% File Acc@5.

• 394 / 500 78.80%, official scoring code, integrated model-and-scaffold result
• 70.1 vs 65.2 File Acc@5, graph vs grep, Sonnet 4.6, stopped run n=465

Xhadow: acoustically grounded pronunciation coaching against your own voice

The learner is compared with a target-language rendering of their own cloned voice. Browser DSP, phoneme assessment, word-anchored segmented DTW and WSOLA compute the evidence, which is translated into actions for the tongue, jaw and lips.

• FFT · formants Browser DSP: F0, spectral peaks, voicing and timbre analysis
• DTW · WSOLA Word-anchored segmented alignment and pitch-preserving resynthesis