Consilience EngramRAG: document-grounded ontology memory for long-horizon human–AI work

Frontier language models can reason remarkably well inside a single interaction. Long-horizon knowledge work, however, requires more than momentary reasoning. A person’s intent and constraints, earlier decisions and their evidence, and facts later withdrawn must be selected correctly again in a future session. A long context window can hold more tokens; it does not decide what should persist, what should be revised, or what belongs in the present context. Lost in the Middle shows that long-context use changes substantially with the position of relevant evidence, while a reality check on retrieved context finds that synthetic tests underrepresent the unreliable, insufficient, and conflicting evidence encountered in real retrieval.

This work calls that problem the shared-context externalization bottleneck.

For long-horizon, tool-using knowledge work, LLM capability is necessary but insufficient. Reliability is co-determined by how human intent, evidence, and revision history are represented as durable, inspectable, and retrievable external state.

EngramRAG, the core memory architecture in Consilience, operationalizes that hypothesis. Human-editable Markdown is the surface of authorial record; an ontology derived from those documents is the relational index a machine can traverse. The agent searches, reads, and edits the same documents, and knowledge learned during work returns as an ordinary note. Beneath that surface, the graph manages provenance, identity, revision, and retrieval without allowing a machine interpretation to overwrite the human record.

This is not a claim that the system reproduces a human brain. The brain is a design analogy. The research object is the representation, selection, and revision lifecycle of external memory.

1. The problem is externalization and selection, not language alone

The intuition that human language has become the bottleneck for AI is only partly right. People can express rich meaning in language. What they cannot do is serialize every piece of implicit context into a complete prompt on every turn. Documents written over time also fragment: the same object appears under different names, and superseded and current decisions are retrieved together.

On the other side, an LLM can interpret the context it receives but cannot read an intention that was never externalized. Even when the source text exists somewhere, the model cannot use it if the system fails to connect it to the present question. The bottleneck is therefore the following transformation:

1. Externalize tacit human knowledge into a durable record.
2. Make identity and relationships across records machine-navigable.
3. Select only the evidence needed for the current bounded context.
4. Let a person inspect and reverse a wrong interpretation or stale fact.
5. Return newly verified knowledge to the same lifecycle.

Markdown and an ontology do not replace each other. Documents preserve narrative, nuance, and evidence. The ontology makes recurring entities and relationships across those documents operational. EngramRAG’s research hypothesis is that this dual representation is a better substrate for sustained human–AI work than either raw documents or a graph alone.

The framing connects to Clark and Chalmers’ extended mind, the account of physical action reducing information-processing demand in cognitive offloading, and Clark and Brennan’s grounding in communication. Shipman and Marshall’s work on incremental, system-assisted formalization is direct prior art for moving from informal material toward structure, so no claim is made to be the first document-to-ontology system. This study has not measured an improvement in human cognition; these works motivate the problem rather than validate the system’s effect.

2. Three memory planes with different authority

EngramRAG does not put every kind of memory into one vector store. It separates three planes with different trust boundaries and revision rules.

Plane	What it stores	Authority and lifecycle
Document memory	Human-authored claims and evidence, source material, and learned notes produced after agent work	A person can read and edit it directly in Markdown. It anchors the authorial record
Relational memory	Entities, relations, types, document provenance, resolution, and revision candidates	Derived RDF and analysis overlays. It can be rebuilt or its judgments reversed
Procedural memory	User preferences, ongoing projects, and how to work in this workspace	A small, separate agent memory. It is not trusted as an instruction merely because a prior session wrote it

The separation is also a security boundary. A session may have read a prompt injection from a web page or a screen before writing memory. Persisted procedural memory is therefore wrapped as external_content when it re-enters the next session’s system prompt. “What is true?” belongs in a knowledge document; “how should I work with this user?” belongs in procedural memory. The two channels are explicitly distinguished.

The agent’s learn operation is neither parameter training nor an unreviewed write into an authoritative fact store. It first searches for a similar title to reduce duplication, then records a verified result as ordinary Markdown under learned/*.md. The note is visible for inspection and does not become graph knowledge immediately; it enters the same later extraction lifecycle as any other document. That delay is a boundary: a sentence written by the machine does not become an authoritative fact merely because it was written.

3. A memory loop that returns to documents

One system cycle is:

1. Capture and ingest. A person or agent writes Markdown. Imported files are converted with format-specific rules while preserving the original attachment.
2. Incremental extraction. Only documents whose body hash changed are processed into entities, relations, and statements. Long documents are split into at most 16 windows, with prior-window entities supplied as hints for later anaphora.
3. Provenance boundary. Quads extracted from one document enter that document’s named graph. Re-extraction replaces the graph transactionally, preventing triples from deleted prose from surviving silently.
4. Resolution and consolidation. Surface form, embeddings, types, document evidence, and a narrow LLM judgment are combined to assess identity. The result is a reversible overlay, not a destructive merge. A human distinct verdict blocks future automatic union.
5. Retrieval and context assembly. The query seeds the graph; Personalized PageRank diffuses over relations; ranked documents and relational evidence are assembled into a bounded context.
6. Action and correction. The agent answers and edits through the same search, read, and write paths. Writes pass through permission modes, diffs, checkpoints, and rewind boundaries.
7. Re-entry. A verified finding or revised decision returns to Markdown, and the next extraction updates the graph.

The objective is not an ever-growing graph. It is that a person can return to the source, correct a machine interpretation, and cause the derived structure to follow the correction.

Markdown is not the only persistence layer for every byte of system state. Merge verdicts, vocabulary, retry state, and other metadata also live in SQLite and sidecars, and LLM extraction is nondeterministic. The precise claim is therefore not that Markdown alone reproduces the entire system. It is the anchor of human-authored knowledge and the rebuild input for the derived graph.

4. EngramRAG retrieval

The retrieval core begins with the knowledge-graph and Personalized PageRank design of HippoRAG and HippoRAG 2. HippoRAG 2 jointly targets factual, associative, and sense-making memory. EngramRAG focuses on the operational lifecycle required to build, correct, and act on such a graph inside a continuously edited personal workspace.

Explicit search fuses four kinds of evidence:

• lexical overlap between the query and entity surfaces;
• semantic similarity to entity-label embeddings;
• query-to-extracted-triple embedding similarity; and
• document and passage embeddings, combined with BM25 document retrieval.

PPR propagates activation from those seeds through relation, synonym, hierarchy, and document-mention edges. Context assembly then reserves space not only for edges whose two endpoints rank highly, but also for edges adjacent to one important entity and for lower-ranked bridges needed by the question. This addresses a last-mile failure in which retrieval finds the right path but the final context drops an intermediate relation.

Dense tables and catalogs produced another failure: embedding hubs that were moderately close to many queries occupied the candidate set. Instead of a fixed cutoff, the system ports Cross-domain Similarity Local Scaling from cross-lingual embeddings into retrieval seeding, subtracting each entity’s local density. In the first experiment, dense-seeding recall@10 rose from 77.5 to 97.0 with no additional API call. At 100,000 labels, however, a loss appeared at the top-40 cutoff. A floor reserving one seed for the top raw-cosine candidate reduced the loss, but a significant −4.9-point @40 residual remained at both 30,000 and 60,000 labels. Breaking the small-scale result at production scale is part of the contribution, not an omitted footnote.

The full four-leg retriever does not run automatically on every turn. To bound latency and tokens, automatic turn grounding currently uses an embedding-free lexical-plus-graph path and injects a short context of at most eight results. Deeper questions use explicit search or query_graph. The injected block itself warns that it may be partial or stale and instructs the agent to verify specific facts.

5. An ontology as a reviewable interpretation, not truth

An LLM extracting a triple from a sentence does not make that triple true. EngramRAG enforces this distinction in its data model.

Document-scoped provenance. Every extracted quad is attributed to a per-document named graph. Statement nodes can carry source text, relation, and separate attribution for author and extractor, using distinctions from W3C PROV-O and PAV.

Assertion and analysis separation. An observation extracted from user prose and a machine inference across documents do not share the same graph. Inferred supersession and other analysis edges enter <document>#analysis; the original extraction graph remains intact.

Reversible identity resolution. same, broader, narrower, related, and distinct verdicts are stored as an overlay on the original triples. A wrong union can be reversed, and a human distinction becomes a hard separator.

Bounded revision tracking. The system proposes supersession only between dated decision statements about a shared entity in different notes. Accept and reject judgments persist, but this is not general temporal reasoning or a complete bitemporal memory.

A limited role for types. Eleven core kinds and workspace-specific subtypes support navigation and incompatible-type merge vetoes. Type edges are deliberately excluded from PPR to avoid ontology hubs. No claim is made that types improve retrieval accuracy until that contribution is isolated experimentally.

The principle is provenance, not truth. The system does not silently declare what should be believed; it preserves why a relation exists and who authored, extracted, or reviewed it.

6. Holding the model fixed

The central empirical question is not whether a graph looks useful. It is what changes when the graph is removed while the model stays the same.

Research question	Held constant	Intervention	Result	Claim boundary
Does code structure improve agent navigation?	Claude Opus 4.8, tasks, repository, loop	grep → embeddings → graph → graph+nudge	14/21 → 14/21 → 18/21 → 19/21	Code localization only; edit quality and note↔code fusion unmeasured
Does graph fusion matter in real notes?	Questions, vault, shipped retrieval path	Full fusion vs keyword-only	90.7% vs 60.0%, 76.0% vs 37.3%	Fact lookup in two real vaults, recall@10
Does multi-hop evidence require structure?	100 MuSiQue questions, 1,730 passages, extraction path	Graph vs graph-off/keyword-only	ALL@10 63% vs 25%/23%; four-hop 23.5% vs 0/17	Retrieval stage; no reader EM/F1
Does structure affect operational efficiency?	Same live code-agent experiment	Tool configuration	Cost/success $0.180 → $0.092, latency 22.0s → 17.5s, calls 7.76 → 5.29	Cost/success, latency, and rounds consistent across two runs; raw tokens and run cost sample-dependent

The first experiment ran a real tool-using agent loop. Adding embeddings alone left completion equal to grep at 14/21; adding graph structure raised it to 18/21. The product prompt reached 19/21. This decomposition attributes the reliability change to relational structure and the tool path that makes the model use it, rather than to a model swap or to embeddings in isolation. On a separate set of four hard tasks, grep completed 6/12 and graph+nudge 12/12.

This does not establish that the model is no longer a bottleneck. It establishes, on one model and bounded task sets, that external structure can be an independent bottleneck even when the model is held constant. Model capability and memory architecture are better understood as multiplicative axes than substitutes.

7. Representative evaluation and preserved failures

A system can easily overfit its product to a public benchmark. EngramRAG evaluates questions closer to the actual work distribution first, using public benchmarks as stress tests for specific failure modes.

On MuSiQue, a 45-configuration fusion sweep placed BM25 weight at zero in all six leading configurations. The same change reduced recall@10 from 95.3% to 88.0% on a real vault. MuSiQue deliberately selects lexically similar distractors; a user’s notes are not constructed that way. The shipped weights were not changed for the benchmark score.

Another experiment extracted 17% more relations, while table note-recall@10 moved only +0.1pp in the first run and +0.7pp in the repeat—within harness variability. A separate subject–predicate occupancy estimate remained positive at +5.7pp and +3.9pp, but its significance threshold did not repeat (p=0.038 → 0.478). A change shipped without measurement was later compared with a model oracle and reverted.

The research process follows these rules:

• Record the hypothesis, primary metric, rejection threshold, and commit before observing the result.
• Call shipped functions rather than an evaluation copy whenever possible.
• Use paired per-question comparisons and inspect failure strata, not means alone.
• Attempt to break a small-corpus win on real vaults and at larger scale.
• Preserve null results, failed replication, post-registration ambiguity, and deleted raw data in the record.

This is aligned with OpenAI’s guidance to evaluate representative workloads, isolate configuration changes, and rerun the same evals. That alignment is methodological, not a performance certification or hiring claim.

8. Relationship to prior work

HippoRAG 2 combines vector retrieval and a knowledge graph to improve factual, associative, and sense-making memory, framing RAG as non-parametric continual learning. EngramRAG carries that retrieval idea into a local workspace edited every day and makes document provenance, identity verdicts, revision, re-extraction, and agent writes part of the research problem: the operational lifecycle of the graph.

Microsoft GraphRAG builds entity graphs and community reports for global questions over large corpora. Consilience also has communities and global sense-making, but its central constraint is a live corpus in which a person continuously edits the sources and the derived graph must follow.

LongMemEval decomposes long-term memory into indexing, retrieval, and reading, and evaluates extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention. EngramRAG currently has stronger evidence for retrieval and system lifecycle than for an end-to-end evaluation across all five abilities. That gap defines its next benchmark.

MemGPT already introduced virtual context management inspired by tiered operating-system memory. Zep/Graphiti builds a temporal knowledge graph that maintains historical relationships across conversations and business data. ACL 2026’s APEX-MEM combines a domain-agnostic ontology, append-only temporal evolution, and query-time conflict resolution; AgeMem learns store, retrieve, update, summarize, and discard as agent tool actions. Tiered memory, temporal graph memory, provenance, and autonomous memory operations are therefore not EngramRAG inventions. Its distinctive emphasis is a workspace where documents a person owns and edits directly remain authoritative and source correction and re-extraction run through the product path.

Does Memory Need Graphs? poses a critical question for this work: a graph is not an intrinsically superior memory, and poor construction or retrieval can make it harmful. EngramRAG treats the graph as an ablated component, not an article of faith. The recent PAGE-RAG preprint likewise treats a graph as a lossy semantic skeleton rather than a replacement for source evidence. This is why the contribution here is not “the first source-grounded graph,” but the end-to-end operation, correction, and evaluation of document–ontology co-evolution in a continuously edited workspace.

CoALA models language agents through modular memory, a structured action space, and a decision process. Consilience can be read as a systems instantiation: user-owned files, derived RDF, search and write tools, permissions, and rewind in one shipped workspace.

OpenAI’s context engineering with long-term memory notes similarly stores session state, distills and consolidates candidate memories, and injects relevant slices in a later run. EngramRAG extends the lifecycle beyond profile memory into general documents that a person edits directly and a provenance-preserving ontology derived from them.

No individual primitive is claimed as a world first. Graph retrieval, PPR, CSLS, named-graph provenance, agent memory, and dynamic schema induction all have prior art. The contribution is their integration under one set of product invariants—a local-first document substrate, continual extraction, reversible resolution, reviewable inference, and agent action—and the isolation of structural contribution with model-fixed experiments.

9. Current limits

The credibility of this record depends more on the boundary than on the headline scores.

• The human-bottleneck hypothesis is not yet causal evidence. There is no user study of cognitive load, externalization time, long-term recall, or decision quality.
• Model–memory interaction is not isolated. The live code ablation centers on one model and small samples. It is unknown whether the structural gain persists across weaker and stronger models.
• The real-vault evidence covers two vaults belonging to one user distribution. It does not establish generalization across professions, languages, or collaborative workspaces.
• Retrieval is not final task success. MuSiQue and real-vault primary metrics evaluate evidence retrieval, not every downstream answer and long-horizon action.
• Entity resolution has produced a real over-merge defect. Local pronouns and generic aliases can leak into global identity; without review, the graph can contaminate context.
• Not every triple has exact character-span provenance. Statement receipts honestly degrade to absence, but relation extraction remains an LLM interpretation.
• Revision is bounded. Supersession covers dated decision statements, not a general temporal knowledge base or bitemporal ledger.
• Document and code graphs are physically separate today. Cross-modal retrieval from a written decision to its implementation is a product hypothesis not validated by the code-graph harness.
• Rebuild is not bit-deterministic. Markdown can regenerate the graph, but nondeterministic model extraction means the same quads are not guaranteed.
• Learning lacks a complete fact verifier. Learned notes are source-bearing and inspectable, but every source and claim is not automatically validated.
• The public maintenance-cost record is incomplete. Incremental extraction p50/p95, cost per document, storage growth, and savings against a full re-index still need one controlled table.

10. Next experiments

Turning the central hypothesis from product philosophy into research evidence requires:

1. A model × architecture factorial. Cross several models with raw files, dense RAG, graph RAG, and full lifecycle memory on the same task set.
2. Longitudinal sessions. Seed decisions, corrections, and withdrawals over multiple days; measure current-fact recall, stale-fact suppression, abstention, and evidence accuracy. Use MemoryAgentBench to separate retrieval, test-time learning, long-range understanding, and selective forgetting, and Mem2ActBench to measure actual tool action separately.
3. A dual-representation ablation. Compare Markdown-only, ontology-only, and Markdown+ontology as people capture, revise, and reuse the same information.
4. Correction-loop evaluation. Measure time, steps, and residual contamination from detecting a wrong relation through source correction and re-extraction.
5. A human study. Compare chat+files with Consilience on externalization friction, cognitive load, recovery of prior decisions, and source understanding by both person and agent.
6. Lifecycle economics. Publish latency, tokens, cost, and storage for incremental extraction against a complete rebuild.
7. Memory-contamination attacks. Test whether web, screen, or agent-generated content is incorrectly promoted into factual or procedural memory.

Conclusion

EngramRAG proposes more than retrieving more documents. It changes the unit of RAG from a single query to the full lifecycle of memory.

A person cannot author a perfect prompt on every turn, and a model cannot use context that was never externalized. Consilience places a user-owned, human-editable document substrate between them. The ontology is not a competing source of truth; it is a reversible index beneath the documents, connecting identity, relations, provenance, and revision. Agent output returns to the same surface and can become the memory of later work.

The controlled evidence supports a bounded conclusion: with the model held constant, external structure and context selection can change reliability and cost. It does not yet establish that human externalization is the dominant bottleneck or that Consilience improves human cognition. Those are falsifiable next questions.

The model is not the whole system. Long-horizon intelligence emerges from the product of model capability and a memory substrate that humans and agents can jointly inspect, correct, and reuse.

Measurement record

Area	Primary internal record
Live code-graph ablation	docs/code-graph-eval/PAPER.md, 7 tasks × 3 runs and an independent hard subset
Real-vault retrieval	docs/details/retrieval-ablation-2026-07-26.md, Market/e2e, n=150 each
MuSiQue multi-hop	docs/details/musique-multihop-2026-07-26.md, 100 questions and 1,730 passages
CSLS and scale validation	docs/details/graphrag-seeding-csls.md, docs/details/csls-scale-eval-2026-07.md
Statement and provenance	docs/details/statement-layer-frontier-research-2026-07.md
Ontology limitation audit	docs/details/ontology-graphrag-sota-audit-2026-07-26.md