Context Compaction in LLM Agents: Part 3 - Post-Hoc Compilation

Introduction

In Part 1 and Part 2, we covered compaction methods that all share one assumption: they optimize within a single run. Semantic compression squeezes tokens. Loss-aware pruning drops low-value content. SelfCompact decides when to summarize mid-trajectory. CompactionRL learns a compaction policy for the current episode. ACON compresses agent tool-call traces.

All of these make the current conversation or agent session more efficient. But when the run ends, everything that was not compacted is lost. The next run starts from scratch.

This is a real problem. Agents do not work in isolation. They run repeatedly on the same codebase, the same support queue, the same operational workflows. Run after run:

• decisions get re-debated
• constraints get rediscovered
• preferences get ignored
• failed paths get retried
• useful corrections never become eval or training signal

The question is: can we compile durable context from completed runs so the next agent starts with operating memory instead of a blank slate?

The Core Insight: Post-Hoc Beats Online

Every compaction method we have discussed so far operates online, meaning during or just before inference. The summarizer or pruner runs while the agent is still working. It has to make decisions under uncertainty: it does not know yet which parts of the context will turn out to matter.

Post-hoc compilation flips this. Instead of compressing mid-run, you wait until the run is complete, then decide what was worth keeping. You know the full trajectory. You know the outcome. You know which tool calls led somewhere and which were dead ends.

Think of it as the difference between a journalist live-tweeting an event and a historian writing about it afterward. The journalist has to guess what matters in real time. The historian knows how things turned out.

For reusable context across runs, post-hoc is fundamentally less lossy. You are not guessing what will be relevant. You know what was relevant because you can see the entire trace and its outcome.

This does not mean online compaction is useless. You need it to keep the current run within the context window. But for building durable memory across runs, post-hoc compilation is the right tool.

Lerim: A Context Compiler

Lerim (open-source by Nablo, Apache-2.0) is built around this insight. Lerim is not a live memory store. It is a compiler that runs a post-hoc compilation pass between one agent run and the next.

The diagram above shows the full Lerim pipeline. Agent traces flow in from completed runs, get filtered and extracted through the compiler, and emerge as cited context records linked back to their source sessions. Those records form a context graph that future agents and humans can query before starting work.

The pipeline has three stages:

1. Capture

Lerim reads completed agent sessions. Native adapters handle the common coding agents: Claude Code, Codex CLI, Cursor, and OpenCode. For any other workflow, you can import clean JSONL traces. The agent does not need to know about Lerim while it is working. The capture happens after the run is done.

2. Compile

This is where the post-hoc advantage matters. Lerim uses DSPy-driven extraction to pull durable signal from the completed trace. It looks for:

• Decisions: choices the agent made that future runs should not re-debate
• Constraints: rules or limitations discovered during the run
• Preferences: style or approach choices that should persist
• Facts: stable information about the project, environment, or domain
• Corrections: mistakes made and how they were fixed
• Handoffs: context that needs to travel to the next run or a different agent

Critically, Lerim is selective. Most routine traces produce no durable record at all. If the agent did a straightforward task with no surprises, there is nothing worth keeping. Lerim drops weak and source-derivable signal instead of growing an ever-larger memory.

Every compiled record links back to its source session. This is the evidence boundary. You can always ask why a record exists, and Lerim will point you to the exact run that produced it. This is not just an audit feature. It is what makes the compiled context trustworthy. If a record says “Use DSPy for extraction, not a hand-written prompt chain,” you can open the source run and see exactly why that decision was made.

3. Reuse

Compiled context is served back into the next run through three interfaces:

• MCP server: any MCP-compatible agent can query Lerim for relevant context
• CLI: lerim answer "What decisions exist about caching?" gives you a synthesized response
• Context briefs: compact summaries of accumulated context that serve as a starting point for the next run

The agent does not need to re-discover what the previous run already figured out. It starts with operating memory.

How Lerim Fits the Compaction Landscape

Lerim occupies a different position in the compaction stack than everything in Parts 1 and 2:

• Parts 1-2 methods compress within a single run to keep it within the context window
• Lerim compiles across runs to give the next agent a head start

These are complementary. You would use SelfCompact or CompactionRL to manage the current run’s context window. You would use Lerim to ensure the next run does not start from zero.

LerimBench

Lerim ships with LerimBench, an evaluation framework that measures whether compiled context actually helps the next run. This is a different question from “did the records look good.” It measures downstream impact: does the agent perform better with the compiled context than without it?

This matters because it is easy to write a memory system that produces impressive-looking records that do not actually help. The eval-first approach keeps the compiler honest.

Beyond Coding Agents

Lerim is not limited to coding workflows. It ships with signal profiles for multiple domains:

Profile	Workflow
coding	Repository and coding-agent work (default)
support	Customer support and customer operations
ops	Incident response, operations, and reliability
research	Research, market intelligence, and analysis
compliance	Compliance, legal, regulatory, and policy review

The compiler is workflow-agnostic. The profiles define what signal to extract and what to ignore. A support trace looks for customer constraints, known fixes, and escalation reasons. An incident trace looks for root causes, mitigations, and rejected hypotheses. The same post-hoc compilation pipeline serves all of them.

How Lerim Compares to Live Memory Stores

Tools like Mem0 and A-Mem also provide persistent memory for agents. The key difference is timing.

Mem0 and A-Mem write memory while the agent is working. They are online systems. The agent (or a sidecar process) saves memories during the conversation. This has an advantage: the memory is available within the current session. But it has a cost: the system has to decide what is worth saving before it knows how the story ends.

Mem0’s architecture, shown above, illustrates the online approach. Memories are extracted and stored as the conversation flows. A retrieval layer serves relevant memories back into the current session’s context window. This keeps long conversations coherent without exceeding the context limit.

The benchmark results above show Mem0’s performance on long-term memory tasks. The system achieves strong retrieval accuracy while keeping the context budget low compared to feeding the full conversation history. But note what is being optimized: within-session and near-session coherence, not cross-run compilation.

A-Mem, shown above, takes a different online approach inspired by the Zettelkasten method. The agent itself decides what to remember, creates memory “notes” during the conversation, and links related notes together. This gives the agent more autonomy over its own memory, but still operates under the same constraint: decisions about what to keep are made before the full trajectory is known.

Lerim waits until the run is done. It makes no attempt to help the current session. But for the next session, its compilation is higher-fidelity because it has the full picture.

These are not competing approaches. They solve different problems:

• Online memory (Mem0, A-Mem): helps the current session maintain coherence over a long conversation
• Post-hoc compilation (Lerim): helps the next session start with durable, curated context

In a mature agent stack, you would use both. Online memory keeps the current run on track. Post-hoc compilation ensures the lessons from that run survive into the next one.

Practical Takeaways

If you are building agents that run repeatedly on the same workflow, whether that is coding, support, operations, or research:

1. Post-hoc compilation gives you the biggest cross-run quality win. The same decisions, constraints, and corrections that one agent discovers should not be lost when that run ends. Lerim captures them with evidence.

2. Selectivity is the feature, not the bug. A memory system that saves everything is just a log. Lerim’s value is that it rejects most traces and keeps only durable signal. Less context, not more.

3. Evidence-backed records build trust. Every record cites its source. This means you can audit why the agent “knows” something. It also means you can catch bad records before they propagate.

4. Start with one workflow. Lerim works best when you point it at one repeated workflow and let it accumulate context over several runs. The native adapters make coding agents the fastest path, but the custom JSONL import handles any workflow that can export a trace.

5. Measure whether it helps. LerimBench evaluates downstream impact, not just record quality. If you are building your own context layer, measure the same thing. Records that look good but do not improve the next run are noise.

In Part 4, we step back to look at the theoretical framework that unifies all compaction methods across the stack, from KV cache eviction to agent memory. We also examine a critical safety concern: compaction can silently erase governance constraints, turning a safe agent into an unsafe one.