Context Compaction in LLM Agents: Part 4 - Theory and Safety

Introduction

Over the previous three posts, we covered a lot of ground. Part 1 covered external heuristics: semantic compression, loss-aware pruning, and dynamic summarization. Part 2 covered learned methods: SelfCompact’s model-driven compaction, CompactionRL’s reinforcement-learning approach, and ACON’s agent-specific trajectory compression. Part 3 covered post-hoc compilation with Lerim, which compiles durable context across runs.

These methods operate at very different layers of the stack. Some compress tokens. Some compress KV cache entries. Some compress agent trajectories. Some compile across entire sessions. But are they actually doing the same thing?

In this final post, we step back to look at the theoretical framework that unifies all of them. Then we examine a critical safety concern that applies to every compaction method: compaction can silently erase governance constraints, turning a safe agent into an unsafe one.

The Unifying Lens: Rate-Distortion

A recent paper by Colaco and Lahjouji, “What to Keep, What to Forget: A Rate-Distortion View of Memory Compaction in LLMs and Agents” (Jul 2026), makes a compelling argument: all compaction methods are instances of the same problem.

Whether you are evicting KV cache entries, pruning prompt tokens, bounding architectural state, or consolidating agent memory, you are making a rate-distortion decision: what context-derived information to retain versus discard, at what fidelity, under a resource budget, so as to preserve downstream task utility.

The paper builds a seven-axis taxonomy that classifies methods from across the stack uniformly:

1. Layer: where the compaction happens (attention, prompt, architecture, agent)
2. Granularity: what unit is compressed (tokens, sentences, turns, sessions)
3. Signal: what determines what to keep (attention magnitude, perplexity, recency, task relevance)
4. Timing: when the decision is made (pre-inference, mid-generation, post-hoc)
5. Reversibility: can discarded information be recovered
6. Adaptivity: is the policy fixed or learned
7. Budget type: tokens, memory, latency, or cost

Two patterns emerge across this taxonomy.

First, at every layer, the signal that decides what to keep is almost always attention magnitude or recency. KV cache eviction uses attention scores. Prompt pruning uses perplexity (a cousin of attention). Agent summarization uses recency (summarize the oldest stuff first). This makes sense. These signals are cheap to compute and correlate with importance. But they all fail in the same way: they discard information before the query is known, with no way to undo it later.

Second, compression is measured carefully on single-turn long context tasks. But the repeated compaction that agents actually perform, compacting the same memory again and again over many sessions, is almost never measured. No benchmark holds one budget axis constant across all layers of the stack simultaneously.

This is where Lerim’s post-hoc approach (Part 3) and SelfCompact’s rubric (Part 2) become interesting from a theoretical standpoint. Lerim changes the timing axis to post-hoc, which sidesteps the “discard before the query is known” problem. SelfCompact changes the signal axis to task-structure-awareness, which means the model can avoid compacting mid-derivation. Both are improvements that the rate-distortion framework helps us see clearly.

The practical value of this unifying lens is that it lets you transfer techniques between layers. An insight from KV cache management (e.g., attention sinks matter) might inform how you design agent memory. A technique from prompt compression (e.g., budget controllers) might apply to trajectory compaction. When you see all methods as instances of the same optimization problem, cross-pollination becomes natural.

The figure above, from the rate-distortion paper, shows the seven-axis taxonomy in action. Every compaction method, whether it operates on KV cache, prompts, architectural state, or agent memory, can be classified along the same dimensions: layer, granularity, signal, timing, reversibility, adaptivity, and budget type. The taxonomy reveals that methods from completely different communities are solving the same underlying rate-distortion problem, just at different layers and with different design choices.

The Safety Dimension: Governance Decay

There is a darker side to compaction that the research community has only recently started to examine. Governance Decay (Chen, Jun 2026) shows that compaction is not just a performance optimization. It is a safety-critical failure surface.

The setup is simple. You give an agent a governance constraint: “Do not perform action X.” The agent reliably obeys while the constraint is visible in the context window. Then compaction runs, and the constraint is summarized or dropped. The same agent, in the same session, now performs action X.

The numbers are alarming. Across 1,323 episodes and seven model families:

• Violation rate with policy in full context: 0%
• Violation rate after compaction: 30% on average, reaching 59% for some models

The critical finding is that constraint survival determines everything. When the governance constraint survives the summary, violation stays at 0%. When it is dropped, violation reaches 38%. The compaction step is the entire failure mode.

The figure above, from the Governance Decay paper, shows the scale of the problem. Across seven model families and 1,323 episodes, violation rates jump from 0% when the policy is in full context to 30% on average after compaction, with some models reaching 59%. The data is clear: compaction is not just a performance concern, it is a safety-critical failure surface.

The Compaction-Eviction Attack

It gets worse. The paper introduces a Compaction-Eviction Attack: adversarial content placed in the context can bias the summarizer to omit a legitimate policy. An attacker who can inject content into the agent’s context (through a tool result, a document, or a user message) can cause the summarizer to drop a safety constraint it would otherwise preserve.

Optimized injections defeat every evaluated model. This means compaction is not just accidentally lossy. It is actively exploitable.

Constraint Pinning

The paper proposes a training-free mitigation called Constraint Pinning: quarantine governance constraints from lossy compaction entirely. Identify the constraints in the context, extract them, and keep them in a separate region that the compaction step cannot touch.

In the benchmark, constraint pinning restores violation to 0%. The cost is a small number of tokens that are never compressed. But the safety benefit is absolute.

The Connection to Evidence-Backed Context

This is where Lerim’s evidence-boundary design (Part 3) becomes more than a nice feature. Lerim’s compiled records are cited, linked, and inspectable. You can always trace why a record exists and what source produced it.

Contrast this with raw summarization. When a summarizer drops a constraint, there is no trace of what was removed. No log entry. No audit trail. The constraint simply vanishes. The agent has no way to know it ever existed.

Evidence-backed compilation does not solve governance decay by itself. A compiled record can still be wrong or incomplete. But it gives you auditability. You can review compiled records, check whether governance constraints survived, and fix the pipeline when they did not. With raw summarization, you are flying blind.

Open Problems

The rate-distortion survey and the governance decay paper reveal several problems that the field has not yet solved:

When to compact is still mostly heuristic

Even SelfCompact’s rubric is hand-crafted. CompactionRL learns the policy, but only within the distribution it was trained on. Nobody has a principled, general-purpose answer to “when should the model compact?” that adapts to new tasks without manual tuning.

Repeated compaction drift is almost never measured

Most benchmarks evaluate single-shot compaction: compress once, measure quality. But agents compact the same memory repeatedly. Each compaction step loses a little more information. Over many cycles, this drift compounds. The rate-distortion paper notes that this repeated-compaction scenario has no standard benchmark. If you are building a production system, you should measure it yourself.

No benchmark spans all layers

Each community (KV cache, prompt compression, agent memory) has its own benchmarks. Nobody holds one budget axis constant across all layers. This makes it impossible to compare, say, whether spending your token budget on KV cache compression or prompt pruning gives you more quality.

Post-hoc and online compaction are not combined

Lerim’s post-hoc approach is high-fidelity for cross-run context but cannot help the current run. SelfCompact and CompactionRL help within the current run but do not carry context across runs. A system that combines both, online compaction within the run and post-hoc compilation across runs, is an open and promising direction.

Series Wrap-Up

Over four posts, we traced the evolution of context compaction:

1. Part 1 (Fundamentals): External heuristics that compress context from the outside. Semantic compression, loss-aware pruning, and dynamic summarization. These are the workhorses deployed in production today.

2. Part 2 (Learning to Compact): Methods that let the model itself decide when and how to compress. SelfCompact’s rubric-driven approach, CompactionRL’s learned policy, and ACON’s agent-specific trajectory compression. The frontier has moved from external heuristics to model-level capabilities.

3. Part 3 (Post-Hoc Compilation): Lerim’s approach of compiling durable context after a run completes, serving it to the next run as cited operating memory. This solves the cross-run problem that within-run methods cannot touch.

4. Part 4 (Theory and Safety): The rate-distortion framework that unifies all compaction methods across the stack, and the governance decay problem that shows compaction is a safety-critical surface.

The frontier has moved from “how do we compress?” to “what should we keep, and can we prove it?” The best practical stack today combines three layers:

• Within-run compaction: SelfCompact-style rubrics for deciding when to summarize during the current run
• Cross-run compilation: Lerim-style post-hoc compilation for building durable, cited context across runs
• Safety: constraint pinning to quarantine governance constraints from lossy compaction

If you are building agents that run for hours or days, across multiple sessions, on real workflows, all three layers matter. Skipping any one of them costs you either efficiency, quality, or safety.

Thanks for reading this series. If you found it valuable, I appreciate reposts and shares so more people can discover it.