Context Distillation
The practice of reducing raw CLI or document output to its essential signal before it enters an AI agent’s context window — preserving meaning while eliminating noise.
Why It Matters
LLM context windows are finite and expensive. Every token sent to the agent costs money and reduces the space available for reasoning. Raw terminal output is extremely noisy:
| Command | Typical raw tokens | After distillation |
|---|---|---|
git diff | ~10,000 | ~1,000 |
cargo test | ~25,000 | ~2,500 |
npm install | ~5,000 | ~500 |
Beyond cost, noise causes cognitive dilution — the LLM’s attention spreads across irrelevant lines, degrading the quality of reasoning on what actually matters.
The Signal Lifecycle
- Classify — identify command type and output structure
- Score — boost lines relevant to current session context (active files, recent errors)
- Compose — tier output into signal levels; emit only what clears the threshold
- Archive — store original output in rewind-store so nothing is lost
Relation to LLM Wiki
Context distillation and the llm-wiki-pattern share the same core insight: compile meaning once, not on every query. The wiki pre-compiles document knowledge; distillation pre-compiles terminal output. Both reduce the token cost of getting signal to the agent.
See Also
- omni — OMNI implements this pattern for terminal output
- rewind-store — how original data is preserved despite distillation
- llm-wiki-vs-rag — same principle applied to documents