---
name: compression
description: Compress tool outputs, logs, and JSON arrays before they enter LLM context. Saves 60-95% tokens.
metadata: { "openclaw": { "emoji": "📦" } }
---

# Compression — Context-Aware Output Compression

Compress large tool outputs before they consume LLM context. Content-type-aware: JSON arrays, build logs, search results, git diffs each get optimal compression.

## When to Use

- Tool output > 500 chars → compress before reading
- JSON arrays > 10 items → SmartCrusher
- Build/CI logs > 50 lines → template dedup
- Search/grep results > 30 lines → file dedup
- Git diffs with lock files → noise filter
- Need to retrieve original compressed data → CCR store

## Tools

### `ctx_compress` — one-stop compression

```bash
# Auto-detect type, compress stdin
cat huge_output.json | node src/compression/ctx-compress.js

# Specify type, bias
cat build.log | node src/compression/ctx-compress.js --type log --json

# Force JSON mode with query context for relevance
cat results.json | node src/compression/ctx-compress.js --type json --query "error timeout" --json

# Retrieve original from CCR store
node src/compression/ctx-compress.js --retrieve <ccr_hash>
```

### `smart-crusher` — direct JSON compression

```bash
node src/compression/smart-crusher.js --bias 0.7 --query "search terms" < input.json
```

### Content type detection

```bash
cat unknown_output.txt | node src/compression/detector.js
```

## Compression Strategies

| Content Type | Strategy | Savings |
|-------------|----------|:-------:|
| JSON array | lossless:csv or smart sample | 50-90% |
| Build log | template dedup | 90-99% |
| Search results | file dedup | 80-95% |
| Git diff | noise filter (lock files etc) | 30-70% |

## Key Feature: CCR (Compress-Cache-Retrieve)

Compressed output includes `<<ccr:HASH N_rows_offloaded>>` markers. If LLM needs original data, retrieve it:

```bash
node src/compression/ctx-compress.js --retrieve <hash>
```

## Integration Pattern

```
large_tool_output → ctx_compress → compressed → LLM context
                                        ↓
                                  <<ccr:abc123>>
                                        ↓
                              LLM needs more? → ctx_compress --retrieve abc123
```

## File Paths

- `src/compression/ctx-compress.js` — main entry
- `src/compression/smart-crusher.js` — JSON compression engine
- `src/compression/detector.js` — content type detection
- `src/compression/ccr-store.js` — reversible storage
- `src/compression/ccr-backends.js` — SQLite/Redis persistence
- `src/compression/byte-surgery.js` — cache-safe byte replacement