16gagent/skills/compression.md

name, description, metadata

name	description	metadata
compression	Compress tool outputs, logs, and JSON arrays before they enter LLM context. Saves 60-95% tokens.	openclaw emoji 📦

Compression — Context-Aware Output Compression

Compress large tool outputs before they consume LLM context. Content-type-aware: JSON arrays, build logs, search results, git diffs each get optimal compression.

When to Use

• Tool output > 500 chars → compress before reading
• JSON arrays > 10 items → SmartCrusher
• Build/CI logs > 50 lines → template dedup
• Search/grep results > 30 lines → file dedup
• Git diffs with lock files → noise filter
• Need to retrieve original compressed data → CCR store

Tools

ctx_compress — one-stop compression

# Auto-detect type, compress stdin
cat huge_output.json | node src/compression/ctx-compress.js

# Specify type, bias
cat build.log | node src/compression/ctx-compress.js --type log --json

# Force JSON mode with query context for relevance
cat results.json | node src/compression/ctx-compress.js --type json --query "error timeout" --json

# Retrieve original from CCR store
node src/compression/ctx-compress.js --retrieve <ccr_hash>

smart-crusher — direct JSON compression

node src/compression/smart-crusher.js --bias 0.7 --query "search terms" < input.json

Content type detection

cat unknown_output.txt | node src/compression/detector.js

Compression Strategies

Content Type	Strategy	Savings
JSON array	lossless:csv or smart sample	50-90%
Build log	template dedup	90-99%
Search results	file dedup	80-95%
Git diff	noise filter (lock files etc)	30-70%

Key Feature: CCR (Compress-Cache-Retrieve)

Compressed output includes <<ccr:HASH N_rows_offloaded>> markers. If LLM needs original data, retrieve it:

node src/compression/ctx-compress.js --retrieve <hash>

Integration Pattern

large_tool_output → ctx_compress → compressed → LLM context
                                        ↓
                                  <<ccr:abc123>>
                                        ↓
                              LLM needs more? → ctx_compress --retrieve abc123

File Paths

• src/compression/ctx-compress.js — main entry
• src/compression/smart-crusher.js — JSON compression engine
• src/compression/detector.js — content type detection
• src/compression/ccr-store.js — reversible storage
• src/compression/ccr-backends.js — SQLite/Redis persistence
• src/compression/byte-surgery.js — cache-safe byte replacement