Initial import of NousResearch/hermes-agent
Deploy Site / deploy-vercel (push) Has been cancelled
Deploy Site / deploy-docs (push) Has been cancelled
Docker / shell lint / Lint Dockerfile (hadolint) (push) Has been cancelled
Docker / shell lint / Lint docker/ shell scripts (shellcheck) (push) Has been cancelled
Docker Build and Publish / build-amd64 (push) Has been cancelled
Docker Build and Publish / build-arm64 (push) Has been cancelled
Lint (ruff + ty) / ruff + ty diff (push) Has been cancelled
Lint (ruff + ty) / ruff enforcement (blocking) (push) Has been cancelled
Lint (ruff + ty) / Windows footguns (blocking) (push) Has been cancelled
Nix Lockfile Fix / auto-fix-main (push) Has been cancelled
Nix Lockfile Fix / fix (push) Has been cancelled
Nix / nix (macos-latest) (push) Has been cancelled
Nix / nix (ubuntu-latest) (push) Has been cancelled
OSV-Scanner / Scan lockfiles (push) Has been cancelled
Build Skills Index / build-index (push) Has been cancelled
Tests / test (1) (push) Has been cancelled
Tests / test (2) (push) Has been cancelled
Tests / test (3) (push) Has been cancelled
Tests / test (4) (push) Has been cancelled
Tests / test (5) (push) Has been cancelled
Tests / test (6) (push) Has been cancelled
Tests / e2e (push) Has been cancelled
uv.lock check / uv lock --check (push) Has been cancelled
Docker Build and Publish / merge (push) Has been cancelled
Build Skills Index / trigger-deploy (push) Has been cancelled
Tests / save-durations (push) Has been cancelled
Deploy Site / deploy-vercel (push) Has been cancelled
Deploy Site / deploy-docs (push) Has been cancelled
Docker / shell lint / Lint Dockerfile (hadolint) (push) Has been cancelled
Docker / shell lint / Lint docker/ shell scripts (shellcheck) (push) Has been cancelled
Docker Build and Publish / build-amd64 (push) Has been cancelled
Docker Build and Publish / build-arm64 (push) Has been cancelled
Lint (ruff + ty) / ruff + ty diff (push) Has been cancelled
Lint (ruff + ty) / ruff enforcement (blocking) (push) Has been cancelled
Lint (ruff + ty) / Windows footguns (blocking) (push) Has been cancelled
Nix Lockfile Fix / auto-fix-main (push) Has been cancelled
Nix Lockfile Fix / fix (push) Has been cancelled
Nix / nix (macos-latest) (push) Has been cancelled
Nix / nix (ubuntu-latest) (push) Has been cancelled
OSV-Scanner / Scan lockfiles (push) Has been cancelled
Build Skills Index / build-index (push) Has been cancelled
Tests / test (1) (push) Has been cancelled
Tests / test (2) (push) Has been cancelled
Tests / test (3) (push) Has been cancelled
Tests / test (4) (push) Has been cancelled
Tests / test (5) (push) Has been cancelled
Tests / test (6) (push) Has been cancelled
Tests / e2e (push) Has been cancelled
uv.lock check / uv lock --check (push) Has been cancelled
Docker Build and Publish / merge (push) Has been cancelled
Build Skills Index / trigger-deploy (push) Has been cancelled
Tests / save-durations (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,24 @@
|
||||
# Optional Skills
|
||||
|
||||
Official skills maintained by Nous Research that are **not activated by default**.
|
||||
|
||||
These skills ship with the hermes-agent repository but are not copied to
|
||||
`~/.hermes/skills/` during setup. They are discoverable via the Skills Hub:
|
||||
|
||||
```bash
|
||||
hermes skills browse # browse all skills, official shown first
|
||||
hermes skills browse --source official # browse only official optional skills
|
||||
hermes skills search <query> # finds optional skills labeled "official"
|
||||
hermes skills install <identifier> # copies to ~/.hermes/skills/ and activates
|
||||
```
|
||||
|
||||
## Why optional?
|
||||
|
||||
Some skills are useful but not broadly needed by every user:
|
||||
|
||||
- **Niche integrations** — specific paid services, specialized tools
|
||||
- **Experimental features** — promising but not yet proven
|
||||
- **Heavyweight dependencies** — require significant setup (API keys, installs)
|
||||
|
||||
By keeping them optional, we keep the default skill set lean while still
|
||||
providing curated, tested, official skills for users who want them.
|
||||
@@ -0,0 +1,2 @@
|
||||
Optional autonomous AI agent integrations — external coding agent CLIs
|
||||
that can be delegated to for independent coding tasks.
|
||||
@@ -0,0 +1,177 @@
|
||||
---
|
||||
name: antigravity-cli
|
||||
description: "Operate the Antigravity CLI (agy): plugins, auth, sandbox."
|
||||
version: 0.1.0
|
||||
author: Tony Simons (asimons81), Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Coding-Agent, Antigravity, CLI, Auth, Plugins, Sandbox]
|
||||
related_skills: [grok, codex, claude-code, hermes-agent]
|
||||
---
|
||||
|
||||
# Antigravity CLI (`agy`)
|
||||
|
||||
Operator guide for the Antigravity CLI, invoked as `agy`. Run all `agy`
|
||||
commands through the Hermes `terminal` tool; inspect its config and logs with
|
||||
`read_file`. This skill is reference + procedure — it does not wrap a network
|
||||
API, so there is nothing to authenticate from Hermes itself.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Installing, updating, or smoke-testing the `agy` binary
|
||||
- Driving non-interactive `agy --print` / `agy -p` one-shots
|
||||
- Debugging Antigravity auth, sandbox, permissions, or plugin state
|
||||
- Reading Antigravity settings, keybindings, conversations, or logs
|
||||
|
||||
## Mental model
|
||||
|
||||
Antigravity has two layers — keep them distinct or the guidance will be wrong:
|
||||
|
||||
1. **Shell wrapper commands** — `agy help`, `agy install`, `agy plugin`,
|
||||
`agy update`, `agy changelog`. Run these through the `terminal` tool.
|
||||
2. **Interactive in-session slash commands** — `/config`, `/permissions`,
|
||||
`/skills`, `/agents`, etc. These only exist inside a running `agy` TUI
|
||||
session, not on the shell wrapper.
|
||||
|
||||
`agy help` shows the shell wrapper surface, NOT the in-session slash commands.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- The `agy` binary on PATH. Verify through the `terminal` tool:
|
||||
`command -v agy && agy --version`.
|
||||
- No env vars or API keys required by this skill — Antigravity manages its own
|
||||
auth via the OS keyring / browser sign-in (see Authentication below).
|
||||
|
||||
## How to Run
|
||||
|
||||
Invoke every `agy` command through the `terminal` tool. Examples:
|
||||
|
||||
```
|
||||
terminal(command="agy --version")
|
||||
terminal(command="agy help")
|
||||
terminal(command="agy plugin list")
|
||||
terminal(command="agy --print 'Summarize the repo in 3 bullets'", workdir="/path/to/project")
|
||||
```
|
||||
|
||||
For an interactive multi-turn TUI session, launch `agy` with `pty=true` (and
|
||||
tmux for capture/monitoring), the same pattern the `codex` / `claude-code`
|
||||
skills use. For one-shot smoke tests and scripted prompts, prefer
|
||||
`agy --print` (non-interactive).
|
||||
|
||||
To inspect Antigravity's own files, use `read_file` on the paths under Core
|
||||
paths below — do not `cat` them through the terminal.
|
||||
|
||||
## Core paths
|
||||
|
||||
- Binary / entrypoint: `agy`
|
||||
- App data dir: `~/.gemini/antigravity-cli/`
|
||||
- Settings file: `~/.gemini/antigravity-cli/settings.json`
|
||||
- Keybindings file: `~/.gemini/antigravity-cli/keybindings.json`
|
||||
- Logs: `~/.gemini/antigravity-cli/log/cli-*.log`
|
||||
- Conversations: `~/.gemini/antigravity-cli/conversations/`
|
||||
- Brain artifacts: `~/.gemini/antigravity-cli/brain/`
|
||||
- History: `~/.gemini/antigravity-cli/history.jsonl`
|
||||
- Plugin staging: `~/.gemini/antigravity-cli/plugins/<plugin_name>/`
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Wrapper commands
|
||||
- `agy changelog`
|
||||
- `agy help`
|
||||
- `agy install`
|
||||
- `agy plugin` / `agy plugins`
|
||||
- `agy update`
|
||||
|
||||
### Useful flags
|
||||
- `--add-dir`
|
||||
- `--continue` / `-c`
|
||||
- `--conversation`
|
||||
- `--dangerously-skip-permissions`
|
||||
- `--print` / `-p`
|
||||
- `--print-timeout`
|
||||
- `--prompt`
|
||||
- `--prompt-interactive` / `-i`
|
||||
- `--sandbox`
|
||||
- `--log-file`
|
||||
- `--version`
|
||||
|
||||
### Plugin subcommands (`agy plugin --help`)
|
||||
- `list`, `import [source]`, `install <target>`, `uninstall <name>`,
|
||||
`enable <name>`, `disable <name>`, `validate [path]`, `link <mp> <target>`,
|
||||
`help`
|
||||
|
||||
### Install flags (`agy install --help`)
|
||||
- `--dir`, `--skip-aliases`, `--skip-path`
|
||||
|
||||
### In-session slash commands
|
||||
- **Conversation control:** `/resume` (`/switch`), `/rewind` (`/undo`),
|
||||
`/rename <name>`, `/clear`, `/fork`, `/reset`, `/new`
|
||||
- **Settings & tools:** `/config`, `/settings`, `/permissions`, `/model`,
|
||||
`/keybindings`, `/statusline`, `/tasks`, `/skills`, `/mcp`, `/open <path>`,
|
||||
`/usage`, `/logout`, `/agents`
|
||||
- **Prompt helpers:** `@` path autocomplete, `esc esc` clears the prompt (when
|
||||
not streaming), `!` runs a terminal command directly, `?` opens help
|
||||
|
||||
## Settings and permissions
|
||||
|
||||
### Common settings keys (`settings.json`)
|
||||
- `allowNonWorkspaceAccess`
|
||||
- `colorScheme`
|
||||
- `permissions.allow`
|
||||
- `trustedWorkspaces`
|
||||
|
||||
### Permission modes
|
||||
`request-review`, `always-proceed`, `strict`, `proceed-in-sandbox`.
|
||||
|
||||
### Sandbox behavior
|
||||
- `enableTerminalSandbox` is a boolean in `settings.json`; default `false`.
|
||||
- Launch-time overrides (`--sandbox`, `--dangerously-skip-permissions`) can
|
||||
supersede persistent settings for the current session.
|
||||
|
||||
## Authentication behavior
|
||||
|
||||
- The CLI tries the OS secure keyring first.
|
||||
- With no saved session, it falls back to browser-based Google sign-in.
|
||||
- Locally it opens the default browser; over SSH it prints an authorization URL
|
||||
and expects the auth code pasted back.
|
||||
- `/logout` removes saved credentials.
|
||||
|
||||
## Plugins
|
||||
|
||||
- Plugins stage under `~/.gemini/antigravity-cli/plugins/<plugin_name>/`.
|
||||
- They can bundle skills, agents, rules, MCP servers, and hooks.
|
||||
- `agy plugin list` returning no imported plugins is a valid empty state.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- `agy help` shows wrapper commands, not interactive slash commands.
|
||||
- `agy --version` is the safe non-interactive version check; `agy version` is
|
||||
interactive and can fail without a real TTY.
|
||||
- First place to look for failures: `~/.gemini/antigravity-cli/log/cli-*.log`
|
||||
(read with `read_file`).
|
||||
- Don't confuse persistent JSON settings with launch-time overrides.
|
||||
- `~/.gemini/antigravity-cli/bin/agentapi` is a thin wrapper to `agy agentapi`.
|
||||
- On WSL, token storage is file-based, so auth issues are usually local-file /
|
||||
session-state problems, not browser-only problems.
|
||||
- Workspace identity can depend on launch directory and the `.antigravitycli`
|
||||
project marker.
|
||||
|
||||
## Verification
|
||||
|
||||
Confirm the install is real and usable, all through the `terminal` tool (read
|
||||
files with `read_file`):
|
||||
|
||||
1. `terminal(command="command -v agy")`
|
||||
2. `terminal(command="agy --version")`
|
||||
3. `terminal(command="agy help")`
|
||||
4. `terminal(command="agy plugin list")`
|
||||
5. `read_file` on `~/.gemini/antigravity-cli/settings.json`
|
||||
6. `read_file` on the latest `~/.gemini/antigravity-cli/log/cli-*.log`
|
||||
7. If needed, `read_file` on `~/.gemini/antigravity-cli/keybindings.json`
|
||||
|
||||
## Support files
|
||||
|
||||
- `references/cli-docs.md` — condensed notes from the getting-started, usage,
|
||||
and features docs.
|
||||
@@ -0,0 +1,64 @@
|
||||
# Antigravity CLI docs, condensed
|
||||
|
||||
Source pages reviewed:
|
||||
- `/docs/cli-getting-started`
|
||||
- `/docs/cli-using`
|
||||
- `/docs/cli-features`
|
||||
|
||||
## Install
|
||||
- macOS/Linux: `curl -fsSL https://antigravity.google/cli/install.sh | bash`
|
||||
- Windows PowerShell: `irm https://antigravity.google/cli/install.ps1 | iex`
|
||||
- Windows CMD: `curl -fsSL https://antigravity.google/cli/install.cmd -o install.cmd && install.cmd && del install.cmd`
|
||||
|
||||
## Authentication
|
||||
- Tries secure keyring first.
|
||||
- If no saved session exists, falls back to browser-based Google sign-in.
|
||||
- Local machine: opens the default browser.
|
||||
- SSH/remote: prints a secure authorization URL, then expects the auth code to be pasted back.
|
||||
- `/logout` removes saved credentials.
|
||||
|
||||
## Config and files
|
||||
- Settings: `~/.gemini/antigravity-cli/settings.json`
|
||||
- Keybindings: `~/.gemini/antigravity-cli/keybindings.json`
|
||||
- Plugins: `~/.gemini/antigravity-cli/plugins/<plugin_name>/`
|
||||
|
||||
## Useful slash commands
|
||||
- `/config`, `/settings`
|
||||
- `/permissions`
|
||||
- `/resume` / `/switch`
|
||||
- `/rewind` / `/undo`
|
||||
- `/rename <name>`
|
||||
- `/model`
|
||||
- `/keybindings`
|
||||
- `/statusline`
|
||||
- `/tasks`
|
||||
- `/skills`
|
||||
- `/mcp`
|
||||
- `/open <path>`
|
||||
- `/usage`
|
||||
- `/logout`
|
||||
- `/agents`
|
||||
|
||||
## Prompt helpers
|
||||
- `@` path autocomplete
|
||||
- `esc esc` clears prompt when not streaming
|
||||
- `!` runs a terminal command
|
||||
- `?` opens help / slash command list
|
||||
|
||||
## Permissions and sandbox
|
||||
- Permission modes: `request-review`, `always-proceed`, `strict`, `proceed-in-sandbox`
|
||||
- Launch overrides: `--sandbox`, `--dangerously-skip-permissions`
|
||||
- Sandbox setting: `enableTerminalSandbox` in `settings.json` (default `false`)
|
||||
|
||||
## Plugins
|
||||
- Plugins can bundle skills, agents, rules, MCP servers, and hooks.
|
||||
- They are staged locally and auto-discovered once installed.
|
||||
|
||||
## Subagents
|
||||
- `/agents` opens the panel for active/completed subagents.
|
||||
- Subagents can run in parallel and request approvals.
|
||||
|
||||
## Keybindings
|
||||
- `~/.gemini/antigravity-cli/keybindings.json`
|
||||
- Malformed JSON falls back to defaults for broken actions.
|
||||
- Docs list default bindings for clear, submit, cancel, exit, suspend, editor, approval yes/no, navigation, clipboard, undo/redo, and newline insertion.
|
||||
@@ -0,0 +1,144 @@
|
||||
---
|
||||
name: blackbox
|
||||
description: Delegate coding tasks to Blackbox AI CLI agent. Multi-model agent with built-in judge that runs tasks through multiple LLMs and picks the best result. Requires the blackbox CLI and a Blackbox AI API key.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent (Nous Research)
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Coding-Agent, Blackbox, Multi-Agent, Judge, Multi-Model]
|
||||
related_skills: [claude-code, codex, hermes-agent]
|
||||
---
|
||||
|
||||
# Blackbox CLI
|
||||
|
||||
Delegate coding tasks to [Blackbox AI](https://www.blackbox.ai/) via the Hermes terminal. Blackbox is a multi-model coding agent CLI that dispatches tasks to multiple LLMs (Claude, Codex, Gemini, Blackbox Pro) and uses a judge to select the best implementation.
|
||||
|
||||
The CLI is [open-source](https://github.com/blackboxaicode/cli) (GPL-3.0, TypeScript, forked from Gemini CLI) and supports interactive sessions, non-interactive one-shots, checkpointing, MCP, and vision model switching.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 20+ installed
|
||||
- Blackbox CLI installed: `npm install -g @blackboxai/cli`
|
||||
- Or install from source:
|
||||
```
|
||||
git clone https://github.com/blackboxaicode/cli.git
|
||||
cd cli && npm install && npm install -g .
|
||||
```
|
||||
- API key from [app.blackbox.ai/dashboard](https://app.blackbox.ai/dashboard)
|
||||
- Configured: run `blackbox configure` and enter your API key
|
||||
- Use `pty=true` in terminal calls — Blackbox CLI is an interactive terminal app
|
||||
|
||||
## One-Shot Tasks
|
||||
|
||||
```
|
||||
terminal(command="blackbox --prompt 'Add JWT authentication with refresh tokens to the Express API'", workdir="/path/to/project", pty=true)
|
||||
```
|
||||
|
||||
For quick scratch work:
|
||||
```
|
||||
terminal(command="cd $(mktemp -d) && git init && blackbox --prompt 'Build a REST API for todos with SQLite'", pty=true)
|
||||
```
|
||||
|
||||
## Background Mode (Long Tasks)
|
||||
|
||||
For tasks that take minutes, use background mode so you can monitor progress:
|
||||
|
||||
```
|
||||
# Start in background with PTY
|
||||
terminal(command="blackbox --prompt 'Refactor the auth module to use OAuth 2.0'", workdir="~/project", background=true, pty=true)
|
||||
# Returns session_id
|
||||
|
||||
# Monitor progress
|
||||
process(action="poll", session_id="<id>")
|
||||
process(action="log", session_id="<id>")
|
||||
|
||||
# Send input if Blackbox asks a question
|
||||
process(action="submit", session_id="<id>", data="yes")
|
||||
|
||||
# Kill if needed
|
||||
process(action="kill", session_id="<id>")
|
||||
```
|
||||
|
||||
## Checkpoints & Resume
|
||||
|
||||
Blackbox CLI has built-in checkpoint support for pausing and resuming tasks:
|
||||
|
||||
```
|
||||
# After a task completes, Blackbox shows a checkpoint tag
|
||||
# Resume with a follow-up task:
|
||||
terminal(command="blackbox --resume-checkpoint 'task-abc123-2026-03-06' --prompt 'Now add rate limiting to the endpoints'", workdir="~/project", pty=true)
|
||||
```
|
||||
|
||||
## Session Commands
|
||||
|
||||
During an interactive session, use these commands:
|
||||
|
||||
| Command | Effect |
|
||||
|---------|--------|
|
||||
| `/compress` | Shrink conversation history to save tokens |
|
||||
| `/clear` | Wipe history and start fresh |
|
||||
| `/stats` | View current token usage |
|
||||
| `Ctrl+C` | Cancel current operation |
|
||||
|
||||
## PR Reviews
|
||||
|
||||
Clone to a temp directory to avoid modifying the working tree:
|
||||
|
||||
```
|
||||
terminal(command="REVIEW=$(mktemp -d) && git clone https://github.com/user/repo.git $REVIEW && cd $REVIEW && gh pr checkout 42 && blackbox --prompt 'Review this PR against main. Check for bugs, security issues, and code quality.'", pty=true)
|
||||
```
|
||||
|
||||
## Parallel Work
|
||||
|
||||
Spawn multiple Blackbox instances for independent tasks:
|
||||
|
||||
```
|
||||
terminal(command="blackbox --prompt 'Fix the login bug'", workdir="/tmp/issue-1", background=true, pty=true)
|
||||
terminal(command="blackbox --prompt 'Add unit tests for auth'", workdir="/tmp/issue-2", background=true, pty=true)
|
||||
|
||||
# Monitor all
|
||||
process(action="list")
|
||||
```
|
||||
|
||||
## Multi-Model Mode
|
||||
|
||||
Blackbox's unique feature is running the same task through multiple models and judging the results. Configure which models to use via `blackbox configure` — select multiple providers to enable the Chairman/judge workflow where the CLI evaluates outputs from different models and picks the best one.
|
||||
|
||||
## Key Flags
|
||||
|
||||
| Flag | Effect |
|
||||
|------|--------|
|
||||
| `--prompt "task"` | Non-interactive one-shot execution |
|
||||
| `--resume-checkpoint "tag"` | Resume from a saved checkpoint |
|
||||
| `--yolo` | Auto-approve all actions and model switches |
|
||||
| `blackbox session` | Start interactive chat session |
|
||||
| `blackbox configure` | Change settings, providers, models |
|
||||
| `blackbox info` | Display system information |
|
||||
|
||||
## Vision Support
|
||||
|
||||
Blackbox automatically detects images in input and can switch to multimodal analysis. VLM modes:
|
||||
- `"once"` — Switch model for current query only
|
||||
- `"session"` — Switch for entire session
|
||||
- `"persist"` — Stay on current model (no switch)
|
||||
|
||||
## Token Limits
|
||||
|
||||
Control token usage via `.blackboxcli/settings.json`:
|
||||
```json
|
||||
{
|
||||
"sessionTokenLimit": 32000
|
||||
}
|
||||
```
|
||||
|
||||
## Rules
|
||||
|
||||
1. **Always use `pty=true`** — Blackbox CLI is an interactive terminal app and will hang without a PTY
|
||||
2. **Use `workdir`** — keep the agent focused on the right directory
|
||||
3. **Background for long tasks** — use `background=true` and monitor with `process` tool
|
||||
4. **Don't interfere** — monitor with `poll`/`log`, don't kill sessions because they're slow
|
||||
5. **Report results** — after completion, check what changed and summarize for the user
|
||||
6. **Credits cost money** — Blackbox uses a credit-based system; multi-model mode consumes credits faster
|
||||
7. **Check prerequisites** — verify `blackbox` CLI is installed before attempting delegation
|
||||
@@ -0,0 +1,301 @@
|
||||
---
|
||||
name: grok
|
||||
description: "Delegate coding to xAI Grok Build CLI (features, PRs)."
|
||||
version: 0.1.0
|
||||
author: Matt Maximo (MattMaximo), Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Coding-Agent, Grok, xAI, Code-Review, Refactoring, Automation]
|
||||
related_skills: [codex, claude-code, hermes-agent]
|
||||
---
|
||||
|
||||
# Grok Build CLI — Hermes Orchestration Guide
|
||||
|
||||
Delegate coding tasks to [Grok Build](https://docs.x.ai/build/overview) (xAI's
|
||||
autonomous coding agent CLI, the `grok` command) via the Hermes terminal. Grok
|
||||
can read files, write code, run shell commands, spawn subagents, and manage git
|
||||
workflows. It runs three ways: an interactive TUI, **headless** (`-p`), and as
|
||||
an **ACP agent** over JSON-RPC.
|
||||
|
||||
This is the third sibling to `codex` and `claude-code`. The orchestration
|
||||
pattern is nearly identical — **prefer headless `-p` for one-shots**, use a PTY
|
||||
for interactive sessions.
|
||||
|
||||
## When to use
|
||||
|
||||
- Building features
|
||||
- Refactoring
|
||||
- PR reviews
|
||||
- Batch issue fixing
|
||||
- Any task where you'd otherwise reach for Codex / Claude Code but want Grok
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Install (preferred):** `npm install -g @xai-official/grok`
|
||||
- The official installer `curl -fsSL https://x.ai/cli/install.sh | bash` also
|
||||
works, but the `x.ai` host is Cloudflare-walled in some environments. The
|
||||
npm path avoids that dependency entirely.
|
||||
- **Auth — SuperGrok / X Premium+ subscription (primary path):**
|
||||
- Run `grok login` once → opens a browser for OAuth → token cached in
|
||||
`~/.grok/auth.json`. This uses your **SuperGrok or X Premium+** subscription
|
||||
(no per-token API billing).
|
||||
- Check sign-in state by looking for `~/.grok/auth.json`, or run a cheap
|
||||
headless smoke test: `grok --no-auto-update -p "Say ok."`
|
||||
- In the TUI, `/logout` signs out and `/login` (or relaunching) signs back in.
|
||||
- **No git repo required** — unlike Codex, Grok runs fine outside a git
|
||||
directory (good for scratch/throwaway tasks).
|
||||
- **Claude Code / AGENTS.md compatible with zero config** — Grok auto-reads
|
||||
`CLAUDE.md`, `.claude/` (skills, agents, MCPs, hooks, rules), and the
|
||||
`AGENTS.md` family. Existing project context just works.
|
||||
|
||||
> **API-key fallback (not the default for this user):** Grok also supports
|
||||
> setting the `XAI_API_KEY` environment variable for pay-as-you-go billing
|
||||
> via `api.x.ai`. Only use
|
||||
> this if `grok login` / SuperGrok auth is unavailable. The subscription path
|
||||
> (`grok login`) is the intended setup here.
|
||||
|
||||
## Two Orchestration Modes
|
||||
|
||||
### Mode 1: Headless (`-p`) — Non-Interactive (PREFERRED)
|
||||
|
||||
Runs a one-shot task, prints the result, and exits. No PTY, no interactive
|
||||
dialogs to navigate. This is the cleanest integration path — the analog of
|
||||
`claude -p` and `codex exec`.
|
||||
|
||||
```
|
||||
terminal(command="grok --no-auto-update -p 'Add a dark mode toggle to settings'", workdir="/path/to/project", timeout=180)
|
||||
```
|
||||
|
||||
Always pass `--no-auto-update` in automation to skip background update checks.
|
||||
|
||||
**When to use headless:**
|
||||
- One-shot coding tasks (fix a bug, add a feature, refactor)
|
||||
- CI/CD automation and scripting
|
||||
- Structured output parsing with `--output-format json`
|
||||
- Any task that doesn't need multi-turn conversation
|
||||
|
||||
### Mode 2: Interactive PTY — Multi-Turn TUI Sessions
|
||||
|
||||
The TUI is a fullscreen, mouse-interactive app. Drive it with `pty=true`. For
|
||||
robust monitoring/input use tmux (same pattern as the `claude-code` skill).
|
||||
|
||||
```
|
||||
# Launch in a tmux session for capture-pane monitoring
|
||||
terminal(command="tmux new-session -d -s grok-work -x 140 -y 40")
|
||||
terminal(command="tmux send-keys -t grok-work 'cd /path/to/project && grok' Enter")
|
||||
|
||||
# Wait for startup, then send a task
|
||||
terminal(command="sleep 5 && tmux send-keys -t grok-work 'Refactor the auth module to use JWT' Enter")
|
||||
|
||||
# Monitor progress
|
||||
terminal(command="sleep 15 && tmux capture-pane -t grok-work -p -S -50")
|
||||
|
||||
# Exit when done
|
||||
terminal(command="tmux send-keys -t grok-work '/quit' Enter && sleep 1 && tmux kill-session -t grok-work")
|
||||
```
|
||||
|
||||
**Tip for headless-but-inline output:** if you want TUI-style output without the
|
||||
fullscreen alt-screen takeover (e.g. for cleaner logs), add `--no-alt-screen`.
|
||||
For pure automation, headless `-p` is still cleaner than the TUI.
|
||||
|
||||
## Headless Deep Dive
|
||||
|
||||
### Common Flags
|
||||
|
||||
| Flag | Effect |
|
||||
|------|--------|
|
||||
| `-p, --single <PROMPT>` | Send one prompt, run headless, exit |
|
||||
| `-m, --model <MODEL>` | Choose a model |
|
||||
| `-s, --session-id <ID>` | Create or resume a named headless session |
|
||||
| `-r, --resume <ID>` | Resume an existing session |
|
||||
| `-c, --continue` | Continue the most recent session in the current directory |
|
||||
| `--cwd <PATH>` | Set the working directory |
|
||||
| `--output-format <FMT>` | `plain` (default), `json`, or `streaming-json` |
|
||||
| `--always-approve` | Auto-approve all tool executions (the `--full-auto` / `--yolo` equivalent) |
|
||||
| `--no-alt-screen` | Run inline, no fullscreen TUI takeover |
|
||||
| `--no-auto-update` | Skip background update checks (use in all automation) |
|
||||
|
||||
### Output Formats
|
||||
|
||||
- `plain` — human-readable text (default)
|
||||
- `json` — one JSON object at the end of the run (parse the result cleanly)
|
||||
- `streaming-json` — newline-delimited JSON events as they arrive
|
||||
|
||||
```
|
||||
# Structured result for parsing
|
||||
terminal(command="grok --no-auto-update -p 'List all TODO comments in src/' --output-format json", workdir="/project", timeout=120)
|
||||
|
||||
# Auto-approve for autonomous building
|
||||
terminal(command="grok --no-auto-update --always-approve -p 'Refactor the database layer and run the tests'", workdir="/project", timeout=300)
|
||||
```
|
||||
|
||||
### Background Mode (Long Tasks)
|
||||
|
||||
```
|
||||
# Start headless in background
|
||||
terminal(command="grok --no-auto-update --always-approve -p 'Refactor the auth module'", workdir="/project", background=true, notify_on_complete=true)
|
||||
# Returns session_id
|
||||
|
||||
# Monitor
|
||||
process(action="poll", session_id="<id>")
|
||||
process(action="log", session_id="<id>")
|
||||
|
||||
# Kill if needed
|
||||
process(action="kill", session_id="<id>")
|
||||
```
|
||||
|
||||
For an interactive (TUI) background session, use `pty=true` + tmux and monitor
|
||||
with `tmux capture-pane`, exactly like the `claude-code` / `codex` skills.
|
||||
|
||||
### Session Continuation
|
||||
|
||||
```
|
||||
# Start a named session
|
||||
terminal(command="grok --no-auto-update -s refactor-db -p 'Start refactoring the database layer' --always-approve", workdir="/project", timeout=240)
|
||||
|
||||
# Resume it later
|
||||
terminal(command="grok --no-auto-update -r refactor-db -p 'Now add connection pooling' --always-approve", workdir="/project", timeout=180)
|
||||
|
||||
# Or continue the most recent session in this directory
|
||||
terminal(command="grok --no-auto-update -c -p 'What did you change last time?'", workdir="/project", timeout=60)
|
||||
```
|
||||
|
||||
## Read-Only Audit → Markdown Note Pattern
|
||||
|
||||
To have Grok review local artifacts and return a clean markdown note (for
|
||||
Obsidian or a repo) without mutating anything:
|
||||
|
||||
1. Prepare stable input files first with Hermes tools (`read_file`,
|
||||
`write_file`). Snapshot only the relevant context into a temp file rather
|
||||
than dumping raw paths.
|
||||
2. Run Grok headless **without** `--always-approve` so it cannot auto-write, and
|
||||
demand `markdown only, no preamble`.
|
||||
3. Save Grok's stdout straight into the destination note with `write_file()`.
|
||||
|
||||
```
|
||||
grok --no-auto-update -p "Read /tmp/current.md and /tmp/inventory.md. Produce markdown only, no preamble. Output a clean note titled 'Cleanup Review'." --output-format plain
|
||||
```
|
||||
|
||||
**Pitfall (same as Claude Code):** for document rewrites, a loose "rewrite this"
|
||||
prompt may return a change summary instead of the full file. Instead: pipe the
|
||||
file in, and demand `Return ONLY the full revised markdown document. No intro,
|
||||
no explanation, no code fences. Start immediately with '# Title'.` Verify the
|
||||
first lines with `read_file()` before overwriting the destination.
|
||||
|
||||
## PR Review Patterns
|
||||
|
||||
### Quick Review (Headless)
|
||||
|
||||
```
|
||||
terminal(command="cd /path/to/repo && git diff main...feature-branch | grok --no-auto-update -p 'Review this diff for bugs, security issues, and style problems. Be thorough.'", timeout=120)
|
||||
```
|
||||
|
||||
### Clone-to-temp Review (safe, no repo mutation)
|
||||
|
||||
```
|
||||
terminal(command="REVIEW=$(mktemp -d) && git clone https://github.com/user/repo.git $REVIEW && cd $REVIEW && gh pr checkout 42 && grok --no-auto-update -p 'Review the changes vs origin/main. Check bugs, security, race conditions, missing tests.'", pty=true, timeout=300)
|
||||
```
|
||||
|
||||
### Post the review
|
||||
|
||||
```
|
||||
terminal(command="gh pr comment 42 --body '<review text>'", workdir="/path/to/repo")
|
||||
```
|
||||
|
||||
## Parallel Issue Fixing with Worktrees
|
||||
|
||||
```
|
||||
# Create worktrees
|
||||
terminal(command="git worktree add -b fix/issue-78 /tmp/issue-78 main", workdir="~/project")
|
||||
terminal(command="git worktree add -b fix/issue-99 /tmp/issue-99 main", workdir="~/project")
|
||||
|
||||
# Launch Grok headless in each (background)
|
||||
terminal(command="grok --no-auto-update --always-approve -p 'Fix issue #78: <description>. Commit when done.'", workdir="/tmp/issue-78", background=true, notify_on_complete=true)
|
||||
terminal(command="grok --no-auto-update --always-approve -p 'Fix issue #99: <description>. Commit when done.'", workdir="/tmp/issue-99", background=true, notify_on_complete=true)
|
||||
|
||||
# Monitor
|
||||
process(action="list")
|
||||
|
||||
# After completion: push and open PRs
|
||||
terminal(command="cd /tmp/issue-78 && git push -u origin fix/issue-78")
|
||||
terminal(command="gh pr create --repo user/repo --head fix/issue-78 --title 'fix: ...' --body '...'")
|
||||
|
||||
# Cleanup
|
||||
terminal(command="git worktree remove /tmp/issue-78", workdir="~/project")
|
||||
```
|
||||
|
||||
## Useful Subcommands & TUI Commands
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `grok` | Start the interactive TUI |
|
||||
| `grok -p "query"` | Headless one-shot |
|
||||
| `grok login` / `grok logout` | Sign in / out (SuperGrok / X Premium+ OAuth) |
|
||||
| `grok inspect` | Show what Grok discovered in cwd: config sources, instructions, skills, plugins, hooks, MCP servers |
|
||||
| `grok agent stdio` | Run as an ACP agent over JSON-RPC (for IDE/tool integration) |
|
||||
| `grok update` | Update the CLI (needs the `x.ai` host; skip in automation) |
|
||||
|
||||
TUI slash commands (interactive only): `/model <name>`, `/always-approve`,
|
||||
`/plan`, `/context`, `/compact`, `/resume`, `/sessions`, `/fork`, `/usage`,
|
||||
`/quit`. `Shift+Tab` cycles session modes (including Plan mode, which blocks
|
||||
write tools except the session plan file).
|
||||
|
||||
## Config (`~/.grok/config.toml`)
|
||||
|
||||
```toml
|
||||
[cli]
|
||||
auto_update = false # skip background update checks persistently
|
||||
|
||||
[ui]
|
||||
permission_mode = "ask" # or "always-approve" to skip tool prompts by default
|
||||
|
||||
[models]
|
||||
default = "grok-build-0.1"
|
||||
```
|
||||
|
||||
Put global preferences in `~/.grok/config.toml` (not project-scoped
|
||||
`.grok/config.toml`). `permission_mode` supersedes the legacy `approval_mode` /
|
||||
`yolo = true` keys.
|
||||
|
||||
## Pitfalls & Gotchas
|
||||
|
||||
1. **Auth is subscription-gated.** `grok login` requires a SuperGrok or X
|
||||
Premium+ subscription. If login fails or there's no `~/.grok/auth.json`,
|
||||
confirm the subscription is active before falling back to `XAI_API_KEY`.
|
||||
2. **Don't conflate Hermes' xAI auth with the `grok` CLI's auth.** Hermes'
|
||||
`x_search` runs on its own xAI OAuth; the standalone `grok` CLI has a
|
||||
separate token in `~/.grok/auth.json`. A working `x_search` does NOT mean
|
||||
`grok` is logged in.
|
||||
3. **Always pass `--no-auto-update` in automation** — otherwise Grok phones home
|
||||
for update checks (and `x.ai`/`storage.googleapis.com` may be unreachable).
|
||||
4. **Prefer npm install over the curl installer** — `npm install -g
|
||||
@xai-official/grok` avoids the Cloudflare-walled `x.ai` host.
|
||||
5. **`--always-approve` is the autonomous-build switch.** Without it, headless
|
||||
runs may stall waiting on tool-approval prompts. Omit it deliberately for
|
||||
read-only review/audit work so Grok can't mutate files.
|
||||
6. **Headless `-p` skips TUI dialogs**; the TUI needs `pty=true` (+ tmux for
|
||||
monitoring), just like Claude Code.
|
||||
7. **Use `--no-alt-screen`** if you run the TUI inline and the fullscreen
|
||||
alt-screen takeover garbles captured output.
|
||||
8. **No git repo needed**, but for PR/commit workflows you still want one — use
|
||||
`mktemp -d && git init` for scratch commit tasks.
|
||||
9. **Clean up tmux sessions** with `tmux kill-session -t <name>` when done.
|
||||
|
||||
## Rules for Hermes Agents
|
||||
|
||||
1. **Prefer headless `-p`** for single tasks — cleanest integration, structured
|
||||
output via `--output-format json`.
|
||||
2. **Always set `workdir`** (or `--cwd`) so Grok targets the right project.
|
||||
3. **Pass `--no-auto-update`** in every automated invocation.
|
||||
4. **Use `--always-approve` only when Grok should write autonomously**; omit it
|
||||
for read-only reviews and audits.
|
||||
5. **Background long tasks** with `background=true, notify_on_complete=true` and
|
||||
monitor via the `process` tool.
|
||||
6. **Use tmux for multi-turn interactive work** and monitor with
|
||||
`tmux capture-pane -t <session> -p -S -50`.
|
||||
7. **Verify auth before relying on it** — check `~/.grok/auth.json` or run a
|
||||
cheap `grok -p "Say ok."` smoke test; don't assume Hermes' xAI auth carries
|
||||
over.
|
||||
8. **Report results to the user** — summarize what Grok changed and what's left.
|
||||
@@ -0,0 +1,431 @@
|
||||
---
|
||||
name: honcho
|
||||
description: Configure and use Honcho memory with Hermes -- cross-session user modeling, multi-profile peer isolation, observation config, dialectic reasoning, session summaries, and context budget enforcement. Use when setting up Honcho, troubleshooting memory, managing profiles with Honcho peers, or tuning observation, recall, and dialectic settings.
|
||||
version: 2.0.0
|
||||
author: Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Honcho, Memory, Profiles, Observation, Dialectic, User-Modeling, Session-Summary]
|
||||
homepage: https://docs.honcho.dev
|
||||
related_skills: [hermes-agent]
|
||||
prerequisites:
|
||||
pip: [honcho-ai]
|
||||
---
|
||||
|
||||
# Honcho Memory for Hermes
|
||||
|
||||
Honcho provides AI-native cross-session user modeling. It learns who the user is across conversations and gives every Hermes profile its own peer identity while sharing a unified view of the user.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Setting up Honcho (cloud or self-hosted)
|
||||
- Troubleshooting memory not working / peers not syncing
|
||||
- Creating multi-profile setups where each agent has its own Honcho peer
|
||||
- Tuning observation, recall, dialectic depth, or write frequency settings
|
||||
- Understanding what the 5 Honcho tools do and when to use them
|
||||
- Configuring context budgets and session summary injection
|
||||
|
||||
## Setup
|
||||
|
||||
### Cloud (app.honcho.dev)
|
||||
|
||||
```bash
|
||||
hermes memory setup honcho
|
||||
# select "cloud", paste API key from https://app.honcho.dev
|
||||
```
|
||||
|
||||
### Self-hosted
|
||||
|
||||
```bash
|
||||
hermes memory setup honcho
|
||||
# select "local", enter base URL (e.g. http://localhost:8000)
|
||||
```
|
||||
|
||||
See: https://docs.honcho.dev/v3/guides/integrations/hermes#running-honcho-locally-with-hermes
|
||||
|
||||
### Verify
|
||||
|
||||
```bash
|
||||
hermes honcho status # shows resolved config, connection test, peer info
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Base Context Injection
|
||||
|
||||
When Honcho injects context into the system prompt (in `hybrid` or `context` recall modes), it assembles the base context block in this order:
|
||||
|
||||
1. **Session summary** -- a short digest of the current session so far (placed first so the model has immediate conversational continuity)
|
||||
2. **User representation** -- Honcho's accumulated model of the user (preferences, facts, patterns)
|
||||
3. **AI peer card** -- the identity card for this Hermes profile's AI peer
|
||||
|
||||
The session summary is generated automatically by Honcho at the start of each turn (when a prior session exists). It gives the model a warm start without replaying full history.
|
||||
|
||||
### Cold / Warm Prompt Selection
|
||||
|
||||
Honcho automatically selects between two prompt strategies:
|
||||
|
||||
| Condition | Strategy | What happens |
|
||||
|-----------|----------|--------------|
|
||||
| No prior session or empty representation | **Cold start** | Lightweight intro prompt; skips summary injection; encourages the model to learn about the user |
|
||||
| Existing representation and/or session history | **Warm start** | Full base context injection (summary → representation → card); richer system prompt |
|
||||
|
||||
You do not need to configure this -- it is automatic based on session state.
|
||||
|
||||
### Peers
|
||||
|
||||
Honcho models conversations as interactions between **peers**. Hermes creates two peers per session:
|
||||
|
||||
- **User peer** (`peerName`): represents the human. Honcho builds a user representation from observed messages.
|
||||
- **AI peer** (`aiPeer`): represents this Hermes instance. Each profile gets its own AI peer so agents develop independent views.
|
||||
|
||||
### Observation
|
||||
|
||||
Each peer has two observation toggles that control what Honcho learns from:
|
||||
|
||||
| Toggle | What it does |
|
||||
|--------|-------------|
|
||||
| `observeMe` | Peer's own messages are observed (builds self-representation) |
|
||||
| `observeOthers` | Other peers' messages are observed (builds cross-peer understanding) |
|
||||
|
||||
Default: all four toggles **on** (full bidirectional observation).
|
||||
|
||||
Configure per-peer in `honcho.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"observation": {
|
||||
"user": { "observeMe": true, "observeOthers": true },
|
||||
"ai": { "observeMe": true, "observeOthers": true }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Or use the shorthand presets:
|
||||
|
||||
| Preset | User | AI | Use case |
|
||||
|--------|------|----|----------|
|
||||
| `"directional"` (default) | me:on, others:on | me:on, others:on | Multi-agent, full memory |
|
||||
| `"unified"` | me:on, others:off | me:off, others:on | Single agent, user-only modeling |
|
||||
|
||||
Settings changed in the [Honcho dashboard](https://app.honcho.dev) are synced back on session init -- server-side config wins over local defaults.
|
||||
|
||||
### Sessions
|
||||
|
||||
Honcho sessions scope where messages and observations land. Strategy options:
|
||||
|
||||
| Strategy | Behavior |
|
||||
|----------|----------|
|
||||
| `per-directory` (default) | One session per working directory |
|
||||
| `per-repo` | One session per git repository root |
|
||||
| `per-session` | New Honcho session each Hermes run |
|
||||
| `global` | Single session across all directories |
|
||||
|
||||
Manual override: `hermes honcho map my-project-name`
|
||||
|
||||
### Recall Modes
|
||||
|
||||
How the agent accesses Honcho memory:
|
||||
|
||||
| Mode | Auto-inject context? | Tools available? | Use case |
|
||||
|------|---------------------|-----------------|----------|
|
||||
| `hybrid` (default) | Yes | Yes | Agent decides when to use tools vs auto context |
|
||||
| `context` | Yes | No (hidden) | Minimal token cost, no tool calls |
|
||||
| `tools` | No | Yes | Agent controls all memory access explicitly |
|
||||
|
||||
## Three Orthogonal Knobs
|
||||
|
||||
Honcho's dialectic behavior is controlled by three independent dimensions. Each can be tuned without affecting the others:
|
||||
|
||||
### Cadence (when)
|
||||
|
||||
Controls **how often** dialectic and context calls happen.
|
||||
|
||||
| Key | Default | Description |
|
||||
|-----|---------|-------------|
|
||||
| `contextCadence` | `1` | Min turns between context API calls |
|
||||
| `dialecticCadence` | `2` | Min turns between dialectic API calls. Recommended 1–5 |
|
||||
| `injectionFrequency` | `every-turn` | `every-turn` or `first-turn` for base context injection |
|
||||
|
||||
Higher cadence values fire the dialectic LLM less often. `dialecticCadence: 2` means the engine fires every other turn. Setting it to `1` fires every turn.
|
||||
|
||||
### Depth (how many)
|
||||
|
||||
Controls **how many rounds** of dialectic reasoning Honcho performs per query.
|
||||
|
||||
| Key | Default | Range | Description |
|
||||
|-----|---------|-------|-------------|
|
||||
| `dialecticDepth` | `1` | 1-3 | Number of dialectic reasoning rounds per query |
|
||||
| `dialecticDepthLevels` | -- | array | Optional per-depth-round level overrides (see below) |
|
||||
|
||||
`dialecticDepth: 2` means Honcho runs two rounds of dialectic synthesis. The first round produces an initial answer; the second refines it.
|
||||
|
||||
`dialecticDepthLevels` lets you set the reasoning level for each round independently:
|
||||
|
||||
```json
|
||||
{
|
||||
"dialecticDepth": 3,
|
||||
"dialecticDepthLevels": ["low", "medium", "high"]
|
||||
}
|
||||
```
|
||||
|
||||
If `dialecticDepthLevels` is omitted, rounds use **proportional levels** derived from `dialecticReasoningLevel` (the base):
|
||||
|
||||
| Depth | Pass levels |
|
||||
|-------|-------------|
|
||||
| 1 | [base] |
|
||||
| 2 | [minimal, base] |
|
||||
| 3 | [minimal, base, low] |
|
||||
|
||||
This keeps earlier passes cheap while using full depth on the final synthesis.
|
||||
|
||||
**Depth at session start.** The session-start prewarm runs the full configured `dialecticDepth` in the background before turn 1. A single-pass prewarm on a cold peer often returns thin output — multi-pass depth runs the audit/reconcile cycle before the user ever speaks. Turn 1 consumes the prewarm result directly; if prewarm hasn't landed in time, turn 1 falls back to a synchronous call with a bounded timeout.
|
||||
|
||||
### Level (how hard)
|
||||
|
||||
Controls the **intensity** of each dialectic reasoning round.
|
||||
|
||||
| Key | Default | Description |
|
||||
|-----|---------|-------------|
|
||||
| `dialecticReasoningLevel` | `low` | `minimal`, `low`, `medium`, `high`, `max` |
|
||||
| `dialecticDynamic` | `true` | When `true`, the model can pass `reasoning_level` to `honcho_reasoning` to override the default per-call. `false` = always use `dialecticReasoningLevel`, model overrides ignored |
|
||||
|
||||
Higher levels produce richer synthesis but cost more tokens on Honcho's backend.
|
||||
|
||||
## Multi-Profile Setup
|
||||
|
||||
Each Hermes profile gets its own Honcho AI peer while sharing the same workspace (user context). This means:
|
||||
|
||||
- All profiles see the same user representation
|
||||
- Each profile builds its own AI identity and observations
|
||||
- Conclusions written by one profile are visible to others via the shared workspace
|
||||
|
||||
### Create a profile with Honcho peer
|
||||
|
||||
```bash
|
||||
hermes profile create coder --clone
|
||||
# creates host block hermes.coder, AI peer "coder", inherits config from default
|
||||
```
|
||||
|
||||
What `--clone` does for Honcho:
|
||||
1. Creates a `hermes.coder` host block in `honcho.json`
|
||||
2. Sets `aiPeer: "coder"` (the profile name)
|
||||
3. Inherits `workspace`, `peerName`, `writeFrequency`, `recallMode`, etc. from default
|
||||
4. Eagerly creates the peer in Honcho so it exists before first message
|
||||
|
||||
### Backfill existing profiles
|
||||
|
||||
```bash
|
||||
hermes honcho sync # creates host blocks for all profiles that don't have one yet
|
||||
```
|
||||
|
||||
### Per-profile config
|
||||
|
||||
Override any setting in the host block:
|
||||
|
||||
```json
|
||||
{
|
||||
"hosts": {
|
||||
"hermes.coder": {
|
||||
"aiPeer": "coder",
|
||||
"recallMode": "tools",
|
||||
"dialecticDepth": 2,
|
||||
"observation": {
|
||||
"user": { "observeMe": true, "observeOthers": false },
|
||||
"ai": { "observeMe": true, "observeOthers": true }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Tools
|
||||
|
||||
The agent has 5 bidirectional Honcho tools (hidden in `context` recall mode):
|
||||
|
||||
| Tool | LLM call? | Cost | Use when |
|
||||
|------|-----------|------|----------|
|
||||
| `honcho_profile` | No | minimal | Quick factual snapshot at conversation start or for fast name/role/pref lookups |
|
||||
| `honcho_search` | No | low | Fetch specific past facts to reason over yourself — raw excerpts, no synthesis |
|
||||
| `honcho_context` | No | low | Full session context snapshot: summary, representation, card, recent messages |
|
||||
| `honcho_reasoning` | Yes | medium–high | Natural language question synthesized by Honcho's dialectic engine |
|
||||
| `honcho_conclude` | No | minimal | Write or delete a persistent fact; pass `peer: "ai"` for AI self-knowledge |
|
||||
|
||||
### `honcho_profile`
|
||||
Read or update a peer card — curated key facts (name, role, preferences, communication style). Pass `card: [...]` to update; omit to read. No LLM call.
|
||||
|
||||
### `honcho_search`
|
||||
Semantic search over stored context for a specific peer. Returns raw excerpts ranked by relevance, no synthesis. Default 800 tokens, max 2000. Good when you need specific past facts to reason over yourself rather than a synthesized answer.
|
||||
|
||||
### `honcho_context`
|
||||
Full session context snapshot from Honcho — session summary, peer representation, peer card, and recent messages. No LLM call. Use when you want to see everything Honcho knows about the current session and peer in one shot.
|
||||
|
||||
### `honcho_reasoning`
|
||||
Natural language question answered by Honcho's dialectic reasoning engine (LLM call on Honcho's backend). Higher cost, higher quality. Pass `reasoning_level` to control depth: `minimal` (fast/cheap) → `low` → `medium` → `high` → `max` (thorough). Omit to use the configured default (`low`). Use for synthesized understanding of the user's patterns, goals, or current state.
|
||||
|
||||
### `honcho_conclude`
|
||||
Write or delete a persistent conclusion about a peer. Pass `conclusion: "..."` to create. Pass `delete_id: "..."` to remove a conclusion (for PII removal — Honcho self-heals incorrect conclusions over time, so deletion is only needed for PII). You MUST pass exactly one of the two.
|
||||
|
||||
### Bidirectional peer targeting
|
||||
|
||||
All 5 tools accept an optional `peer` parameter:
|
||||
- `peer: "user"` (default) — operates on the user peer
|
||||
- `peer: "ai"` — operates on this profile's AI peer
|
||||
- `peer: "<explicit-id>"` — any peer ID in the workspace
|
||||
|
||||
Examples:
|
||||
```
|
||||
honcho_profile # read user's card
|
||||
honcho_profile peer="ai" # read AI peer's card
|
||||
honcho_reasoning query="What does this user care about most?"
|
||||
honcho_reasoning query="What are my interaction patterns?" peer="ai" reasoning_level="medium"
|
||||
honcho_conclude conclusion="Prefers terse answers"
|
||||
honcho_conclude conclusion="I tend to over-explain code" peer="ai"
|
||||
honcho_conclude delete_id="abc123" # PII removal
|
||||
```
|
||||
|
||||
## Agent Usage Patterns
|
||||
|
||||
Guidelines for Hermes when Honcho memory is active.
|
||||
|
||||
### On conversation start
|
||||
|
||||
```
|
||||
1. honcho_profile → fast warmup, no LLM cost
|
||||
2. If context looks thin → honcho_context (full snapshot, still no LLM)
|
||||
3. If deep synthesis needed → honcho_reasoning (LLM call, use sparingly)
|
||||
```
|
||||
|
||||
Do NOT call `honcho_reasoning` on every turn. Auto-injection already handles ongoing context refresh. Use the reasoning tool only when you genuinely need synthesized insight the base context doesn't provide.
|
||||
|
||||
### When the user shares something to remember
|
||||
|
||||
```
|
||||
honcho_conclude conclusion="<specific, actionable fact>"
|
||||
```
|
||||
|
||||
Good conclusions: "Prefers code examples over prose explanations", "Working on a Rust async project through April 2026"
|
||||
Bad conclusions: "User said something about Rust" (too vague), "User seems technical" (already in representation)
|
||||
|
||||
### When the user asks about past context / you need to recall specifics
|
||||
|
||||
```
|
||||
honcho_search query="<topic>" → fast, no LLM, good for specific facts
|
||||
honcho_context → full snapshot with summary + messages
|
||||
honcho_reasoning query="<question>" → synthesized answer, use when search isn't enough
|
||||
```
|
||||
|
||||
### When to use `peer: "ai"`
|
||||
|
||||
Use AI peer targeting to build and query the agent's own self-knowledge:
|
||||
- `honcho_conclude conclusion="I tend to be verbose when explaining architecture" peer="ai"` — self-correction
|
||||
- `honcho_reasoning query="How do I typically handle ambiguous requests?" peer="ai"` — self-audit
|
||||
- `honcho_profile peer="ai"` — review own identity card
|
||||
|
||||
### When NOT to call tools
|
||||
|
||||
In `hybrid` and `context` modes, base context (user representation + card + session summary) is auto-injected before every turn. Do not re-fetch what was already injected. Call tools only when:
|
||||
- You need something the injected context doesn't have
|
||||
- The user explicitly asks you to recall or check memory
|
||||
- You're writing a conclusion about something new
|
||||
|
||||
### Cadence awareness
|
||||
|
||||
`honcho_reasoning` on the tool side shares the same cost as auto-injection dialectic. After an explicit tool call, the auto-injection cadence resets — avoiding double-charging the same turn.
|
||||
|
||||
## Config Reference
|
||||
|
||||
Config file: `$HERMES_HOME/honcho.json` (profile-local) or `~/.honcho/config.json` (global).
|
||||
|
||||
### Key settings
|
||||
|
||||
| Key | Default | Description |
|
||||
|-----|---------|-------------|
|
||||
| `apiKey` | -- | API key ([get one](https://app.honcho.dev)) |
|
||||
| `baseUrl` | -- | Base URL for self-hosted Honcho |
|
||||
| `peerName` | -- | User peer identity |
|
||||
| `aiPeer` | host key | AI peer identity |
|
||||
| `workspace` | host key | Shared workspace ID |
|
||||
| `recallMode` | `hybrid` | `hybrid`, `context`, or `tools` |
|
||||
| `observation` | all on | Per-peer `observeMe`/`observeOthers` booleans |
|
||||
| `writeFrequency` | `async` | `async`, `turn`, `session`, or integer N |
|
||||
| `sessionStrategy` | `per-directory` | `per-directory`, `per-repo`, `per-session`, `global` |
|
||||
| `messageMaxChars` | `25000` | Max chars per message (chunked if exceeded) |
|
||||
|
||||
### Dialectic settings
|
||||
|
||||
| Key | Default | Description |
|
||||
|-----|---------|-------------|
|
||||
| `dialecticReasoningLevel` | `low` | `minimal`, `low`, `medium`, `high`, `max` |
|
||||
| `dialecticDynamic` | `true` | Auto-bump reasoning by query complexity. `false` = fixed level |
|
||||
| `dialecticDepth` | `1` | Number of dialectic rounds per query (1-3) |
|
||||
| `dialecticDepthLevels` | -- | Optional array of per-round levels, e.g. `["low", "high"]` |
|
||||
| `dialecticMaxInputChars` | `10000` | Max chars for dialectic query input |
|
||||
|
||||
### Context budget and injection
|
||||
|
||||
| Key | Default | Description |
|
||||
|-----|---------|-------------|
|
||||
| `contextTokens` | uncapped | Max tokens for the combined base context injection (summary + representation + card). Opt-in cap — omit to leave uncapped, set to an integer to bound injection size. |
|
||||
| `injectionFrequency` | `every-turn` | `every-turn` or `first-turn` |
|
||||
| `contextCadence` | `1` | Min turns between context API calls |
|
||||
| `dialecticCadence` | `2` | Min turns between dialectic LLM calls (recommended 1–5) |
|
||||
|
||||
The `contextTokens` budget is enforced at injection time. If the session summary + representation + card exceed the budget, Honcho trims the summary first, then the representation, preserving the card. This prevents context blowup in long sessions.
|
||||
|
||||
### Memory-context sanitization
|
||||
|
||||
Honcho sanitizes the `memory-context` block before injection to prevent prompt injection and malformed content:
|
||||
|
||||
- Strips XML/HTML tags from user-authored conclusions
|
||||
- Normalizes whitespace and control characters
|
||||
- Truncates individual conclusions that exceed `messageMaxChars`
|
||||
- Escapes delimiter sequences that could break the system prompt structure
|
||||
|
||||
This fix addresses edge cases where raw user conclusions containing markup or special characters could corrupt the injected context block.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Honcho not configured"
|
||||
Run `hermes honcho setup`. Ensure `memory.provider: honcho` is in `~/.hermes/config.yaml`.
|
||||
|
||||
### Memory not persisting across sessions
|
||||
Check `hermes honcho status` -- verify `saveMessages: true` and `writeFrequency` isn't `session` (which only writes on exit).
|
||||
|
||||
### Profile not getting its own peer
|
||||
Use `--clone` when creating: `hermes profile create <name> --clone`. For existing profiles: `hermes honcho sync`.
|
||||
|
||||
### Observation changes in dashboard not reflected
|
||||
Observation config is synced from the server on each session init. Start a new session after changing settings in the Honcho UI.
|
||||
|
||||
### Messages truncated
|
||||
Messages over `messageMaxChars` (default 25k) are automatically chunked with `[continued]` markers. If you're hitting this often, check if tool results or skill content is inflating message size.
|
||||
|
||||
### Context injection too large
|
||||
If you see warnings about context budget exceeded, lower `contextTokens` or reduce `dialecticDepth`. The session summary is trimmed first when the budget is tight.
|
||||
|
||||
### Session summary missing
|
||||
Session summary requires at least one prior turn in the current Honcho session. On cold start (new session, no history), the summary is omitted and Honcho uses the cold-start prompt strategy instead.
|
||||
|
||||
## CLI Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `hermes honcho setup` | Interactive setup wizard (cloud/local, identity, observation, recall, sessions) |
|
||||
| `hermes honcho status` | Show resolved config, connection test, peer info for active profile |
|
||||
| `hermes honcho enable` | Enable Honcho for the active profile (creates host block if needed) |
|
||||
| `hermes honcho disable` | Disable Honcho for the active profile |
|
||||
| `hermes honcho peer` | Show or update peer names (`--user <name>`, `--ai <name>`, `--reasoning <level>`) |
|
||||
| `hermes honcho peers` | Show peer identities across all profiles |
|
||||
| `hermes honcho mode` | Show or set recall mode (`hybrid`, `context`, `tools`) |
|
||||
| `hermes honcho tokens` | Show or set token budgets (`--context <N>`, `--dialectic <N>`) |
|
||||
| `hermes honcho sessions` | List known directory-to-session-name mappings |
|
||||
| `hermes honcho map <name>` | Map current working directory to a Honcho session name |
|
||||
| `hermes honcho identity` | Seed AI peer identity or show both peer representations |
|
||||
| `hermes honcho sync` | Create host blocks for all Hermes profiles that don't have one yet |
|
||||
| `hermes honcho migrate` | Step-by-step migration guide from OpenClaw native memory to Hermes + Honcho |
|
||||
| `hermes memory setup` | Generic memory provider picker (selecting "honcho" runs the same wizard) |
|
||||
| `hermes memory status` | Show active memory provider and config |
|
||||
| `hermes memory off` | Disable external memory provider |
|
||||
@@ -0,0 +1,149 @@
|
||||
---
|
||||
name: openhands
|
||||
description: Delegate coding to OpenHands CLI (model-agnostic, LiteLLM).
|
||||
version: 0.1.0
|
||||
author: Tim Koepsel (xzessmedia), Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Coding-Agent, OpenHands, Model-Agnostic, LiteLLM]
|
||||
related_skills: [claude-code, codex, opencode, hermes-agent]
|
||||
---
|
||||
|
||||
# OpenHands CLI
|
||||
|
||||
Delegate coding tasks to the [OpenHands CLI](https://github.com/All-Hands-AI/OpenHands) via the `terminal` tool. OpenHands is model-agnostic: any LiteLLM-supported provider (OpenAI, Anthropic, OpenRouter, DeepSeek, Ollama, vLLM, etc.).
|
||||
|
||||
This skill is the headless-mode wrapper for batch / one-shot delegation. The interactive textual UI is not used from Hermes.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User wants a coding task delegated to OpenHands specifically.
|
||||
- User wants a coding agent that can run on a non-Anthropic / non-OpenAI provider (DeepSeek, Qwen, Ollama, vLLM, Nous, etc.) — sibling skills `claude-code` and `codex` are tied to one vendor.
|
||||
- Multi-step file edits + shell commands inside a workspace.
|
||||
|
||||
For Claude-native, prefer `claude-code`. For OpenAI-native, prefer `codex`. For Hermes-native subagents, use `delegate_task`.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Install upstream (requires Python 3.12+ and `uv`):
|
||||
|
||||
```
|
||||
terminal(command="uv tool install openhands --python 3.12")
|
||||
```
|
||||
|
||||
Verify: `openhands --version` (currently `OpenHands CLI 1.16.0` / `SDK v1.21.0` at time of writing).
|
||||
|
||||
2. Pick a model and set env vars for `--override-with-envs`:
|
||||
|
||||
```
|
||||
export LLM_MODEL=openrouter/openai/gpt-4o-mini # or any LiteLLM slug
|
||||
export LLM_API_KEY=$OPENROUTER_API_KEY
|
||||
export LLM_BASE_URL=https://openrouter.ai/api/v1 # omit for native OpenAI
|
||||
```
|
||||
|
||||
`LLM_MODEL` uses LiteLLM's full slug. When the provider is OpenRouter the slug is doubly-prefixed: `openrouter/<vendor>/<model>` (e.g. `openrouter/anthropic/claude-sonnet-4.5`). For native Anthropic: `anthropic/claude-sonnet-4-5`. For native OpenAI: `openai/gpt-4o-mini`.
|
||||
|
||||
3. Suppress the startup banner so JSON output isn't preceded by ASCII art:
|
||||
|
||||
```
|
||||
export OPENHANDS_SUPPRESS_BANNER=1
|
||||
```
|
||||
|
||||
## How to Run
|
||||
|
||||
Always invoke through the `terminal` tool. Always pass `--headless --json --override-with-envs --exit-without-confirmation` for automation.
|
||||
|
||||
### One-shot task
|
||||
|
||||
```
|
||||
terminal(
|
||||
command="OPENHANDS_SUPPRESS_BANNER=1 LLM_MODEL=openrouter/openai/gpt-4o-mini LLM_API_KEY=$OPENROUTER_API_KEY LLM_BASE_URL=https://openrouter.ai/api/v1 openhands --headless --json --override-with-envs --exit-without-confirmation -t 'Add error handling to all API calls in src/'",
|
||||
workdir="/path/to/project",
|
||||
timeout=600
|
||||
)
|
||||
```
|
||||
|
||||
### Background for long tasks
|
||||
|
||||
```
|
||||
terminal(command="<same as above>", workdir="/path/to/project", background=true, notify_on_complete=true)
|
||||
process(action="poll", session_id="<id>")
|
||||
process(action="log", session_id="<id>")
|
||||
```
|
||||
|
||||
### Resume a previous conversation
|
||||
|
||||
OpenHands prints `Conversation ID: <32-hex>` and a `Hint: openhands --resume <dashed-uuid>` line at the end of each run. Use the dashed form to resume:
|
||||
|
||||
```
|
||||
terminal(
|
||||
command="OPENHANDS_SUPPRESS_BANNER=1 LLM_MODEL=... openhands --headless --json --override-with-envs --exit-without-confirmation --resume <dashed-uuid> -t 'Now fix the bug you found'",
|
||||
workdir="/path/to/project"
|
||||
)
|
||||
```
|
||||
|
||||
## Real Flag List
|
||||
|
||||
Verified against `openhands --help` (CLI 1.16.0). Anything not in this table is not a flag — pass it via env var or settings file.
|
||||
|
||||
| Flag | Effect |
|
||||
|------|--------|
|
||||
| `--headless` | No UI, requires `-t` or `-f`. Auto-approves all actions (no `--llm-approve` in this mode). |
|
||||
| `--json` | JSONL event stream (requires `--headless`). |
|
||||
| `-t TEXT` | Task prompt. |
|
||||
| `-f PATH` | Read task from file. |
|
||||
| `--resume [ID]` | Resume conversation. No ID → list recent. |
|
||||
| `--last` | Resume most recent (with `--resume`). |
|
||||
| `--override-with-envs` | Apply `LLM_API_KEY` / `LLM_BASE_URL` / `LLM_MODEL` env vars. Without this, OpenHands uses `~/.openhands/settings.json` and ignores the env. |
|
||||
| `--exit-without-confirmation` | Don't show the "are you sure" exit dialog. |
|
||||
| `--always-approve` / `--yolo` | Auto-approve every action (default in `--headless`). |
|
||||
| `--llm-approve` | LLM-based security gate (interactive only — does NOT work in headless). |
|
||||
| `--version` / `-v` | Print version and exit. |
|
||||
|
||||
**There is no `--model`, `--max-iterations`, `--workspace`, `--sandbox`, `--sandbox-type` flag.** Model is `LLM_MODEL`. Workspace is the `workdir` you pass to the `terminal` tool. Sandbox / runtime is the `RUNTIME` and `SANDBOX_VOLUMES` env vars.
|
||||
|
||||
## JSON Event Schema
|
||||
|
||||
With `--json --headless`, OpenHands emits JSONL — one JSON object per line, plus a handful of non-JSON status lines (`Initializing agent...`, `Agent is working`, `Agent finished`, the final summary box, `Goodbye!`, `Conversation ID:`, `Hint:`). Filter for lines starting with `{`.
|
||||
|
||||
Top-level `kind` field discriminates events:
|
||||
|
||||
- `MessageEvent` — user / agent text turn. `source` is `user` or `agent`.
|
||||
- `ActionEvent` — agent picked a tool. Read `tool_name` (`file_editor`, `terminal`, `finish`) and `action.kind` (`FileEditorAction`, `TerminalAction`, `FinishAction`).
|
||||
- `ObservationEvent` — tool result. `observation.is_error` is the success flag. `source` is `environment`.
|
||||
- `FinishAction` inside an `ActionEvent` carries the agent's final message in `action.message`.
|
||||
|
||||
The cli prints all stderr from LiteLLM/Authlib first — see Pitfalls. Parse only stdout, line by line, ignoring lines that don't start with `{`.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **LiteLLM warnings on every invocation.** The CLI prints `bedrock-runtime` and `sagemaker-runtime` warnings to stderr because `botocore` isn't installed. Plus an Authlib deprecation. These are noise, not failures. Pipe stderr to `/dev/null` or filter it out before showing the user.
|
||||
- **Banner spam.** Without `OPENHANDS_SUPPRESS_BANNER=1`, every run starts with a multi-line `+--+` ASCII box advertising the SDK. Always export it.
|
||||
- **`--override-with-envs` is mandatory for automation.** Without it, OpenHands ignores `LLM_API_KEY` / `LLM_BASE_URL` / `LLM_MODEL` and falls back to `~/.openhands/settings.json`. On a fresh install this file doesn't exist and the CLI hangs waiting for first-run setup.
|
||||
- **Model slug is LiteLLM's, not the provider's.** `openrouter/openai/gpt-4o-mini` works; `openai/gpt-4o-mini` while pointed at OpenRouter does not. `anthropic/claude-sonnet-4-5` (hyphen) is native Anthropic; `openrouter/anthropic/claude-sonnet-4.5` (dot) is via OpenRouter. Get it wrong → cryptic LiteLLM 400.
|
||||
- **`pip install openhands-ai` is the wrong package.** That's the legacy V0 SDK. The new CLI is `uv tool install openhands --python 3.12`. There is no maintained conda package.
|
||||
- **Resume ID format is fiddly.** The CLI ends with `Conversation ID: f46573d9cfdb45e492ca189bde40019b` (no dashes) and then a `Hint: openhands --resume f46573d9-cfdb-45e4-92ca-189bde40019b` (with dashes). Use the dashed form.
|
||||
- **Headless ignores `--llm-approve`.** If you pass it, you get an argparse error. Headless mode hardcodes always-approve.
|
||||
- **No Windows support upstream.** The OpenHands docs require WSL on Windows. This skill is gated `[linux, macos]` accordingly.
|
||||
- **`~/.openhands/conversations/<id>/` accumulates.** Each run persists a trajectory. Clean it up if running batches.
|
||||
- **Heavy install (~200 packages).** Use `uv tool install` (isolated venv) to avoid dependency conflicts with the active project.
|
||||
|
||||
## Verification
|
||||
|
||||
```
|
||||
terminal(
|
||||
command="OPENHANDS_SUPPRESS_BANNER=1 LLM_MODEL=openrouter/openai/gpt-4o-mini LLM_API_KEY=$OPENROUTER_API_KEY LLM_BASE_URL=https://openrouter.ai/api/v1 openhands --headless --json --override-with-envs --exit-without-confirmation -t 'Print the string OPENHANDS_OK to stdout via the terminal tool.'",
|
||||
workdir="/tmp",
|
||||
timeout=120
|
||||
)
|
||||
```
|
||||
|
||||
If the JSONL stream ends with a `FinishAction` whose `action.message` mentions `OPENHANDS_OK`, the install is working.
|
||||
|
||||
## Related
|
||||
|
||||
- [OpenHands GitHub](https://github.com/All-Hands-AI/OpenHands)
|
||||
- [OpenHands CLI command reference](https://docs.openhands.dev/openhands/usage/cli/command-reference)
|
||||
- Sibling skills: `claude-code` (Anthropic-only), `codex` (OpenAI-only), `opencode` (multi-provider via OpenCode), `hermes-agent` (Hermes subagents via `delegate_task`).
|
||||
@@ -0,0 +1,211 @@
|
||||
---
|
||||
name: evm
|
||||
description: "Read-only EVM client: wallets, tokens, gas across 8 chains."
|
||||
version: 1.0.0
|
||||
author: Mibayy (@Mibayy), youssefea (@youssefea), ethernet8023 (@ethernet8023), Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [EVM, Ethereum, BNB, BSC, Base, Arbitrum, Polygon, Optimism, Avalanche, zkSync, Blockchain, Crypto, Web3, DeFi, NFT, ENS, Whale, Security]
|
||||
category: blockchain
|
||||
related_skills: [solana]
|
||||
requires_toolsets: [terminal]
|
||||
---
|
||||
|
||||
# EVM Blockchain Skill
|
||||
|
||||
Query EVM-compatible blockchain data across 8 chains with USD pricing.
|
||||
14 commands: wallet portfolio, token info, transactions, activity, gas tracker,
|
||||
network stats, price lookup, multi-chain scan, whale detection, ENS resolution,
|
||||
allowance checker, contract inspector, and transaction decoder.
|
||||
|
||||
Supports 8 chains: Ethereum, BNB Chain (BSC), Base, Arbitrum One, Polygon,
|
||||
Optimism, Avalanche (C-Chain), zkSync Era.
|
||||
|
||||
No API key needed. Zero external dependencies — Python standard library only
|
||||
(urllib, json, argparse, threading).
|
||||
|
||||
> **Supersedes the standalone `base` skill.** Base-specific tokens (AERO, DEGEN,
|
||||
> TOSHI, BRETT, WELL, cbETH, cbBTC, wstETH, rETH) and all Base RPC functionality
|
||||
> previously living under `optional-skills/blockchain/base/` have been folded
|
||||
> into this skill. Pass `--chain base` to any command for Base coverage.
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
- User asks for a wallet balance or portfolio on any EVM chain
|
||||
- User wants to check the same wallet across ALL chains at once
|
||||
- User wants to inspect a transaction by hash (or decode what it did)
|
||||
- User wants ERC-20 token metadata, price, supply, or market cap
|
||||
- User wants recent transaction history for an address
|
||||
- User wants current gas prices or to compare fees across chains
|
||||
- User wants to find large whale transfers in recent blocks
|
||||
- User asks to resolve an ENS name (vitalik.eth) or reverse-lookup an address
|
||||
- User wants to check if a contract has dangerous token approvals
|
||||
- User wants to inspect a smart contract (proxy? ERC-20? ERC-721? bytecode size?)
|
||||
- User wants to compare gas costs across chains before a transaction
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
Python 3.8+ standard library only. No pip installs required.
|
||||
Pricing: CoinGecko free API (rate-limited, ~10-30 req/min).
|
||||
ENS: ensideas.com public API.
|
||||
Tx decoding: 4byte.directory public API.
|
||||
|
||||
Override RPC endpoint: `export EVM_RPC_URL=https://your-rpc.com`
|
||||
|
||||
Helper script path: `~/.hermes/skills/blockchain/evm/scripts/evm_client.py`
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
```
|
||||
SCRIPT=~/.hermes/skills/blockchain/evm/scripts/evm_client.py
|
||||
|
||||
# Network & prices
|
||||
python3 $SCRIPT stats # Ethereum stats
|
||||
python3 $SCRIPT stats --chain arbitrum # Arbitrum stats
|
||||
python3 $SCRIPT compare # Gas + prices ALL 8 chains
|
||||
|
||||
# Wallet
|
||||
python3 $SCRIPT wallet 0xd8dA...96045 # Portfolio (ETH + ERC-20)
|
||||
python3 $SCRIPT wallet 0xd8dA...96045 --chain bsc
|
||||
python3 $SCRIPT multichain 0xd8dA...96045 # Same wallet on ALL chains
|
||||
|
||||
# Tokens & prices
|
||||
python3 $SCRIPT price ETH
|
||||
python3 $SCRIPT price 0xdAC1...1ec7 # By contract address
|
||||
python3 $SCRIPT token 0xdAC1...1ec7 # ERC-20 metadata + market cap
|
||||
|
||||
# Transactions
|
||||
python3 $SCRIPT tx 0x5c50...f060 # Transaction details
|
||||
python3 $SCRIPT decode 0x5c50...f060 # Decode input data (4byte.directory)
|
||||
python3 $SCRIPT activity 0xd8dA...96045 # Recent transactions
|
||||
|
||||
# Gas
|
||||
python3 $SCRIPT gas # Gas prices + cost estimates
|
||||
python3 $SCRIPT gas --chain optimism
|
||||
|
||||
# Security
|
||||
python3 $SCRIPT allowance 0xd8dA...96045 # Dangerous ERC-20 approvals
|
||||
python3 $SCRIPT contract 0xdAC1...1ec7 # Contract inspection (proxy? standards?)
|
||||
|
||||
# ENS
|
||||
python3 $SCRIPT ens vitalik.eth # Name -> address + profile
|
||||
python3 $SCRIPT ens 0xd8dA...96045 # Address -> ENS name
|
||||
|
||||
# Whale detection
|
||||
python3 $SCRIPT whale # Large transfers (last 20 blocks, >$10k)
|
||||
python3 $SCRIPT whale --blocks 50 --min-usd 100000 --chain arbitrum
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Procedure
|
||||
|
||||
### 0. Setup Check
|
||||
```bash
|
||||
python3 --version # 3.8+ required
|
||||
python3 ~/.hermes/skills/blockchain/evm/scripts/evm_client.py stats
|
||||
```
|
||||
|
||||
### 1. Wallet Portfolio
|
||||
Native balance + known ERC-20 tokens, sorted by USD value.
|
||||
```bash
|
||||
python3 $SCRIPT wallet 0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045
|
||||
python3 $SCRIPT wallet 0xd8dA... --chain bsc --no-prices # faster
|
||||
```
|
||||
|
||||
### 2. Multi-Chain Scan
|
||||
Scans all 8 chains simultaneously for the same address using threads.
|
||||
```bash
|
||||
python3 $SCRIPT multichain 0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045
|
||||
```
|
||||
Output: per-chain native balance + token holdings + grand total USD.
|
||||
|
||||
### 3. Compare (Gas + Prices)
|
||||
All 8 chains queried in parallel. Shows cheapest/most expensive chain.
|
||||
```bash
|
||||
python3 $SCRIPT compare
|
||||
```
|
||||
|
||||
### 4. Transaction Details & Decode
|
||||
```bash
|
||||
python3 $SCRIPT tx 0x5c504ed432cb51138bcf09aa5e8a410dd4a1e204ef84bfed1be16dfba1b22060
|
||||
python3 $SCRIPT decode 0x5c504ed... # Shows human-readable function signature
|
||||
```
|
||||
Decode uses 4byte.directory to translate 0xa9059cbb -> transfer(address,uint256).
|
||||
|
||||
### 5. ENS Resolution
|
||||
```bash
|
||||
python3 $SCRIPT ens vitalik.eth # -> 0xd8dA... + avatar + social links
|
||||
python3 $SCRIPT ens 0xd8dA...96045 # -> vitalik.eth
|
||||
```
|
||||
|
||||
### 6. Allowance Checker (Security)
|
||||
Checks ERC-20 approvals granted to known DEX/bridge contracts.
|
||||
```bash
|
||||
python3 $SCRIPT allowance 0xYourWallet
|
||||
```
|
||||
Flags UNLIMITED approvals as HIGH risk.
|
||||
|
||||
### 7. Contract Inspector
|
||||
```bash
|
||||
python3 $SCRIPT contract 0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48 # USDC (proxy)
|
||||
python3 $SCRIPT contract 0xdAC17F958D2ee523a2206206994597C13D831ec7 # USDT (ERC-20)
|
||||
```
|
||||
Detects: proxy (EIP-1967/EIP-1167), ERC-20, ERC-721, ERC-165. Shows bytecode size and implementation address for proxies.
|
||||
|
||||
### 8. Whale Detection
|
||||
```bash
|
||||
python3 $SCRIPT whale # ETH, last 20 blocks, >$10k
|
||||
python3 $SCRIPT whale --blocks 50 --min-usd 50000 --chain bsc
|
||||
```
|
||||
|
||||
### 9. Gas Tracker
|
||||
```bash
|
||||
python3 $SCRIPT gas
|
||||
python3 $SCRIPT gas --chain polygon
|
||||
```
|
||||
Shows gwei price + USD cost for: transfer, ERC-20 transfer, approve, swap, NFT mint, NFT transfer.
|
||||
|
||||
---
|
||||
|
||||
## Supported Chains
|
||||
| Key | Name | Native | Chain ID |
|
||||
|-----------|----------------|--------|----------|
|
||||
| ethereum | Ethereum | ETH | 1 |
|
||||
| bsc | BNB Chain | BNB | 56 |
|
||||
| base | Base | ETH | 8453 |
|
||||
| arbitrum | Arbitrum One | ETH | 42161 |
|
||||
| polygon | Polygon | POL | 137 |
|
||||
| optimism | Optimism | ETH | 10 |
|
||||
| avalanche | Avalanche C | AVAX | 43114 |
|
||||
| zksync | zkSync Era | ETH | 324 |
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
- CoinGecko free tier: ~10-30 req/min. Use `--no-prices` for faster wallet scans.
|
||||
- Public RPCs may throttle. Set EVM_RPC_URL to a private endpoint for production.
|
||||
- `wallet` and `allowance` only check known token list (~30 tokens per chain). Use a block explorer for complete token discovery.
|
||||
- `activity` scans recent blocks only (max 200). For full history, use Etherscan API.
|
||||
- `multichain` runs 8 parallel threads — can trigger rate limits on public RPCs.
|
||||
- ENS resolution depends on a single public endpoint (ensideas.com / ens.vitalik.ca) with no fallback. If that endpoint is down, `ens` will fail — re-run later or use a block explorer.
|
||||
- Tx decoding depends on a single public endpoint (4byte.directory) with no fallback. Selectors not in their database show up as `unknown`.
|
||||
- **L2 gas estimates are L2-execution only.** On rollups like Base, Arbitrum, Optimism, and zkSync, the actual transaction cost also includes an L1 data-posting fee that depends on calldata size and current L1 gas prices. The `gas` command does not estimate that L1 component. For Base specifically, see the network's L1 fee oracle (contract `0x420000000000000000000000000000000000000F`).
|
||||
- Address / tx-hash inputs are validated for 0x-prefix + correct length + hex, but EIP-55 checksum casing is **not** enforced (RPC endpoints accept any-case hex).
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
```bash
|
||||
# Should print current block, gas price, ETH price
|
||||
python3 ~/.hermes/skills/blockchain/evm/scripts/evm_client.py stats
|
||||
|
||||
# Should resolve vitalik.eth to 0xd8dA...
|
||||
python3 ~/.hermes/skills/blockchain/evm/scripts/evm_client.py ens vitalik.eth
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,211 @@
|
||||
---
|
||||
name: hyperliquid
|
||||
description: Hyperliquid market data, account history, trade review.
|
||||
version: 0.1.0
|
||||
author: Hugo Sequier (Hugo-SEQUIER), Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Hyperliquid, Blockchain, Crypto, Trading, Perpetuals, Spot, DeFi]
|
||||
related_skills: []
|
||||
---
|
||||
|
||||
# Hyperliquid Skill
|
||||
|
||||
Query Hyperliquid market and account data through the public `/info` endpoint.
|
||||
Read-only — no API key, no signing, no order placement.
|
||||
|
||||
12 commands: `dexs`, `markets`, `spots`, `candles`, `funding`, `l2`, `state`,
|
||||
`spot-balances`, `fills`, `orders`, `review`, `export`. Stdlib only
|
||||
(`urllib`, `json`, `argparse`).
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks for Hyperliquid perp or spot market data, candles, funding, or L2 book
|
||||
- User wants to inspect a wallet's perp positions, spot balances, fills, or orders
|
||||
- User wants a post-trade review combining recent fills with market context
|
||||
- User wants to inspect builder-deployed perp dexs or HIP-3 markets
|
||||
- User wants a normalized JSON export of candles + funding for backtesting prep
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Stdlib only — no external packages, no API key.
|
||||
|
||||
The script reads `~/.hermes/.env` for two optional defaults:
|
||||
|
||||
- `HYPERLIQUID_API_URL` — defaults to `https://api.hyperliquid.xyz`. Set to
|
||||
`https://api.hyperliquid-testnet.xyz` for testnet.
|
||||
- `HYPERLIQUID_USER_ADDRESS` — default address for `state`, `spot-balances`,
|
||||
`fills`, `orders`, and `review`. If unset, pass the address as the first
|
||||
positional argument.
|
||||
|
||||
A project `.env` in the current working directory is honored as a dev fallback.
|
||||
|
||||
Helper script: `~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py`
|
||||
|
||||
---
|
||||
|
||||
## How to Run
|
||||
|
||||
Invoke through the `terminal` tool:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py <command> [args]
|
||||
```
|
||||
|
||||
Add `--json` to any command for machine-readable output.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
```bash
|
||||
hyperliquid_client.py dexs
|
||||
hyperliquid_client.py markets [--dex DEX] [--limit N] [--sort volume|oi|funding_abs|change_abs|name]
|
||||
hyperliquid_client.py spots [--limit N]
|
||||
hyperliquid_client.py candles <coin> [--interval 1h] [--hours 24] [--limit N]
|
||||
hyperliquid_client.py funding <coin> [--hours 72] [--limit N]
|
||||
hyperliquid_client.py l2 <coin> [--levels N]
|
||||
hyperliquid_client.py state [address] [--dex DEX]
|
||||
hyperliquid_client.py spot-balances [address] [--limit N]
|
||||
hyperliquid_client.py fills [address] [--hours N] [--limit N] [--aggregate-by-time]
|
||||
hyperliquid_client.py orders [address] [--limit N]
|
||||
hyperliquid_client.py review [address] [--coin COIN] [--hours N] [--fills N]
|
||||
hyperliquid_client.py export <coin> [--interval 1h] [--hours N] [--output PATH]
|
||||
```
|
||||
|
||||
For `state`, `spot-balances`, `fills`, `orders`, and `review`, the address is
|
||||
optional when `HYPERLIQUID_USER_ADDRESS` is set in `~/.hermes/.env`.
|
||||
|
||||
---
|
||||
|
||||
## Procedure
|
||||
|
||||
### 1. Discover DEXs and Markets
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py dexs
|
||||
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
markets --limit 15 --sort volume
|
||||
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
spots --limit 15
|
||||
```
|
||||
|
||||
- `--dex` only applies to perp endpoints; omit for the first perp dex.
|
||||
- Spot pairs may show as `PURR/USDC` or aliases like `@107`.
|
||||
- HIP-3 markets prefix the coin with the dex, e.g. `mydex:BTC`.
|
||||
|
||||
### 2. Pull Historical Market Data
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
candles BTC --interval 1h --hours 72 --limit 48
|
||||
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
funding BTC --hours 168 --limit 30
|
||||
```
|
||||
|
||||
Time-range endpoints paginate. For larger windows, repeat with a later
|
||||
`startTime` or use `export` (below).
|
||||
|
||||
### 3. Inspect Live Order Book
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
l2 BTC --levels 10
|
||||
```
|
||||
|
||||
Use when asked about book depth, near-term liquidity, or potential market
|
||||
impact of a large order.
|
||||
|
||||
### 4. Review an Account
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
state 0xabc...
|
||||
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
spot-balances
|
||||
```
|
||||
|
||||
`state` returns perp positions; `spot-balances` returns spot inventory.
|
||||
Use these for "how are my positions?", "what am I holding?", "how much is
|
||||
withdrawable?".
|
||||
|
||||
### 5. Review Fills and Orders
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
fills 0xabc... --hours 72 --limit 25
|
||||
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
orders --limit 25
|
||||
```
|
||||
|
||||
### 6. Generate a Trade Review
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
review 0xabc... --hours 72 --fills 50
|
||||
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
review --coin BTC --hours 168
|
||||
```
|
||||
|
||||
Reports realized PnL, fees, win/loss counts, coin breakdowns, market trend
|
||||
and average funding for each traded perp, plus heuristics (fee drag,
|
||||
concentration, counter-trend losses).
|
||||
|
||||
For deeper post-trade analysis: start with `review` to find problem coins
|
||||
or windows → pull `fills` and `orders` for that period → pull `candles`
|
||||
and `funding` for each traded coin → judge decision quality separately
|
||||
from outcome quality.
|
||||
|
||||
### 7. Export a Reusable Dataset
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
export BTC --interval 1h --hours 168 --output ./btc-1h-7d.json
|
||||
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
export BTC --interval 15m --hours 72 --end-time-ms 1760000000000
|
||||
```
|
||||
|
||||
Output JSON contains: schema version, source metadata, exact time window,
|
||||
normalized candle rows, normalized funding rows, summary stats. Use
|
||||
`--end-time-ms` for reproducible windows.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Public info endpoints are rate-limited. Large historical queries may
|
||||
return capped windows; iterate with later `startTime` values.
|
||||
- `fills --hours ...` uses `userFillsByTime`, which only exposes a
|
||||
recent rolling window — not full archive history.
|
||||
- `historicalOrders` returns recent orders only; not a full export.
|
||||
- The `review` command is heuristic. It cannot reconstruct intent,
|
||||
order placement quality, or true slippage from fills alone.
|
||||
- The `export` command writes a normalized dataset, not a backtest
|
||||
engine. You still need your own slippage/fill model.
|
||||
- Spot aliases like `@107` are valid identifiers even when the UI shows
|
||||
a friendlier name.
|
||||
- `l2` is a point-in-time snapshot, not a time series.
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/hyperliquid/scripts/hyperliquid_client.py \
|
||||
markets --limit 5
|
||||
```
|
||||
|
||||
Should print the top Hyperliquid perp markets by 24h notional volume.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,208 @@
|
||||
---
|
||||
name: solana
|
||||
description: Query Solana blockchain data with USD pricing — wallet balances, token portfolios with values, transaction details, NFTs, whale detection, and live network stats. Uses Solana RPC + CoinGecko. No API key required.
|
||||
version: 0.2.0
|
||||
author: Deniz Alagoz (gizdusum), enhanced by Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Solana, Blockchain, Crypto, Web3, RPC, DeFi, NFT]
|
||||
related_skills: []
|
||||
---
|
||||
|
||||
# Solana Blockchain Skill
|
||||
|
||||
Query Solana on-chain data enriched with USD pricing via CoinGecko.
|
||||
8 commands: wallet portfolio, token info, transactions, activity, NFTs,
|
||||
whale detection, network stats, and price lookup.
|
||||
|
||||
No API key needed. Uses only Python standard library (urllib, json, argparse).
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks for a Solana wallet balance, token holdings, or portfolio value
|
||||
- User wants to inspect a specific transaction by signature
|
||||
- User wants SPL token metadata, price, supply, or top holders
|
||||
- User wants recent transaction history for an address
|
||||
- User wants NFTs owned by a wallet
|
||||
- User wants to find large SOL transfers (whale detection)
|
||||
- User wants Solana network health, TPS, epoch, or SOL price
|
||||
- User asks "what's the price of BONK/JUP/SOL?"
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The helper script uses only Python standard library (urllib, json, argparse).
|
||||
No external packages required.
|
||||
|
||||
Pricing data comes from CoinGecko's free API (no key needed, rate-limited
|
||||
to ~10-30 requests/minute). For faster lookups, use `--no-prices` flag.
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
RPC endpoint (default): https://api.mainnet-beta.solana.com
|
||||
Override: export SOLANA_RPC_URL=https://your-private-rpc.com
|
||||
|
||||
Helper script path: ~/.hermes/skills/blockchain/solana/scripts/solana_client.py
|
||||
|
||||
```
|
||||
python3 solana_client.py wallet <address> [--limit N] [--all] [--no-prices]
|
||||
python3 solana_client.py tx <signature>
|
||||
python3 solana_client.py token <mint_address>
|
||||
python3 solana_client.py activity <address> [--limit N]
|
||||
python3 solana_client.py nft <address>
|
||||
python3 solana_client.py whales [--min-sol N]
|
||||
python3 solana_client.py stats
|
||||
python3 solana_client.py price <mint_or_symbol>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Procedure
|
||||
|
||||
### 0. Setup Check
|
||||
|
||||
```bash
|
||||
python3 --version
|
||||
|
||||
# Optional: set a private RPC for better rate limits
|
||||
export SOLANA_RPC_URL="https://api.mainnet-beta.solana.com"
|
||||
|
||||
# Confirm connectivity
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py stats
|
||||
```
|
||||
|
||||
### 1. Wallet Portfolio
|
||||
|
||||
Get SOL balance, SPL token holdings with USD values, NFT count, and
|
||||
portfolio total. Tokens sorted by value, dust filtered, known tokens
|
||||
labeled by name (BONK, JUP, USDC, etc.).
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py \
|
||||
wallet 9WzDXwBbmkg8ZTbNMqUxvQRAyrZzDsGYdLVL9zYtAWWM
|
||||
```
|
||||
|
||||
Flags:
|
||||
- `--limit N` — show top N tokens (default: 20)
|
||||
- `--all` — show all tokens, no dust filter, no limit
|
||||
- `--no-prices` — skip CoinGecko price lookups (faster, RPC-only)
|
||||
|
||||
Output includes: SOL balance + USD value, token list with prices sorted
|
||||
by value, dust count, NFT summary, total portfolio value in USD.
|
||||
|
||||
### 2. Transaction Details
|
||||
|
||||
Inspect a full transaction by its base58 signature. Shows balance changes
|
||||
in both SOL and USD.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py \
|
||||
tx 5j7s8K...your_signature_here
|
||||
```
|
||||
|
||||
Output: slot, timestamp, fee, status, balance changes (SOL + USD),
|
||||
program invocations.
|
||||
|
||||
### 3. Token Info
|
||||
|
||||
Get SPL token metadata, current price, market cap, supply, decimals,
|
||||
mint/freeze authorities, and top 5 holders.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py \
|
||||
token DezXAZ8z7PnrnRJjz3wXBoRgixCa6xjnB7YaB1pPB263
|
||||
```
|
||||
|
||||
Output: name, symbol, decimals, supply, price, market cap, top 5
|
||||
holders with percentages.
|
||||
|
||||
### 4. Recent Activity
|
||||
|
||||
List recent transactions for an address (default: last 10, max: 25).
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py \
|
||||
activity 9WzDXwBbmkg8ZTbNMqUxvQRAyrZzDsGYdLVL9zYtAWWM --limit 25
|
||||
```
|
||||
|
||||
### 5. NFT Portfolio
|
||||
|
||||
List NFTs owned by a wallet (heuristic: SPL tokens with amount=1, decimals=0).
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py \
|
||||
nft 9WzDXwBbmkg8ZTbNMqUxvQRAyrZzDsGYdLVL9zYtAWWM
|
||||
```
|
||||
|
||||
Note: Compressed NFTs (cNFTs) are not detected by this heuristic.
|
||||
|
||||
### 6. Whale Detector
|
||||
|
||||
Scan the most recent block for large SOL transfers with USD values.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py \
|
||||
whales --min-sol 500
|
||||
```
|
||||
|
||||
Note: scans the latest block only — point-in-time snapshot, not historical.
|
||||
|
||||
### 7. Network Stats
|
||||
|
||||
Live Solana network health: current slot, epoch, TPS, supply, validator
|
||||
version, SOL price, and market cap.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py stats
|
||||
```
|
||||
|
||||
### 8. Price Lookup
|
||||
|
||||
Quick price check for any token by mint address or known symbol.
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py price BONK
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py price JUP
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py price SOL
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py price DezXAZ8z7PnrnRJjz3wXBoRgixCa6xjnB7YaB1pPB263
|
||||
```
|
||||
|
||||
Known symbols: SOL, USDC, USDT, BONK, JUP, WETH, JTO, mSOL, stSOL,
|
||||
PYTH, HNT, RNDR, WEN, W, TNSR, DRIFT, bSOL, JLP, WIF, MEW, BOME, PENGU.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **CoinGecko rate-limits** — free tier allows ~10-30 requests/minute.
|
||||
Price lookups use 1 request per token. Wallets with many tokens may
|
||||
not get prices for all of them. Use `--no-prices` for speed.
|
||||
- **Public RPC rate-limits** — Solana mainnet public RPC limits requests.
|
||||
For production use, set SOLANA_RPC_URL to a private endpoint
|
||||
(Helius, QuickNode, Triton).
|
||||
- **NFT detection is heuristic** — amount=1 + decimals=0. Compressed
|
||||
NFTs (cNFTs) and Token-2022 NFTs won't appear.
|
||||
- **Whale detector scans latest block only** — not historical. Results
|
||||
vary by the moment you query.
|
||||
- **Transaction history** — public RPC keeps ~2 days. Older transactions
|
||||
may not be available.
|
||||
- **Token names** — ~25 well-known tokens are labeled by name. Others
|
||||
show abbreviated mint addresses. Use the `token` command for full info.
|
||||
- **Retry on 429** — both RPC and CoinGecko calls retry up to 2 times
|
||||
with exponential backoff on rate-limit errors.
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# Should print current Solana slot, TPS, and SOL price
|
||||
python3 ~/.hermes/skills/blockchain/solana/scripts/solana_client.py stats
|
||||
```
|
||||
@@ -0,0 +1,698 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Solana Blockchain CLI Tool for Hermes Agent
|
||||
--------------------------------------------
|
||||
Queries the Solana JSON-RPC API and CoinGecko for enriched on-chain data.
|
||||
Uses only Python standard library — no external packages required.
|
||||
|
||||
Usage:
|
||||
python3 solana_client.py stats
|
||||
python3 solana_client.py wallet <address> [--limit N] [--all] [--no-prices]
|
||||
python3 solana_client.py tx <signature>
|
||||
python3 solana_client.py token <mint_address>
|
||||
python3 solana_client.py activity <address> [--limit N]
|
||||
python3 solana_client.py nft <address>
|
||||
python3 solana_client.py whales [--min-sol N]
|
||||
python3 solana_client.py price <mint_address_or_symbol>
|
||||
|
||||
Environment:
|
||||
SOLANA_RPC_URL Override the default RPC endpoint (default: mainnet-beta public)
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
import urllib.error
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
RPC_URL = os.environ.get(
|
||||
"SOLANA_RPC_URL",
|
||||
"https://api.mainnet-beta.solana.com",
|
||||
)
|
||||
|
||||
LAMPORTS_PER_SOL = 1_000_000_000
|
||||
|
||||
# Well-known Solana token names — avoids API calls for common tokens.
|
||||
# Maps mint address → (symbol, name).
|
||||
KNOWN_TOKENS: Dict[str, tuple] = {
|
||||
"So11111111111111111111111111111111111111112": ("SOL", "Solana"),
|
||||
"EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v": ("USDC", "USD Coin"),
|
||||
"Es9vMFrzaCERmJfrF4H2FYD4KCoNkY11McCe8BenwNYB": ("USDT", "Tether"),
|
||||
"DezXAZ8z7PnrnRJjz3wXBoRgixCa6xjnB7YaB1pPB263": ("BONK", "Bonk"),
|
||||
"JUPyiwrYJFskUPiHa7hkeR8VUtAeFoSYbKedZNsDvCN": ("JUP", "Jupiter"),
|
||||
"7vfCXTUXx5WJV5JADk17DUJ4ksgau7utNKj4b963voxs": ("WETH", "Wrapped Ether"),
|
||||
"jtojtomepa8beP8AuQc6eXt5FriJwfFMwQx2v2f9mCL": ("JTO", "Jito"),
|
||||
"mSoLzYCxHdYgdzU16g5QSh3i5K3z3KZK7ytfqcJm7So": ("mSOL", "Marinade Staked SOL"),
|
||||
"7dHbWXmci3dT8UFYWYZweBLXgycu7Y3iL6trKn1Y7ARj": ("stSOL", "Lido Staked SOL"),
|
||||
"HZ1JovNiVvGrGNiiYvEozEVgZ58xaU3RKwX8eACQBCt3": ("PYTH", "Pyth Network"),
|
||||
"RLBxxFkseAZ4RgJH3Sqn8jXxhmGoz9jWxDNJMh8pL7a": ("RLBB", "Rollbit"),
|
||||
"hntyVP6YFm1Hg25TN9WGLqM12b8TQmcknKrdu1oxWux": ("HNT", "Helium"),
|
||||
"rndrizKT3MK1iimdxRdWabcF7Zg7AR5T4nud4EkHBof": ("RNDR", "Render"),
|
||||
"WENWENvqqNya429ubCdR81ZmD69brwQaaBYY6p91oHQQ": ("WEN", "Wen"),
|
||||
"85VBFQZC9TZkfaptBWjvUw7YbZjy52A6mjtPGjstQAmQ": ("W", "Wormhole"),
|
||||
"TNSRxcUxoT9xBG3de7PiJyTDYu7kskLqcpddxnEJAS6": ("TNSR", "Tensor"),
|
||||
"DriFtupJYLTosbwoN8koMbEYSx54aFAVLddWsbksjwg7": ("DRIFT", "Drift"),
|
||||
"bSo13r4TkiE4KumL71LsHTPpL2euBYLFx6h9HP3piy1": ("bSOL", "BlazeStake Staked SOL"),
|
||||
"27G8MtK7VtTcCHkpASjSDdkWWYfoqT6ggEuKidVJidD4": ("JLP", "Jupiter LP"),
|
||||
"EKpQGSJtjMFqKZ9KQanSqYXRcF8fBopzLHYxdM65zcjm": ("WIF", "dogwifhat"),
|
||||
"MEW1gQWJ3nEXg2qgERiKu7FAFj79PHvQVREQUzScPP5": ("MEW", "cat in a dogs world"),
|
||||
"ukHH6c7mMyiWCf1b9pnWe25TSpkDDt3H5pQZgZ74J82": ("BOME", "Book of Meme"),
|
||||
"A8C3xuqscfmyLrte3VwJvtPHXvcSN3FjDbUaSMAkQrCS": ("PENGU", "Pudgy Penguins"),
|
||||
}
|
||||
|
||||
# Reverse lookup: symbol → mint (for the `price` command).
|
||||
_SYMBOL_TO_MINT = {v[0].upper(): k for k, v in KNOWN_TOKENS.items()}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# HTTP / RPC helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _http_get_json(url: str, timeout: int = 10, retries: int = 2) -> Any:
|
||||
"""GET JSON from a URL with retry on 429 rate-limit. Returns parsed JSON or None."""
|
||||
for attempt in range(retries + 1):
|
||||
req = urllib.request.Request(
|
||||
url, headers={"Accept": "application/json", "User-Agent": "HermesAgent/1.0"},
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
return json.load(resp)
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 429 and attempt < retries:
|
||||
time.sleep(2.0 * (attempt + 1))
|
||||
continue
|
||||
return None
|
||||
except Exception:
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def _rpc_call(method: str, params: list = None, retries: int = 2) -> Any:
|
||||
"""Send a JSON-RPC request with retry on 429 rate-limit."""
|
||||
payload = json.dumps({
|
||||
"jsonrpc": "2.0", "id": 1,
|
||||
"method": method, "params": params or [],
|
||||
}).encode()
|
||||
|
||||
for attempt in range(retries + 1):
|
||||
req = urllib.request.Request(
|
||||
RPC_URL, data=payload,
|
||||
headers={"Content-Type": "application/json"}, method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=20) as resp:
|
||||
body = json.load(resp)
|
||||
if "error" in body:
|
||||
err = body["error"]
|
||||
# Rate-limit: retry after delay
|
||||
if isinstance(err, dict) and err.get("code") == 429:
|
||||
if attempt < retries:
|
||||
time.sleep(1.5 * (attempt + 1))
|
||||
continue
|
||||
sys.exit(f"RPC error: {err}")
|
||||
return body.get("result")
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 429 and attempt < retries:
|
||||
time.sleep(1.5 * (attempt + 1))
|
||||
continue
|
||||
sys.exit(f"RPC HTTP error: {exc}")
|
||||
except urllib.error.URLError as exc:
|
||||
sys.exit(f"RPC connection error: {exc}")
|
||||
return None
|
||||
|
||||
|
||||
# Keep backward compat — the rest of the code uses `rpc()`.
|
||||
rpc = _rpc_call
|
||||
|
||||
|
||||
def rpc_batch(calls: list) -> list:
|
||||
"""Send a batch of JSON-RPC requests (with retry on 429)."""
|
||||
payload = json.dumps([
|
||||
{"jsonrpc": "2.0", "id": i, "method": c["method"], "params": c.get("params", [])}
|
||||
for i, c in enumerate(calls)
|
||||
]).encode()
|
||||
|
||||
for attempt in range(3):
|
||||
req = urllib.request.Request(
|
||||
RPC_URL, data=payload,
|
||||
headers={"Content-Type": "application/json"}, method="POST",
|
||||
)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=20) as resp:
|
||||
return json.load(resp)
|
||||
except urllib.error.HTTPError as exc:
|
||||
if exc.code == 429 and attempt < 2:
|
||||
time.sleep(1.5 * (attempt + 1))
|
||||
continue
|
||||
sys.exit(f"RPC batch HTTP error: {exc}")
|
||||
except urllib.error.URLError as exc:
|
||||
sys.exit(f"RPC batch error: {exc}")
|
||||
return []
|
||||
|
||||
|
||||
def lamports_to_sol(lamports: int) -> float:
|
||||
return lamports / LAMPORTS_PER_SOL
|
||||
|
||||
|
||||
def print_json(obj: Any) -> None:
|
||||
print(json.dumps(obj, indent=2))
|
||||
|
||||
|
||||
def _short_mint(mint: str) -> str:
|
||||
"""Abbreviate a mint address for display: first 4 + last 4."""
|
||||
if len(mint) <= 12:
|
||||
return mint
|
||||
return f"{mint[:4]}...{mint[-4:]}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Price & token name helpers (CoinGecko — free, no API key)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def fetch_prices(mints: List[str], max_lookups: int = 20) -> Dict[str, float]:
|
||||
"""Fetch USD prices for mint addresses via CoinGecko (one per request).
|
||||
|
||||
CoinGecko free tier doesn't support batch Solana token lookups,
|
||||
so we do individual calls — capped at *max_lookups* to stay within
|
||||
rate limits. Returns {mint: usd_price}.
|
||||
"""
|
||||
prices: Dict[str, float] = {}
|
||||
for i, mint in enumerate(mints[:max_lookups]):
|
||||
url = (
|
||||
f"https://api.coingecko.com/api/v3/simple/token_price/solana"
|
||||
f"?contract_addresses={mint}&vs_currencies=usd"
|
||||
)
|
||||
data = _http_get_json(url, timeout=10)
|
||||
if data and isinstance(data, dict):
|
||||
for addr, info in data.items():
|
||||
if isinstance(info, dict) and "usd" in info:
|
||||
prices[mint] = info["usd"]
|
||||
break
|
||||
# Pause between calls to respect CoinGecko free-tier rate-limits
|
||||
if i < len(mints[:max_lookups]) - 1:
|
||||
time.sleep(1.0)
|
||||
return prices
|
||||
|
||||
|
||||
def fetch_sol_price() -> Optional[float]:
|
||||
"""Fetch current SOL price in USD via CoinGecko."""
|
||||
data = _http_get_json(
|
||||
"https://api.coingecko.com/api/v3/simple/price?ids=solana&vs_currencies=usd"
|
||||
)
|
||||
if data and "solana" in data:
|
||||
return data["solana"].get("usd")
|
||||
return None
|
||||
|
||||
|
||||
def resolve_token_name(mint: str) -> Optional[Dict[str, str]]:
|
||||
"""Look up token name and symbol from CoinGecko by mint address.
|
||||
|
||||
Returns {"name": ..., "symbol": ...} or None.
|
||||
"""
|
||||
if mint in KNOWN_TOKENS:
|
||||
sym, name = KNOWN_TOKENS[mint]
|
||||
return {"symbol": sym, "name": name}
|
||||
url = f"https://api.coingecko.com/api/v3/coins/solana/contract/{mint}"
|
||||
data = _http_get_json(url, timeout=10)
|
||||
if data and "symbol" in data:
|
||||
return {"symbol": data["symbol"].upper(), "name": data.get("name", "")}
|
||||
return None
|
||||
|
||||
|
||||
def _token_label(mint: str) -> str:
|
||||
"""Return a human-readable label for a mint: symbol if known, else abbreviated address."""
|
||||
if mint in KNOWN_TOKENS:
|
||||
return KNOWN_TOKENS[mint][0]
|
||||
return _short_mint(mint)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. Network Stats
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_stats(_args):
|
||||
"""Live Solana network: slot, epoch, TPS, supply, version, SOL price."""
|
||||
results = rpc_batch([
|
||||
{"method": "getSlot"},
|
||||
{"method": "getEpochInfo"},
|
||||
{"method": "getRecentPerformanceSamples", "params": [1]},
|
||||
{"method": "getSupply"},
|
||||
{"method": "getVersion"},
|
||||
])
|
||||
|
||||
by_id = {r["id"]: r.get("result") for r in results}
|
||||
|
||||
slot = by_id.get(0)
|
||||
epoch_info = by_id.get(1)
|
||||
perf_samples = by_id.get(2)
|
||||
supply = by_id.get(3)
|
||||
version = by_id.get(4)
|
||||
|
||||
tps = None
|
||||
if perf_samples:
|
||||
s = perf_samples[0]
|
||||
tps = round(s["numTransactions"] / s["samplePeriodSecs"], 1)
|
||||
|
||||
total_supply = lamports_to_sol(supply["value"]["total"]) if supply else None
|
||||
circ_supply = lamports_to_sol(supply["value"]["circulating"]) if supply else None
|
||||
|
||||
sol_price = fetch_sol_price()
|
||||
|
||||
out = {
|
||||
"slot": slot,
|
||||
"epoch": epoch_info.get("epoch") if epoch_info else None,
|
||||
"slot_in_epoch": epoch_info.get("slotIndex") if epoch_info else None,
|
||||
"tps": tps,
|
||||
"total_supply_SOL": round(total_supply, 2) if total_supply else None,
|
||||
"circulating_supply_SOL": round(circ_supply, 2) if circ_supply else None,
|
||||
"validator_version": version.get("solana-core") if version else None,
|
||||
}
|
||||
if sol_price is not None:
|
||||
out["sol_price_usd"] = sol_price
|
||||
if circ_supply:
|
||||
out["market_cap_usd"] = round(sol_price * circ_supply, 0)
|
||||
print_json(out)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Wallet Info (enhanced with prices, sorting, filtering)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_wallet(args):
|
||||
"""SOL balance + SPL token holdings with USD values."""
|
||||
address = args.address
|
||||
show_all = getattr(args, "all", False)
|
||||
limit = getattr(args, "limit", 20) or 20
|
||||
skip_prices = getattr(args, "no_prices", False)
|
||||
|
||||
# Fetch SOL balance
|
||||
balance_result = rpc("getBalance", [address])
|
||||
sol_balance = lamports_to_sol(balance_result["value"])
|
||||
|
||||
# Fetch all SPL token accounts
|
||||
token_result = rpc("getTokenAccountsByOwner", [
|
||||
address,
|
||||
{"programId": "TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA"},
|
||||
{"encoding": "jsonParsed"},
|
||||
])
|
||||
|
||||
raw_tokens = []
|
||||
for acct in (token_result.get("value") or []):
|
||||
info = acct["account"]["data"]["parsed"]["info"]
|
||||
ta = info["tokenAmount"]
|
||||
amount = float(ta.get("uiAmountString") or 0)
|
||||
if amount > 0:
|
||||
raw_tokens.append({
|
||||
"mint": info["mint"],
|
||||
"amount": amount,
|
||||
"decimals": ta["decimals"],
|
||||
})
|
||||
|
||||
# Separate NFTs (amount=1, decimals=0) from fungible tokens
|
||||
nfts = [t for t in raw_tokens if t["decimals"] == 0 and t["amount"] == 1]
|
||||
fungible = [t for t in raw_tokens if not (t["decimals"] == 0 and t["amount"] == 1)]
|
||||
|
||||
# Fetch prices for fungible tokens (cap lookups to avoid API abuse)
|
||||
sol_price = None
|
||||
prices: Dict[str, float] = {}
|
||||
if not skip_prices and fungible:
|
||||
sol_price = fetch_sol_price()
|
||||
# Prioritize known tokens, then a small sample of unknowns.
|
||||
# CoinGecko free tier = 1 request per mint, so we cap lookups.
|
||||
known_mints = [t["mint"] for t in fungible if t["mint"] in KNOWN_TOKENS]
|
||||
other_mints = [t["mint"] for t in fungible if t["mint"] not in KNOWN_TOKENS][:15]
|
||||
mints_to_price = known_mints + other_mints
|
||||
if mints_to_price:
|
||||
prices = fetch_prices(mints_to_price, max_lookups=30)
|
||||
|
||||
# Enrich tokens with labels and USD values
|
||||
enriched = []
|
||||
dust_count = 0
|
||||
dust_value = 0.0
|
||||
for t in fungible:
|
||||
mint = t["mint"]
|
||||
label = _token_label(mint)
|
||||
usd_price = prices.get(mint)
|
||||
usd_value = round(usd_price * t["amount"], 2) if usd_price else None
|
||||
|
||||
# Filter dust (< $0.01) unless --all
|
||||
if not show_all and usd_value is not None and usd_value < 0.01:
|
||||
dust_count += 1
|
||||
dust_value += usd_value
|
||||
continue
|
||||
|
||||
entry = {"token": label, "mint": mint, "amount": t["amount"]}
|
||||
if usd_price is not None:
|
||||
entry["price_usd"] = usd_price
|
||||
entry["value_usd"] = usd_value
|
||||
enriched.append(entry)
|
||||
|
||||
# Sort: tokens with known USD value first (highest→lowest), then unknowns
|
||||
enriched.sort(key=lambda x: (x.get("value_usd") is not None, x.get("value_usd") or 0), reverse=True)
|
||||
|
||||
# Apply limit unless --all
|
||||
total_tokens = len(enriched)
|
||||
if not show_all and len(enriched) > limit:
|
||||
enriched = enriched[:limit]
|
||||
|
||||
# Compute portfolio total
|
||||
total_usd = sum(t.get("value_usd", 0) for t in enriched)
|
||||
sol_value_usd = round(sol_price * sol_balance, 2) if sol_price else None
|
||||
if sol_value_usd:
|
||||
total_usd += sol_value_usd
|
||||
total_usd += dust_value
|
||||
|
||||
output = {
|
||||
"address": address,
|
||||
"sol_balance": round(sol_balance, 9),
|
||||
}
|
||||
if sol_price:
|
||||
output["sol_price_usd"] = sol_price
|
||||
output["sol_value_usd"] = sol_value_usd
|
||||
output["tokens_shown"] = len(enriched)
|
||||
if total_tokens > len(enriched):
|
||||
output["tokens_hidden"] = total_tokens - len(enriched)
|
||||
output["spl_tokens"] = enriched
|
||||
if dust_count > 0:
|
||||
output["dust_filtered"] = {"count": dust_count, "total_value_usd": round(dust_value, 4)}
|
||||
output["nft_count"] = len(nfts)
|
||||
if nfts:
|
||||
output["nfts"] = [_token_label(n["mint"]) + f" ({_short_mint(n['mint'])})" for n in nfts[:10]]
|
||||
if len(nfts) > 10:
|
||||
output["nfts"].append(f"... and {len(nfts) - 10} more")
|
||||
if total_usd > 0:
|
||||
output["portfolio_total_usd"] = round(total_usd, 2)
|
||||
|
||||
print_json(output)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Transaction Details
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_tx(args):
|
||||
"""Full transaction details by signature."""
|
||||
result = rpc("getTransaction", [
|
||||
args.signature,
|
||||
{"encoding": "jsonParsed", "maxSupportedTransactionVersion": 0},
|
||||
])
|
||||
|
||||
if result is None:
|
||||
sys.exit("Transaction not found (may be too old for public RPC history).")
|
||||
|
||||
meta = result.get("meta", {}) or {}
|
||||
msg = result.get("transaction", {}).get("message", {})
|
||||
account_keys = msg.get("accountKeys", [])
|
||||
|
||||
pre = meta.get("preBalances", [])
|
||||
post = meta.get("postBalances", [])
|
||||
|
||||
balance_changes = []
|
||||
for i, key in enumerate(account_keys):
|
||||
acct_key = key["pubkey"] if isinstance(key, dict) else key
|
||||
if i < len(pre) and i < len(post):
|
||||
change = lamports_to_sol(post[i] - pre[i])
|
||||
if change != 0:
|
||||
balance_changes.append({"account": acct_key, "change_SOL": round(change, 9)})
|
||||
|
||||
programs = []
|
||||
for ix in msg.get("instructions", []):
|
||||
prog = ix.get("programId")
|
||||
if prog is None and "programIdIndex" in ix:
|
||||
k = account_keys[ix["programIdIndex"]]
|
||||
prog = k["pubkey"] if isinstance(k, dict) else k
|
||||
if prog:
|
||||
programs.append(prog)
|
||||
|
||||
# Add USD value for SOL changes
|
||||
sol_price = fetch_sol_price()
|
||||
if sol_price and balance_changes:
|
||||
for bc in balance_changes:
|
||||
bc["change_USD"] = round(bc["change_SOL"] * sol_price, 2)
|
||||
|
||||
print_json({
|
||||
"signature": args.signature,
|
||||
"slot": result.get("slot"),
|
||||
"block_time": result.get("blockTime"),
|
||||
"fee_SOL": lamports_to_sol(meta.get("fee", 0)),
|
||||
"status": "success" if meta.get("err") is None else "failed",
|
||||
"balance_changes": balance_changes,
|
||||
"programs_invoked": list(dict.fromkeys(programs)),
|
||||
})
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. Token Info (enhanced with name + price)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_token(args):
|
||||
"""SPL token metadata, supply, decimals, price, top holders."""
|
||||
mint = args.mint
|
||||
|
||||
mint_info = rpc("getAccountInfo", [mint, {"encoding": "jsonParsed"}])
|
||||
if mint_info is None or mint_info.get("value") is None:
|
||||
sys.exit("Mint account not found.")
|
||||
|
||||
parsed = mint_info["value"]["data"]["parsed"]["info"]
|
||||
decimals = parsed.get("decimals", 0)
|
||||
supply_raw = int(parsed.get("supply", 0))
|
||||
supply_human = supply_raw / (10 ** decimals) if decimals else supply_raw
|
||||
|
||||
largest = rpc("getTokenLargestAccounts", [mint])
|
||||
holders = []
|
||||
for acct in (largest.get("value") or [])[:5]:
|
||||
amount = float(acct.get("uiAmountString") or 0)
|
||||
pct = round((amount / supply_human * 100), 4) if supply_human > 0 else 0
|
||||
holders.append({
|
||||
"account": acct["address"],
|
||||
"amount": amount,
|
||||
"percent": pct,
|
||||
})
|
||||
|
||||
# Resolve name + price
|
||||
token_meta = resolve_token_name(mint)
|
||||
price_data = fetch_prices([mint])
|
||||
|
||||
out = {"mint": mint}
|
||||
if token_meta:
|
||||
out["name"] = token_meta["name"]
|
||||
out["symbol"] = token_meta["symbol"]
|
||||
out["decimals"] = decimals
|
||||
out["supply"] = round(supply_human, min(decimals, 6))
|
||||
out["mint_authority"] = parsed.get("mintAuthority")
|
||||
out["freeze_authority"] = parsed.get("freezeAuthority")
|
||||
if mint in price_data:
|
||||
out["price_usd"] = price_data[mint]
|
||||
out["market_cap_usd"] = round(price_data[mint] * supply_human, 0)
|
||||
out["top_5_holders"] = holders
|
||||
|
||||
print_json(out)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. Recent Activity
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_activity(args):
|
||||
"""Recent transaction signatures for an address."""
|
||||
limit = min(args.limit, 25)
|
||||
result = rpc("getSignaturesForAddress", [args.address, {"limit": limit}])
|
||||
|
||||
txs = [
|
||||
{
|
||||
"signature": item["signature"],
|
||||
"slot": item.get("slot"),
|
||||
"block_time": item.get("blockTime"),
|
||||
"err": item.get("err"),
|
||||
}
|
||||
for item in (result or [])
|
||||
]
|
||||
|
||||
print_json({"address": args.address, "transactions": txs})
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 6. NFT Portfolio
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_nft(args):
|
||||
"""NFTs owned by a wallet (amount=1 && decimals=0 heuristic)."""
|
||||
result = rpc("getTokenAccountsByOwner", [
|
||||
args.address,
|
||||
{"programId": "TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA"},
|
||||
{"encoding": "jsonParsed"},
|
||||
])
|
||||
|
||||
nfts = [
|
||||
acct["account"]["data"]["parsed"]["info"]["mint"]
|
||||
for acct in (result.get("value") or [])
|
||||
if acct["account"]["data"]["parsed"]["info"]["tokenAmount"]["decimals"] == 0
|
||||
and int(acct["account"]["data"]["parsed"]["info"]["tokenAmount"]["amount"]) == 1
|
||||
]
|
||||
|
||||
print_json({
|
||||
"address": args.address,
|
||||
"nft_count": len(nfts),
|
||||
"nfts": nfts,
|
||||
"note": "Heuristic only. Compressed NFTs (cNFTs) are not detected.",
|
||||
})
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 7. Whale Detector (enhanced with USD values)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_whales(args):
|
||||
"""Scan the latest block for large SOL transfers."""
|
||||
min_lamports = int(args.min_sol * LAMPORTS_PER_SOL)
|
||||
|
||||
slot = rpc("getSlot")
|
||||
block = rpc("getBlock", [
|
||||
slot,
|
||||
{
|
||||
"encoding": "jsonParsed",
|
||||
"transactionDetails": "full",
|
||||
"maxSupportedTransactionVersion": 0,
|
||||
"rewards": False,
|
||||
},
|
||||
])
|
||||
|
||||
if block is None:
|
||||
sys.exit("Could not retrieve latest block.")
|
||||
|
||||
sol_price = fetch_sol_price()
|
||||
|
||||
whales = []
|
||||
for tx in (block.get("transactions") or []):
|
||||
meta = tx.get("meta", {}) or {}
|
||||
if meta.get("err") is not None:
|
||||
continue
|
||||
|
||||
msg = tx["transaction"].get("message", {})
|
||||
account_keys = msg.get("accountKeys", [])
|
||||
pre = meta.get("preBalances", [])
|
||||
post = meta.get("postBalances", [])
|
||||
|
||||
for i in range(len(pre)):
|
||||
change = post[i] - pre[i]
|
||||
if change >= min_lamports:
|
||||
k = account_keys[i]
|
||||
receiver = k["pubkey"] if isinstance(k, dict) else k
|
||||
sender = None
|
||||
for j in range(len(pre)):
|
||||
if pre[j] - post[j] >= min_lamports:
|
||||
sk = account_keys[j]
|
||||
sender = sk["pubkey"] if isinstance(sk, dict) else sk
|
||||
break
|
||||
entry = {
|
||||
"sender": sender,
|
||||
"receiver": receiver,
|
||||
"amount_SOL": round(lamports_to_sol(change), 4),
|
||||
}
|
||||
if sol_price:
|
||||
entry["amount_USD"] = round(lamports_to_sol(change) * sol_price, 2)
|
||||
whales.append(entry)
|
||||
|
||||
out = {
|
||||
"slot": slot,
|
||||
"min_threshold_SOL": args.min_sol,
|
||||
"large_transfers": whales,
|
||||
"note": "Scans latest block only — point-in-time snapshot.",
|
||||
}
|
||||
if sol_price:
|
||||
out["sol_price_usd"] = sol_price
|
||||
print_json(out)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 8. Price Lookup
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def cmd_price(args):
|
||||
"""Quick price lookup for a token by mint address or known symbol."""
|
||||
query = args.token
|
||||
|
||||
# Check if it's a known symbol
|
||||
mint = _SYMBOL_TO_MINT.get(query.upper(), query)
|
||||
|
||||
# Try to resolve name
|
||||
token_meta = resolve_token_name(mint)
|
||||
|
||||
# Fetch price
|
||||
prices = fetch_prices([mint])
|
||||
|
||||
out = {"query": query, "mint": mint}
|
||||
if token_meta:
|
||||
out["name"] = token_meta["name"]
|
||||
out["symbol"] = token_meta["symbol"]
|
||||
if mint in prices:
|
||||
out["price_usd"] = prices[mint]
|
||||
else:
|
||||
out["price_usd"] = None
|
||||
out["note"] = "Price not available — token may not be listed on CoinGecko."
|
||||
print_json(out)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="solana_client.py",
|
||||
description="Solana blockchain query tool for Hermes Agent",
|
||||
)
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
sub.add_parser("stats", help="Network stats: slot, epoch, TPS, supply, SOL price")
|
||||
|
||||
p_wallet = sub.add_parser("wallet", help="SOL balance + SPL tokens with USD values")
|
||||
p_wallet.add_argument("address")
|
||||
p_wallet.add_argument("--limit", type=int, default=20,
|
||||
help="Max tokens to display (default: 20)")
|
||||
p_wallet.add_argument("--all", action="store_true",
|
||||
help="Show all tokens (no limit, no dust filter)")
|
||||
p_wallet.add_argument("--no-prices", action="store_true",
|
||||
help="Skip price lookups (faster, RPC-only)")
|
||||
|
||||
p_tx = sub.add_parser("tx", help="Transaction details by signature")
|
||||
p_tx.add_argument("signature")
|
||||
|
||||
p_token = sub.add_parser("token", help="SPL token metadata, price, and top holders")
|
||||
p_token.add_argument("mint")
|
||||
|
||||
p_activity = sub.add_parser("activity", help="Recent transactions for an address")
|
||||
p_activity.add_argument("address")
|
||||
p_activity.add_argument("--limit", type=int, default=10,
|
||||
help="Number of transactions (max 25, default 10)")
|
||||
|
||||
p_nft = sub.add_parser("nft", help="NFT portfolio for a wallet")
|
||||
p_nft.add_argument("address")
|
||||
|
||||
p_whales = sub.add_parser("whales", help="Large SOL transfers in the latest block")
|
||||
p_whales.add_argument("--min-sol", type=float, default=1000.0,
|
||||
help="Minimum SOL transfer size (default: 1000)")
|
||||
|
||||
p_price = sub.add_parser("price", help="Quick price lookup by mint or symbol")
|
||||
p_price.add_argument("token", help="Mint address or known symbol (SOL, BONK, JUP, ...)")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
dispatch = {
|
||||
"stats": cmd_stats,
|
||||
"wallet": cmd_wallet,
|
||||
"tx": cmd_tx,
|
||||
"token": cmd_token,
|
||||
"activity": cmd_activity,
|
||||
"nft": cmd_nft,
|
||||
"whales": cmd_whales,
|
||||
"price": cmd_price,
|
||||
}
|
||||
dispatch[args.command](args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1 @@
|
||||
Communication and decision-making frameworks — structured response formats for proposals, trade-off analysis, and stakeholder-ready recommendations.
|
||||
@@ -0,0 +1,104 @@
|
||||
---
|
||||
name: one-three-one-rule
|
||||
description: >
|
||||
Structured decision-making framework for technical proposals and trade-off analysis.
|
||||
When the user faces a choice between multiple approaches (architecture decisions,
|
||||
tool selection, refactoring strategies, migration paths), this skill produces a
|
||||
1-3-1 format: one clear problem statement, three distinct options with pros/cons,
|
||||
and one concrete recommendation with definition of done and implementation plan.
|
||||
Use when the user asks for a "1-3-1", says "give me options", or needs help
|
||||
choosing between competing approaches.
|
||||
platforms: [linux, macos, windows]
|
||||
version: 1.0.0
|
||||
author: Willard Moore
|
||||
license: MIT
|
||||
category: communication
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [communication, decision-making, proposals, trade-offs]
|
||||
---
|
||||
|
||||
# 1-3-1 Communication Rule
|
||||
|
||||
Structured decision-making format for when a task has multiple viable approaches and the user needs a clear recommendation. Produces a concise problem framing, three options with trade-offs, and an actionable plan for the recommended path.
|
||||
|
||||
## When to Use
|
||||
|
||||
- The user explicitly asks for a "1-3-1" response.
|
||||
- The user says "give me options" or "what are my choices" for a technical decision.
|
||||
- A task has multiple viable approaches with meaningful trade-offs (architecture, tooling, migration strategy).
|
||||
- The user needs a proposal they can forward to a team or stakeholder.
|
||||
|
||||
Do NOT use for simple questions with one obvious answer, debugging sessions, or tasks where the user has already decided on an approach.
|
||||
|
||||
## Procedure
|
||||
|
||||
1. **Problem** (one sentence)
|
||||
- State the core decision or desired outcome in a single concise sentence.
|
||||
- Focus on the *what*, not the *how* — no implementation details, tool names, or specific technologies.
|
||||
- Keep it tight. If you need "and", you're describing two problems.
|
||||
|
||||
2. **Options** (exactly three)
|
||||
- Present three distinct, viable approaches labeled A, B, C.
|
||||
- Each option gets a brief description, pros, and cons.
|
||||
- Options should represent genuinely different strategies, not minor variations of the same approach.
|
||||
|
||||
3. **Recommendation** (one option)
|
||||
- State which option you recommend and why, based on the user's context and priorities.
|
||||
- Be direct — this is your professional judgment, not a hedge.
|
||||
|
||||
4. **Definition of Done**
|
||||
- List the specific success criteria for the recommended option.
|
||||
- These are concrete, verifiable outcomes — not vague aspirations.
|
||||
- If the user picks a different option, revise this section to match.
|
||||
|
||||
5. **Implementation Plan**
|
||||
- Concrete steps to execute the recommended option.
|
||||
- Include specific commands, tools, or actions where applicable.
|
||||
- If the user picks a different option, revise this section to match.
|
||||
|
||||
## Verification
|
||||
|
||||
- Response has exactly one Problem sentence.
|
||||
- Response has exactly three Options (A, B, C) with pros and cons for each.
|
||||
- Response has a single Recommendation that picks one option with reasoning.
|
||||
- Definition of Done and Implementation Plan align with the recommended option.
|
||||
- If the user selects a different option, Recommendation, DoD, and Implementation Plan update accordingly.
|
||||
|
||||
## Example
|
||||
|
||||
User: "Give me a 1-3-1 for adding retry logic to our API client."
|
||||
|
||||
Agent:
|
||||
|
||||
**Problem:** API calls to external services fail intermittently under load and need graceful retry handling.
|
||||
|
||||
**Options:**
|
||||
|
||||
- **Option A: Exponential backoff with jitter in a shared utility.**
|
||||
Pros: Simple, well-understood pattern; single implementation shared across all callers.
|
||||
Cons: No circuit-breaking; callers still block during retries.
|
||||
|
||||
- **Option B: Retry library (tenacity) as a decorator on each endpoint call.**
|
||||
Pros: Configurable per-endpoint; built-in support for circuit breakers and rate limits.
|
||||
Cons: Adds a dependency; decorator pattern can obscure control flow.
|
||||
|
||||
- **Option C: Queue-based async retry with dead letter handling.**
|
||||
Pros: Non-blocking; failed calls are tracked and replayable; scales well.
|
||||
Cons: Significant complexity increase; requires a queue backend; overkill for low-volume services.
|
||||
|
||||
**Recommendation:** Option A. The call volume doesn't justify queue infrastructure, and a shared utility with exponential backoff + jitter covers the failure mode cleanly without new dependencies.
|
||||
|
||||
**Definition of Done:**
|
||||
- `api_retry()` utility handles transient HTTP errors (429, 502, 503, 504).
|
||||
- Retries up to 3 times with exponential backoff and jitter.
|
||||
- All external API calls route through the retry utility.
|
||||
- Logs each retry attempt with endpoint and attempt number.
|
||||
- Tests cover retry exhaustion, immediate success, and transient-then-success scenarios.
|
||||
|
||||
**Implementation Plan:**
|
||||
1. Create `utils/api_retry.py` with configurable max retries, base delay, and retryable status codes.
|
||||
2. Add jitter using `random.uniform(0, base_delay)` to prevent thundering herd.
|
||||
3. Wrap existing API calls in `api_client.py` with the retry utility.
|
||||
4. Add unit tests mocking HTTP responses for each retry scenario.
|
||||
5. Verify under load with a simple stress test against a flaky endpoint mock.
|
||||
@@ -0,0 +1,117 @@
|
||||
---
|
||||
name: blender-mcp
|
||||
description: Control Blender directly from Hermes via socket connection to the blender-mcp addon. Create 3D objects, materials, animations, and run arbitrary Blender Python (bpy) code. Use when user wants to create or modify anything in Blender.
|
||||
version: 1.0.0
|
||||
requires: Blender 4.3+ (desktop instance required, headless not supported)
|
||||
author: alireza78a
|
||||
tags: [blender, 3d, animation, modeling, bpy, mcp]
|
||||
platforms: [linux, macos, windows]
|
||||
---
|
||||
|
||||
# Blender MCP
|
||||
|
||||
Control a running Blender instance from Hermes via socket on TCP port 9876.
|
||||
|
||||
## Setup (one-time)
|
||||
|
||||
### 1. Install the Blender addon
|
||||
|
||||
curl -sL https://raw.githubusercontent.com/ahujasid/blender-mcp/main/addon.py -o ~/Desktop/blender_mcp_addon.py
|
||||
|
||||
In Blender:
|
||||
Edit > Preferences > Add-ons > Install > select blender_mcp_addon.py
|
||||
Enable "Interface: Blender MCP"
|
||||
|
||||
### 2. Start the socket server in Blender
|
||||
|
||||
Press N in Blender viewport to open sidebar.
|
||||
Find "BlenderMCP" tab and click "Start Server".
|
||||
|
||||
### 3. Verify connection
|
||||
|
||||
nc -z -w2 localhost 9876 && echo "OPEN" || echo "CLOSED"
|
||||
|
||||
## Protocol
|
||||
|
||||
Plain UTF-8 JSON over TCP -- no length prefix.
|
||||
|
||||
Send: {"type": "<command>", "params": {<kwargs>}}
|
||||
Receive: {"status": "success", "result": <value>}
|
||||
{"status": "error", "message": "<reason>"}
|
||||
|
||||
## Available Commands
|
||||
|
||||
| type | params | description |
|
||||
|-------------------------|-------------------|---------------------------------|
|
||||
| execute_code | code (str) | Run arbitrary bpy Python code |
|
||||
| get_scene_info | (none) | List all objects in scene |
|
||||
| get_object_info | object_name (str) | Details on a specific object |
|
||||
| get_viewport_screenshot | (none) | Screenshot of current viewport |
|
||||
|
||||
## Python Helper
|
||||
|
||||
Use this inside execute_code tool calls:
|
||||
|
||||
import socket, json
|
||||
|
||||
def blender_exec(code: str, host="localhost", port=9876, timeout=15):
|
||||
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
s.connect((host, port))
|
||||
s.settimeout(timeout)
|
||||
payload = json.dumps({"type": "execute_code", "params": {"code": code}})
|
||||
s.sendall(payload.encode("utf-8"))
|
||||
buf = b""
|
||||
while True:
|
||||
try:
|
||||
chunk = s.recv(4096)
|
||||
if not chunk:
|
||||
break
|
||||
buf += chunk
|
||||
try:
|
||||
json.loads(buf.decode("utf-8"))
|
||||
break
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
except socket.timeout:
|
||||
break
|
||||
s.close()
|
||||
return json.loads(buf.decode("utf-8"))
|
||||
|
||||
## Common bpy Patterns
|
||||
|
||||
### Clear scene
|
||||
bpy.ops.object.select_all(action='SELECT')
|
||||
bpy.ops.object.delete()
|
||||
|
||||
### Add mesh objects
|
||||
bpy.ops.mesh.primitive_uv_sphere_add(radius=1, location=(0, 0, 0))
|
||||
bpy.ops.mesh.primitive_cube_add(size=2, location=(3, 0, 0))
|
||||
bpy.ops.mesh.primitive_cylinder_add(radius=0.5, depth=2, location=(-3, 0, 0))
|
||||
|
||||
### Create and assign material
|
||||
mat = bpy.data.materials.new(name="MyMat")
|
||||
mat.use_nodes = True
|
||||
bsdf = mat.node_tree.nodes.get("Principled BSDF")
|
||||
bsdf.inputs["Base Color"].default_value = (R, G, B, 1.0)
|
||||
bsdf.inputs["Roughness"].default_value = 0.3
|
||||
bsdf.inputs["Metallic"].default_value = 0.0
|
||||
obj.data.materials.append(mat)
|
||||
|
||||
### Keyframe animation
|
||||
obj.location = (0, 0, 0)
|
||||
obj.keyframe_insert(data_path="location", frame=1)
|
||||
obj.location = (0, 0, 3)
|
||||
obj.keyframe_insert(data_path="location", frame=60)
|
||||
|
||||
### Render to file
|
||||
bpy.context.scene.render.filepath = "/tmp/render.png"
|
||||
bpy.context.scene.render.engine = 'CYCLES'
|
||||
bpy.ops.render.render(write_still=True)
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Must check socket is open before running (nc -z localhost 9876)
|
||||
- Addon server must be started inside Blender each session (N-panel > BlenderMCP > Connect)
|
||||
- Break complex scenes into multiple smaller execute_code calls to avoid timeouts
|
||||
- Render output path must be absolute (/tmp/...) not relative
|
||||
- shade_smooth() requires object to be selected and in object mode
|
||||
@@ -0,0 +1,362 @@
|
||||
---
|
||||
name: concept-diagrams
|
||||
description: Generate flat, minimal light/dark-aware SVG diagrams as standalone HTML files, using a unified educational visual language with 9 semantic color ramps, sentence-case typography, and automatic dark mode. Best suited for educational and non-software visuals — physics setups, chemistry mechanisms, math curves, physical objects (aircraft, turbines, smartphones, mechanical watches), anatomy, floor plans, cross-sections, narrative journeys (lifecycle of X, process of Y), hub-spoke system integrations (smart city, IoT), and exploded layer views. If a more specialized skill exists for the subject (dedicated software/cloud architecture, hand-drawn sketches, animated explainers, etc.), prefer that — otherwise this skill can also serve as a general-purpose SVG diagram fallback with a clean educational look. Ships with 15 example diagrams.
|
||||
version: 0.1.0
|
||||
author: v1k22 (original PR), ported into hermes-agent
|
||||
license: MIT
|
||||
dependencies: []
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [diagrams, svg, visualization, education, physics, chemistry, engineering]
|
||||
related_skills: [architecture-diagram, excalidraw, generative-widgets]
|
||||
---
|
||||
|
||||
# Concept Diagrams
|
||||
|
||||
Generate production-quality SVG diagrams with a unified flat, minimal design system. Output is a single self-contained HTML file that renders identically in any modern browser, with automatic light/dark mode.
|
||||
|
||||
## Scope
|
||||
|
||||
**Best suited for:**
|
||||
- Physics setups, chemistry mechanisms, math curves, biology
|
||||
- Physical objects (aircraft, turbines, smartphones, mechanical watches, cells)
|
||||
- Anatomy, cross-sections, exploded layer views
|
||||
- Floor plans, architectural conversions
|
||||
- Narrative journeys (lifecycle of X, process of Y)
|
||||
- Hub-spoke system integrations (smart city, IoT networks, electricity grids)
|
||||
- Educational / textbook-style visuals in any domain
|
||||
- Quantitative charts (grouped bars, energy profiles)
|
||||
|
||||
**Look elsewhere first for:**
|
||||
- Dedicated software / cloud infrastructure architecture with a dark tech aesthetic (consider `architecture-diagram` if available)
|
||||
- Hand-drawn whiteboard sketches (consider `excalidraw` if available)
|
||||
- Animated explainers or video output (consider an animation skill)
|
||||
|
||||
If a more specialized skill is available for the subject, prefer that. If none fits, this skill can serve as a general-purpose SVG diagram fallback — the output will carry the clean educational aesthetic described below, which is a reasonable default for almost any subject.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Decide on the diagram type (see Diagram Types below).
|
||||
2. Lay out components using the Design System rules.
|
||||
3. Write the full HTML page using `templates/template.html` as the wrapper — paste your SVG where the template says `<!-- PASTE SVG HERE -->`.
|
||||
4. Save as a standalone `.html` file (for example `~/my-diagram.html` or `./my-diagram.html`).
|
||||
5. User opens it directly in a browser — no server, no dependencies.
|
||||
|
||||
Optional: if the user wants a browsable gallery of multiple diagrams, see "Local Preview Server" at the bottom.
|
||||
|
||||
Load the HTML template:
|
||||
```
|
||||
skill_view(name="concept-diagrams", file_path="templates/template.html")
|
||||
```
|
||||
|
||||
The template embeds the full CSS design system (`c-*` color classes, text classes, light/dark variables, arrow marker styles). The SVG you generate relies on these classes being present on the hosting page.
|
||||
|
||||
---
|
||||
|
||||
## Design System
|
||||
|
||||
### Philosophy
|
||||
|
||||
- **Flat**: no gradients, drop shadows, blur, glow, or neon effects.
|
||||
- **Minimal**: show the essential. No decorative icons inside boxes.
|
||||
- **Consistent**: same colors, spacing, typography, and stroke widths across every diagram.
|
||||
- **Dark-mode ready**: all colors auto-adapt via CSS classes — no per-mode SVG.
|
||||
|
||||
### Color Palette
|
||||
|
||||
9 color ramps, each with 7 stops. Put the class name on a `<g>` or shape element; the template CSS handles both modes.
|
||||
|
||||
| Class | 50 (lightest) | 100 | 200 | 400 | 600 | 800 | 900 (darkest) |
|
||||
|------------|---------------|---------|---------|---------|---------|---------|---------------|
|
||||
| `c-purple` | #EEEDFE | #CECBF6 | #AFA9EC | #7F77DD | #534AB7 | #3C3489 | #26215C |
|
||||
| `c-teal` | #E1F5EE | #9FE1CB | #5DCAA5 | #1D9E75 | #0F6E56 | #085041 | #04342C |
|
||||
| `c-coral` | #FAECE7 | #F5C4B3 | #F0997B | #D85A30 | #993C1D | #712B13 | #4A1B0C |
|
||||
| `c-pink` | #FBEAF0 | #F4C0D1 | #ED93B1 | #D4537E | #993556 | #72243E | #4B1528 |
|
||||
| `c-gray` | #F1EFE8 | #D3D1C7 | #B4B2A9 | #888780 | #5F5E5A | #444441 | #2C2C2A |
|
||||
| `c-blue` | #E6F1FB | #B5D4F4 | #85B7EB | #378ADD | #185FA5 | #0C447C | #042C53 |
|
||||
| `c-green` | #EAF3DE | #C0DD97 | #97C459 | #639922 | #3B6D11 | #27500A | #173404 |
|
||||
| `c-amber` | #FAEEDA | #FAC775 | #EF9F27 | #BA7517 | #854F0B | #633806 | #412402 |
|
||||
| `c-red` | #FCEBEB | #F7C1C1 | #F09595 | #E24B4A | #A32D2D | #791F1F | #501313 |
|
||||
|
||||
#### Color Assignment Rules
|
||||
|
||||
Color encodes **meaning**, not sequence. Never cycle through colors like a rainbow.
|
||||
|
||||
- Group nodes by **category** — all nodes of the same type share one color.
|
||||
- Use `c-gray` for neutral/structural nodes (start, end, generic steps, users).
|
||||
- Use **2-3 colors per diagram**, not 6+.
|
||||
- Prefer `c-purple`, `c-teal`, `c-coral`, `c-pink` for general categories.
|
||||
- Reserve `c-blue`, `c-green`, `c-amber`, `c-red` for semantic meaning (info, success, warning, error).
|
||||
|
||||
Light/dark stop mapping (handled by the template CSS — just use the class):
|
||||
- Light mode: 50 fill + 600 stroke + 800 title / 600 subtitle
|
||||
- Dark mode: 800 fill + 200 stroke + 100 title / 200 subtitle
|
||||
|
||||
### Typography
|
||||
|
||||
Only two font sizes. No exceptions.
|
||||
|
||||
| Class | Size | Weight | Use |
|
||||
|-------|------|--------|-----|
|
||||
| `th` | 14px | 500 | Node titles, region labels |
|
||||
| `ts` | 12px | 400 | Subtitles, descriptions, arrow labels |
|
||||
| `t` | 14px | 400 | General text |
|
||||
|
||||
- **Sentence case always.** Never Title Case, never ALL CAPS.
|
||||
- Every `<text>` MUST carry a class (`t`, `ts`, or `th`). No unclassed text.
|
||||
- `dominant-baseline="central"` on all text inside boxes.
|
||||
- `text-anchor="middle"` for centered text in boxes.
|
||||
|
||||
**Width estimation (approx):**
|
||||
- 14px weight 500: ~8px per character
|
||||
- 12px weight 400: ~6.5px per character
|
||||
- Always verify: `box_width >= (char_count × px_per_char) + 48` (24px padding each side)
|
||||
|
||||
### Spacing & Layout
|
||||
|
||||
- **ViewBox**: `viewBox="0 0 680 H"` where H = content height + 40px buffer.
|
||||
- **Safe area**: x=40 to x=640, y=40 to y=(H-40).
|
||||
- **Between boxes**: 60px minimum gap.
|
||||
- **Inside boxes**: 24px horizontal padding, 12px vertical padding.
|
||||
- **Arrowhead gap**: 10px between arrowhead and box edge.
|
||||
- **Single-line box**: 44px height.
|
||||
- **Two-line box**: 56px height, 18px between title and subtitle baselines.
|
||||
- **Container padding**: 20px minimum inside every container.
|
||||
- **Max nesting**: 2-3 levels deep. Deeper gets unreadable at 680px width.
|
||||
|
||||
### Stroke & Shape
|
||||
|
||||
- **Stroke width**: 0.5px on all node borders. Not 1px, not 2px.
|
||||
- **Rect rounding**: `rx="8"` for nodes, `rx="12"` for inner containers, `rx="16"` to `rx="20"` for outer containers.
|
||||
- **Connector paths**: MUST have `fill="none"`. SVG defaults to `fill: black` otherwise.
|
||||
|
||||
### Arrow Marker
|
||||
|
||||
Include this `<defs>` block at the start of **every** SVG:
|
||||
|
||||
```xml
|
||||
<defs>
|
||||
<marker id="arrow" viewBox="0 0 10 10" refX="8" refY="5"
|
||||
markerWidth="6" markerHeight="6" orient="auto-start-reverse">
|
||||
<path d="M2 1L8 5L2 9" fill="none" stroke="context-stroke"
|
||||
stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
</marker>
|
||||
</defs>
|
||||
```
|
||||
|
||||
Use `marker-end="url(#arrow)"` on lines. The arrowhead inherits the line color via `context-stroke`.
|
||||
|
||||
### CSS Classes (Provided by the Template)
|
||||
|
||||
The template page provides:
|
||||
|
||||
- Text: `.t`, `.ts`, `.th`
|
||||
- Neutral: `.box`, `.arr`, `.leader`, `.node`
|
||||
- Color ramps: `.c-purple`, `.c-teal`, `.c-coral`, `.c-pink`, `.c-gray`, `.c-blue`, `.c-green`, `.c-amber`, `.c-red` (all with automatic light/dark mode)
|
||||
|
||||
You do **not** need to redefine these — just apply them in your SVG. The template file contains the full CSS definitions.
|
||||
|
||||
---
|
||||
|
||||
## SVG Boilerplate
|
||||
|
||||
Every SVG inside the template page starts with this exact structure:
|
||||
|
||||
```xml
|
||||
<svg width="100%" viewBox="0 0 680 {HEIGHT}" xmlns="http://www.w3.org/2000/svg">
|
||||
<defs>
|
||||
<marker id="arrow" viewBox="0 0 10 10" refX="8" refY="5"
|
||||
markerWidth="6" markerHeight="6" orient="auto-start-reverse">
|
||||
<path d="M2 1L8 5L2 9" fill="none" stroke="context-stroke"
|
||||
stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
</marker>
|
||||
</defs>
|
||||
|
||||
<!-- Diagram content here -->
|
||||
|
||||
</svg>
|
||||
```
|
||||
|
||||
Replace `{HEIGHT}` with the actual computed height (last element bottom + 40px).
|
||||
|
||||
### Node Patterns
|
||||
|
||||
**Single-line node (44px):**
|
||||
```xml
|
||||
<g class="node c-blue">
|
||||
<rect x="100" y="20" width="180" height="44" rx="8" stroke-width="0.5"/>
|
||||
<text class="th" x="190" y="42" text-anchor="middle" dominant-baseline="central">Service name</text>
|
||||
</g>
|
||||
```
|
||||
|
||||
**Two-line node (56px):**
|
||||
```xml
|
||||
<g class="node c-teal">
|
||||
<rect x="100" y="20" width="200" height="56" rx="8" stroke-width="0.5"/>
|
||||
<text class="th" x="200" y="38" text-anchor="middle" dominant-baseline="central">Service name</text>
|
||||
<text class="ts" x="200" y="56" text-anchor="middle" dominant-baseline="central">Short description</text>
|
||||
</g>
|
||||
```
|
||||
|
||||
**Connector (no label):**
|
||||
```xml
|
||||
<line x1="200" y1="76" x2="200" y2="120" class="arr" marker-end="url(#arrow)"/>
|
||||
```
|
||||
|
||||
**Container (dashed or solid):**
|
||||
```xml
|
||||
<g class="c-purple">
|
||||
<rect x="40" y="92" width="600" height="300" rx="16" stroke-width="0.5"/>
|
||||
<text class="th" x="66" y="116">Container label</text>
|
||||
<text class="ts" x="66" y="134">Subtitle info</text>
|
||||
</g>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Diagram Types
|
||||
|
||||
Choose the layout that fits the subject:
|
||||
|
||||
1. **Flowchart** — CI/CD pipelines, request lifecycles, approval workflows, data processing. Single-direction flow (top-down or left-right). Max 4-5 nodes per row.
|
||||
2. **Structural / Containment** — Cloud infrastructure nesting, system architecture with layers. Large outer containers with inner regions. Dashed rects for logical groupings.
|
||||
3. **API / Endpoint Map** — REST routes, GraphQL schemas. Tree from root, branching to resource groups, each containing endpoint nodes.
|
||||
4. **Microservice Topology** — Service mesh, event-driven systems. Services as nodes, arrows for communication patterns, message queues between.
|
||||
5. **Data Flow** — ETL pipelines, streaming architectures. Left-to-right flow from sources through processing to sinks.
|
||||
6. **Physical / Structural** — Vehicles, buildings, hardware, anatomy. Use shapes that match the physical form — `<path>` for curved bodies, `<polygon>` for tapered shapes, `<ellipse>`/`<circle>` for cylindrical parts, nested `<rect>` for compartments. See `references/physical-shape-cookbook.md`.
|
||||
7. **Infrastructure / Systems Integration** — Smart cities, IoT networks, multi-domain systems. Hub-spoke layout with central platform connecting subsystems. Semantic line styles (`.data-line`, `.power-line`, `.water-pipe`, `.road`). See `references/infrastructure-patterns.md`.
|
||||
8. **UI / Dashboard Mockups** — Admin panels, monitoring dashboards. Screen frame with nested chart/gauge/indicator elements. See `references/dashboard-patterns.md`.
|
||||
|
||||
For physical, infrastructure, and dashboard diagrams, load the matching reference file before generating — each one provides ready-made CSS classes and shape primitives.
|
||||
|
||||
---
|
||||
|
||||
## Validation Checklist
|
||||
|
||||
Before finalizing any SVG, verify ALL of the following:
|
||||
|
||||
1. Every `<text>` has class `t`, `ts`, or `th`.
|
||||
2. Every `<text>` inside a box has `dominant-baseline="central"`.
|
||||
3. Every connector `<path>` or `<line>` used as arrow has `fill="none"`.
|
||||
4. No arrow line crosses through an unrelated box.
|
||||
5. `box_width >= (longest_label_chars × 8) + 48` for 14px text.
|
||||
6. `box_width >= (longest_label_chars × 6.5) + 48` for 12px text.
|
||||
7. ViewBox height = bottom-most element + 40px.
|
||||
8. All content stays within x=40 to x=640.
|
||||
9. Color classes (`c-*`) are on `<g>` or shape elements, never on `<path>` connectors.
|
||||
10. Arrow `<defs>` block is present.
|
||||
11. No gradients, shadows, blur, or glow effects.
|
||||
12. Stroke width is 0.5px on all node borders.
|
||||
|
||||
---
|
||||
|
||||
## Output & Preview
|
||||
|
||||
### Default: standalone HTML file
|
||||
|
||||
Write a single `.html` file the user can open directly. No server, no dependencies, works offline. Pattern:
|
||||
|
||||
```python
|
||||
# 1. Load the template
|
||||
template = skill_view("concept-diagrams", "templates/template.html")
|
||||
|
||||
# 2. Fill in title, subtitle, and paste your SVG
|
||||
html = template.replace(
|
||||
"<!-- DIAGRAM TITLE HERE -->", "SN2 reaction mechanism"
|
||||
).replace(
|
||||
"<!-- OPTIONAL SUBTITLE HERE -->", "Bimolecular nucleophilic substitution"
|
||||
).replace(
|
||||
"<!-- PASTE SVG HERE -->", svg_content
|
||||
)
|
||||
|
||||
# 3. Write to a user-chosen path (or ./ by default)
|
||||
write_file("./sn2-mechanism.html", html)
|
||||
```
|
||||
|
||||
Tell the user how to open it:
|
||||
|
||||
```
|
||||
# macOS
|
||||
open ./sn2-mechanism.html
|
||||
# Linux
|
||||
xdg-open ./sn2-mechanism.html
|
||||
```
|
||||
|
||||
### Optional: local preview server (multi-diagram gallery)
|
||||
|
||||
Only use this when the user explicitly wants a browsable gallery of multiple diagrams.
|
||||
|
||||
**Rules:**
|
||||
- Bind to `127.0.0.1` only. Never `0.0.0.0`. Exposing diagrams on all network interfaces is a security hazard on shared networks.
|
||||
- Pick a free port (do NOT hard-code one) and tell the user the chosen URL.
|
||||
- The server is optional and opt-in — prefer the standalone HTML file first.
|
||||
|
||||
Recommended pattern (lets the OS pick a free ephemeral port):
|
||||
|
||||
```bash
|
||||
# Put each diagram in its own folder under .diagrams/
|
||||
mkdir -p .diagrams/sn2-mechanism
|
||||
# ...write .diagrams/sn2-mechanism/index.html...
|
||||
|
||||
# Serve on loopback only, free port
|
||||
cd .diagrams && python3 -c "
|
||||
import http.server, socketserver
|
||||
with socketserver.TCPServer(('127.0.0.1', 0), http.server.SimpleHTTPRequestHandler) as s:
|
||||
print(f'Serving at http://127.0.0.1:{s.server_address[1]}/')
|
||||
s.serve_forever()
|
||||
" &
|
||||
```
|
||||
|
||||
If the user insists on a fixed port, use `127.0.0.1:<port>` — still never `0.0.0.0`. Document how to stop the server (`kill %1` or `pkill -f "http.server"`).
|
||||
|
||||
---
|
||||
|
||||
## Examples Reference
|
||||
|
||||
The `examples/` directory ships 15 complete, tested diagrams. Browse them for working patterns before writing a new diagram of a similar type:
|
||||
|
||||
| File | Type | Demonstrates |
|
||||
|------|------|--------------|
|
||||
| `hospital-emergency-department-flow.md` | Flowchart | Priority routing with semantic colors |
|
||||
| `feature-film-production-pipeline.md` | Flowchart | Phased workflow, horizontal sub-flows |
|
||||
| `automated-password-reset-flow.md` | Flowchart | Auth flow with error branches |
|
||||
| `autonomous-llm-research-agent-flow.md` | Flowchart | Loop-back arrows, decision branches |
|
||||
| `place-order-uml-sequence.md` | Sequence | UML sequence diagram style |
|
||||
| `commercial-aircraft-structure.md` | Physical | Paths, polygons, ellipses for realistic shapes |
|
||||
| `wind-turbine-structure.md` | Physical cross-section | Underground/above-ground separation, color coding |
|
||||
| `smartphone-layer-anatomy.md` | Exploded view | Alternating left/right labels, layered components |
|
||||
| `apartment-floor-plan-conversion.md` | Floor plan | Walls, doors, proposed changes in dotted red |
|
||||
| `banana-journey-tree-to-smoothie.md` | Narrative journey | Winding path, progressive state changes |
|
||||
| `cpu-ooo-microarchitecture.md` | Hardware pipeline | Fan-out, memory hierarchy sidebar |
|
||||
| `sn2-reaction-mechanism.md` | Chemistry | Molecules, curved arrows, energy profile |
|
||||
| `smart-city-infrastructure.md` | Hub-spoke | Semantic line styles per system |
|
||||
| `electricity-grid-flow.md` | Multi-stage flow | Voltage hierarchy, flow markers |
|
||||
| `ml-benchmark-grouped-bar-chart.md` | Chart | Grouped bars, dual axis |
|
||||
|
||||
Load any example with:
|
||||
```
|
||||
skill_view(name="concept-diagrams", file_path="examples/<filename>")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference: What to Use When
|
||||
|
||||
| User says | Diagram type | Suggested colors |
|
||||
|-----------|--------------|------------------|
|
||||
| "show the pipeline" | Flowchart | gray start/end, purple steps, red errors, teal deploy |
|
||||
| "draw the data flow" | Data pipeline (left-right) | gray sources, purple processing, teal sinks |
|
||||
| "visualize the system" | Structural (containment) | purple container, teal services, coral data |
|
||||
| "map the endpoints" | API tree | purple root, one ramp per resource group |
|
||||
| "show the services" | Microservice topology | gray ingress, teal services, purple bus, coral workers |
|
||||
| "draw the aircraft/vehicle" | Physical | paths, polygons, ellipses for realistic shapes |
|
||||
| "smart city / IoT" | Hub-spoke integration | semantic line styles per subsystem |
|
||||
| "show the dashboard" | UI mockup | dark screen, chart colors: teal, purple, coral for alerts |
|
||||
| "power grid / electricity" | Multi-stage flow | voltage hierarchy (HV/MV/LV line weights) |
|
||||
| "wind turbine / turbine" | Physical cross-section | foundation + tower cutaway + nacelle color-coded |
|
||||
| "journey of X / lifecycle" | Narrative journey | winding path, progressive state changes |
|
||||
| "layers of X / exploded" | Exploded layer view | vertical stack, alternating labels |
|
||||
| "CPU / pipeline" | Hardware pipeline | vertical stages, fan-out to execution ports |
|
||||
| "floor plan / apartment" | Floor plan | walls, doors, proposed changes in dotted red |
|
||||
| "reaction mechanism" | Chemistry | atoms, bonds, curved arrows, transition state, energy profile |
|
||||
@@ -0,0 +1,43 @@
|
||||
# Dashboard Patterns
|
||||
|
||||
Building blocks for UI/dashboard mockups inside a concept diagram — admin panels, monitoring dashboards, control interfaces, status displays.
|
||||
|
||||
## Pattern
|
||||
|
||||
A "screen" is a rounded dark rect inside a lighter "frame" rect, with chart/gauge/indicator elements nested on top.
|
||||
|
||||
```xml
|
||||
<!-- Monitor frame -->
|
||||
<rect class="dashboard" x="0" y="0" width="200" height="120" rx="8"/>
|
||||
<!-- Screen -->
|
||||
<rect class="screen" x="10" y="10" width="180" height="85" rx="4"/>
|
||||
<!-- Mini bar chart -->
|
||||
<rect class="screen-content" x="18" y="18" width="50" height="35" rx="2"/>
|
||||
<rect class="screen-chart" x="22" y="38" width="8" height="12"/>
|
||||
<rect class="screen-chart" x="33" y="32" width="8" height="18"/>
|
||||
<!-- Gauge -->
|
||||
<circle class="screen-bar" cx="100" cy="35" r="12"/>
|
||||
<text x="100" y="39" text-anchor="middle" fill="#E8E6DE" style="font-size:8px">78%</text>
|
||||
<!-- Status indicators -->
|
||||
<circle cx="35" cy="74" r="6" fill="#97C459"/> <!-- green = ok -->
|
||||
<circle cx="75" cy="74" r="6" fill="#EF9F27"/> <!-- amber = warning -->
|
||||
<circle cx="115" cy="74" r="6" fill="#E24B4A"/> <!-- red = alert -->
|
||||
```
|
||||
|
||||
## CSS
|
||||
|
||||
```css
|
||||
.dashboard { fill: #F1EFE8; stroke: #5F5E5A; stroke-width: 1.5; }
|
||||
.screen { fill: #1a1a18; }
|
||||
.screen-content { fill: #2C2C2A; }
|
||||
.screen-chart { fill: #5DCAA5; }
|
||||
.screen-bar { fill: #7F77DD; }
|
||||
.screen-alert { fill: #E24B4A; }
|
||||
```
|
||||
|
||||
## Tips
|
||||
|
||||
- Dashboard screens stay dark in both light and dark mode — they represent actual monitor glass.
|
||||
- Keep on-screen text small (`font-size:8px` or `10px`) and high-contrast (near-white fill on dark).
|
||||
- Use the status triad green/amber/red consistently — OK / warning / alert.
|
||||
- A single dashboard usually sits on top of an infrastructure hub diagram as a unified view (see `examples/smart-city-infrastructure.md`).
|
||||
@@ -0,0 +1,144 @@
|
||||
# Infrastructure Patterns
|
||||
|
||||
Reusable shapes and line styles for infrastructure / systems-integration diagrams (smart cities, IoT networks, industrial systems, multi-domain architectures).
|
||||
|
||||
## Layout pattern: hub-spoke
|
||||
|
||||
- **Central hub**: Hexagon or circle representing the integration platform
|
||||
- **Radiating connections**: Data lines from hub to each subsystem with connection dots
|
||||
- **Subsystem sections**: Each system (power, water, transport) in its own region
|
||||
- **Dashboard on top**: Optional UI mockup showing a unified view (see `dashboard-patterns.md`)
|
||||
|
||||
```xml
|
||||
<!-- Central hub (hexagon) -->
|
||||
<polygon class="iot-hex" points="0,-45 39,-22 39,22 0,45 -39,22 -39,-22"/>
|
||||
|
||||
<!-- Data lines with connection dots -->
|
||||
<path class="data-line" d="M 321 248 L 200 248 L 120 380" stroke-dasharray="4 3"/>
|
||||
<circle cx="321" cy="248" r="4" fill="#7F77DD"/>
|
||||
```
|
||||
|
||||
## Semantic line styles
|
||||
|
||||
Use a dedicated CSS class per subsystem so every diagram reads the same way:
|
||||
|
||||
```css
|
||||
.data-line { stroke: #7F77DD; stroke-width: 2; fill: none; stroke-dasharray: 4 3; }
|
||||
.power-line { stroke: #EF9F27; stroke-width: 2; fill: none; }
|
||||
.water-pipe { stroke: #378ADD; stroke-width: 4; stroke-linecap: round; fill: none; }
|
||||
.road { stroke: #888780; stroke-width: 8; stroke-linecap: round; fill: none; }
|
||||
```
|
||||
|
||||
## Power systems
|
||||
|
||||
**Solar panel (angled):**
|
||||
```xml
|
||||
<polygon class="solar-panel" points="0,25 35,8 38,12 3,29"/>
|
||||
<line class="solar-frame" x1="12" y1="22" x2="24" y2="13"/>
|
||||
```
|
||||
|
||||
**Wind turbine:**
|
||||
```xml
|
||||
<polygon class="wind-tower" points="20,70 30,70 28,25 22,25"/>
|
||||
<circle class="wind-hub" cx="25" cy="18" r="5"/>
|
||||
<ellipse class="wind-blade" cx="25" cy="5" rx="3" ry="13"/>
|
||||
<ellipse class="wind-blade" cx="14" cy="26" rx="3" ry="13" transform="rotate(-120, 25, 18)"/>
|
||||
<ellipse class="wind-blade" cx="36" cy="26" rx="3" ry="13" transform="rotate(120, 25, 18)"/>
|
||||
```
|
||||
|
||||
**Battery with charge level:**
|
||||
```xml
|
||||
<rect class="battery" x="0" y="0" width="45" height="65" rx="5"/>
|
||||
<rect x="10" y="-6" width="10" height="8" rx="2" fill="#27500A"/> <!-- terminal -->
|
||||
<rect class="battery-level" x="5" y="12" width="35" height="48" rx="3"/> <!-- fill level -->
|
||||
```
|
||||
|
||||
**Power pylon:**
|
||||
```xml
|
||||
<polygon class="pylon" points="30,0 35,0 40,60 25,60"/>
|
||||
<line x1="15" y1="10" x2="45" y2="10" stroke="#5F5E5A" stroke-width="3"/>
|
||||
<circle cx="18" cy="10" r="3" fill="#FAEEDA" stroke="#854F0B"/> <!-- insulator -->
|
||||
```
|
||||
|
||||
## Water systems
|
||||
|
||||
**Reservoir/dam:**
|
||||
```xml
|
||||
<polygon class="reservoir-wall" points="0,60 10,0 70,0 80,60"/>
|
||||
<polygon class="water" points="12,10 68,10 68,55 75,55 75,58 5,58 5,55 12,55"/>
|
||||
<!-- Wave effect -->
|
||||
<path d="M 15 25 Q 25 22 35 25 Q 45 28 55 25" fill="none" stroke="#378ADD" opacity="0.5"/>
|
||||
```
|
||||
|
||||
**Treatment tank:**
|
||||
```xml
|
||||
<ellipse class="treatment-tank" cx="35" cy="45" rx="30" ry="18"/>
|
||||
<rect class="treatment-tank" x="5" y="20" width="60" height="25"/>
|
||||
<!-- Bubbles -->
|
||||
<circle cx="20" cy="32" r="2" fill="#378ADD" opacity="0.6"/>
|
||||
```
|
||||
|
||||
**Pipe with joint and valve:**
|
||||
```xml
|
||||
<path class="pipe" d="M 80 85 L 110 85"/>
|
||||
<circle class="pipe-joint" cx="110" cy="85" r="8"/>
|
||||
<circle class="valve" cx="95" cy="85" r="6"/>
|
||||
```
|
||||
|
||||
## Transport systems
|
||||
|
||||
**Road with lane markings:**
|
||||
```xml
|
||||
<line class="road" x1="0" y1="50" x2="170" y2="50"/>
|
||||
<line class="road-mark" x1="10" y1="50" x2="160" y2="50"/>
|
||||
```
|
||||
|
||||
**Traffic light:**
|
||||
```xml
|
||||
<rect class="traffic-light" x="0" y="0" width="14" height="32" rx="3"/>
|
||||
<circle class="light-red" cx="7" cy="8" r="4"/>
|
||||
<circle class="light-off" cx="7" cy="16" r="4"/>
|
||||
<circle class="light-green" cx="7" cy="24" r="4"/>
|
||||
```
|
||||
|
||||
**Bus:**
|
||||
```xml
|
||||
<rect class="bus" x="0" y="0" width="55" height="28" rx="6"/>
|
||||
<rect class="bus-window" x="5" y="5" width="12" height="12" rx="2"/>
|
||||
<circle cx="14" cy="30" r="6" fill="#2C2C2A"/> <!-- wheel -->
|
||||
<circle cx="14" cy="30" r="3" fill="#5F5E5A"/> <!-- hubcap -->
|
||||
```
|
||||
|
||||
## Full CSS block (add to the host page or inline <style>)
|
||||
|
||||
```css
|
||||
/* Power */
|
||||
.solar-panel { fill: #3C3489; stroke: #534AB7; stroke-width: 0.5; }
|
||||
.wind-tower { fill: #B4B2A9; stroke: #5F5E5A; stroke-width: 1; }
|
||||
.wind-blade { fill: #F1EFE8; stroke: #888780; stroke-width: 0.5; }
|
||||
.battery { fill: #27500A; stroke: #3B6D11; stroke-width: 1.5; }
|
||||
.battery-level { fill: #97C459; }
|
||||
.power-line { stroke: #EF9F27; stroke-width: 2; fill: none; }
|
||||
|
||||
/* Water */
|
||||
.reservoir-wall { fill: #B4B2A9; stroke: #5F5E5A; stroke-width: 1; }
|
||||
.water { fill: #85B7EB; stroke: #378ADD; stroke-width: 0.5; }
|
||||
.pipe { fill: none; stroke: #378ADD; stroke-width: 4; stroke-linecap: round; }
|
||||
.pipe-joint { fill: #185FA5; stroke: #0C447C; stroke-width: 1; }
|
||||
.valve { fill: #0C447C; stroke: #185FA5; stroke-width: 1; }
|
||||
|
||||
/* Transport */
|
||||
.road { stroke: #888780; stroke-width: 8; fill: none; stroke-linecap: round; }
|
||||
.road-mark { stroke: #F1EFE8; stroke-width: 1; stroke-dasharray: 6 4; fill: none; }
|
||||
.traffic-light { fill: #444441; stroke: #2C2C2A; stroke-width: 0.5; }
|
||||
.light-red { fill: #E24B4A; }
|
||||
.light-green { fill: #97C459; }
|
||||
.light-off { fill: #2C2C2A; }
|
||||
.bus { fill: #E1F5EE; stroke: #0F6E56; stroke-width: 1.5; }
|
||||
```
|
||||
|
||||
## Reference examples
|
||||
|
||||
- `examples/smart-city-infrastructure.md` — hub-spoke with multiple subsystems
|
||||
- `examples/electricity-grid-flow.md` — voltage hierarchy, flow markers
|
||||
- `examples/wind-turbine-structure.md` — cross-section with legend
|
||||
@@ -0,0 +1,42 @@
|
||||
# Physical Shape Cookbook
|
||||
|
||||
Guidance for drawing physical objects (vehicles, buildings, hardware, mechanical systems, anatomy) — when rectangles aren't enough.
|
||||
|
||||
## Shape selection
|
||||
|
||||
| Physical form | SVG element | Example use |
|
||||
|---------------|-------------|-------------|
|
||||
| Curved bodies | `<path>` with Q/C curves | Fuselage, tanks, pipes |
|
||||
| Tapered/angular shapes | `<polygon>` | Wings, fins, wedges |
|
||||
| Cylindrical/round | `<ellipse>`, `<circle>` | Engines, wheels, buttons |
|
||||
| Linear structures | `<line>` | Struts, beams, connections |
|
||||
| Internal sections | `<rect>` inside parent | Compartments, rooms |
|
||||
| Dashed boundaries | `stroke-dasharray` | Hidden parts, fuel tanks |
|
||||
|
||||
## Layering approach
|
||||
|
||||
1. Draw outer structure first (fuselage, frame, hull)
|
||||
2. Add internal sections on top (cabins, compartments)
|
||||
3. Add detail elements (engines, wheels, controls)
|
||||
4. Add leader lines with labels
|
||||
|
||||
## Semantic CSS classes (instead of c-* ramps)
|
||||
|
||||
For physical diagrams, define component-specific classes directly rather than applying `c-*` color classes. This makes each part self-documenting and lets you keep a restrained palette:
|
||||
|
||||
```css
|
||||
.fuselage { fill: #F1EFE8; stroke: #5F5E5A; stroke-width: 1; }
|
||||
.wing { fill: #E6F1FB; stroke: #185FA5; stroke-width: 1; }
|
||||
.engine { fill: #FAECE7; stroke: #993C1D; stroke-width: 1; }
|
||||
```
|
||||
|
||||
Add these to a local `<style>` inside the SVG (or extend the host page's `<style>` block). The light-mode/dark-mode pattern still works — use the CSS variables from the template (`var(--bg-secondary)`, `var(--border)`, `var(--text-primary)`) if you want dark-mode awareness.
|
||||
|
||||
## Reference examples
|
||||
|
||||
Look at these example files for working physical-diagram patterns:
|
||||
|
||||
- `examples/commercial-aircraft-structure.md` — fuselage curves + tapered wings + ellipse engines
|
||||
- `examples/wind-turbine-structure.md` — underground foundation, tubular tower, nacelle cutaway
|
||||
- `examples/smartphone-layer-anatomy.md` — exploded-view stack with alternating labels
|
||||
- `examples/apartment-floor-plan-conversion.md` — walls, doors, windows, proposed changes
|
||||
@@ -0,0 +1,174 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Concept Diagram</title>
|
||||
<style>
|
||||
:root {
|
||||
--text-primary: #1a1a18;
|
||||
--text-secondary: #5f5e5a;
|
||||
--text-tertiary: #88877f;
|
||||
--bg-primary: #ffffff;
|
||||
--bg-secondary: #f6f5f0;
|
||||
--bg-tertiary: #eeedeb;
|
||||
--border: rgba(0,0,0,0.15);
|
||||
--border-hover: rgba(0,0,0,0.3);
|
||||
}
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--text-primary: #e8e6de;
|
||||
--text-secondary: #b4b2a9;
|
||||
--text-tertiary: #888780;
|
||||
--bg-primary: #1a1a18;
|
||||
--bg-secondary: #2c2c2a;
|
||||
--bg-tertiary: #3d3d3a;
|
||||
--border: rgba(255,255,255,0.15);
|
||||
--border-hover: rgba(255,255,255,0.3);
|
||||
}
|
||||
}
|
||||
* { margin: 0; padding: 0; box-sizing: border-box; }
|
||||
body {
|
||||
font-family: system-ui, -apple-system, sans-serif;
|
||||
background: var(--bg-tertiary);
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: flex-start;
|
||||
min-height: 100vh;
|
||||
padding: 40px 20px;
|
||||
}
|
||||
.card {
|
||||
background: var(--bg-primary);
|
||||
border-radius: 16px;
|
||||
padding: 32px;
|
||||
max-width: 780px;
|
||||
width: 100%;
|
||||
box-shadow: 0 1px 3px rgba(0,0,0,0.08);
|
||||
}
|
||||
h1 {
|
||||
font-size: 18px;
|
||||
font-weight: 500;
|
||||
color: var(--text-primary);
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
.subtitle {
|
||||
font-size: 13px;
|
||||
color: var(--text-tertiary);
|
||||
margin-bottom: 24px;
|
||||
}
|
||||
svg { width: 100%; height: auto; }
|
||||
|
||||
/* === SVG Design System Classes === */
|
||||
|
||||
/* Text classes */
|
||||
.t { font-family: system-ui, -apple-system, sans-serif; font-size: 14px; fill: var(--text-primary); }
|
||||
.ts { font-family: system-ui, -apple-system, sans-serif; font-size: 12px; fill: var(--text-secondary); }
|
||||
.th { font-family: system-ui, -apple-system, sans-serif; font-size: 14px; fill: var(--text-primary); font-weight: 500; }
|
||||
|
||||
/* Neutral box */
|
||||
.box { fill: var(--bg-secondary); stroke: var(--border); stroke-width: 0.5px; }
|
||||
|
||||
/* Arrow */
|
||||
.arr { stroke: var(--text-secondary); stroke-width: 1.5px; fill: none; }
|
||||
|
||||
/* Leader line */
|
||||
.leader { stroke: var(--text-tertiary); stroke-width: 0.5px; stroke-dasharray: 4 3; fill: none; }
|
||||
|
||||
/* Clickable node */
|
||||
.node { cursor: pointer; transition: opacity 0.15s; }
|
||||
.node:hover { opacity: 0.82; }
|
||||
|
||||
/* === Color Ramp Classes (light mode) === */
|
||||
.c-purple > rect, .c-purple > circle, .c-purple > ellipse { fill: #EEEDFE; stroke: #534AB7; }
|
||||
.c-purple > .th, .c-purple > text.th { fill: #3C3489; }
|
||||
.c-purple > .ts, .c-purple > text.ts { fill: #534AB7; }
|
||||
.c-purple > .t, .c-purple > text.t { fill: #3C3489; }
|
||||
|
||||
.c-teal > rect, .c-teal > circle, .c-teal > ellipse { fill: #E1F5EE; stroke: #0F6E56; }
|
||||
.c-teal > .th, .c-teal > text.th { fill: #085041; }
|
||||
.c-teal > .ts, .c-teal > text.ts { fill: #0F6E56; }
|
||||
.c-teal > .t, .c-teal > text.t { fill: #085041; }
|
||||
|
||||
.c-coral > rect, .c-coral > circle, .c-coral > ellipse { fill: #FAECE7; stroke: #993C1D; }
|
||||
.c-coral > .th, .c-coral > text.th { fill: #712B13; }
|
||||
.c-coral > .ts, .c-coral > text.ts { fill: #993C1D; }
|
||||
.c-coral > .t, .c-coral > text.t { fill: #712B13; }
|
||||
|
||||
.c-pink > rect, .c-pink > circle, .c-pink > ellipse { fill: #FBEAF0; stroke: #993556; }
|
||||
.c-pink > .th, .c-pink > text.th { fill: #72243E; }
|
||||
.c-pink > .ts, .c-pink > text.ts { fill: #993556; }
|
||||
.c-pink > .t, .c-pink > text.t { fill: #72243E; }
|
||||
|
||||
.c-gray > rect, .c-gray > circle, .c-gray > ellipse { fill: #F1EFE8; stroke: #5F5E5A; }
|
||||
.c-gray > .th, .c-gray > text.th { fill: #444441; }
|
||||
.c-gray > .ts, .c-gray > text.ts { fill: #5F5E5A; }
|
||||
.c-gray > .t, .c-gray > text.t { fill: #444441; }
|
||||
|
||||
.c-blue > rect, .c-blue > circle, .c-blue > ellipse { fill: #E6F1FB; stroke: #185FA5; }
|
||||
.c-blue > .th, .c-blue > text.th { fill: #0C447C; }
|
||||
.c-blue > .ts, .c-blue > text.ts { fill: #185FA5; }
|
||||
.c-blue > .t, .c-blue > text.t { fill: #0C447C; }
|
||||
|
||||
.c-green > rect, .c-green > circle, .c-green > ellipse { fill: #EAF3DE; stroke: #3B6D11; }
|
||||
.c-green > .th, .c-green > text.th { fill: #27500A; }
|
||||
.c-green > .ts, .c-green > text.ts { fill: #3B6D11; }
|
||||
.c-green > .t, .c-green > text.t { fill: #27500A; }
|
||||
|
||||
.c-amber > rect, .c-amber > circle, .c-amber > ellipse { fill: #FAEEDA; stroke: #854F0B; }
|
||||
.c-amber > .th, .c-amber > text.th { fill: #633806; }
|
||||
.c-amber > .ts, .c-amber > text.ts { fill: #854F0B; }
|
||||
.c-amber > .t, .c-amber > text.t { fill: #633806; }
|
||||
|
||||
.c-red > rect, .c-red > circle, .c-red > ellipse { fill: #FCEBEB; stroke: #A32D2D; }
|
||||
.c-red > .th, .c-red > text.th { fill: #791F1F; }
|
||||
.c-red > .ts, .c-red > text.ts { fill: #A32D2D; }
|
||||
.c-red > .t, .c-red > text.t { fill: #791F1F; }
|
||||
|
||||
/* === Dark mode overrides === */
|
||||
@media (prefers-color-scheme: dark) {
|
||||
.c-purple > rect, .c-purple > circle, .c-purple > ellipse { fill: #3C3489; stroke: #AFA9EC; }
|
||||
.c-purple > .th, .c-purple > text.th { fill: #CECBF6; }
|
||||
.c-purple > .ts, .c-purple > text.ts { fill: #AFA9EC; }
|
||||
|
||||
.c-teal > rect, .c-teal > circle, .c-teal > ellipse { fill: #085041; stroke: #5DCAA5; }
|
||||
.c-teal > .th, .c-teal > text.th { fill: #9FE1CB; }
|
||||
.c-teal > .ts, .c-teal > text.ts { fill: #5DCAA5; }
|
||||
|
||||
.c-coral > rect, .c-coral > circle, .c-coral > ellipse { fill: #712B13; stroke: #F0997B; }
|
||||
.c-coral > .th, .c-coral > text.th { fill: #F5C4B3; }
|
||||
.c-coral > .ts, .c-coral > text.ts { fill: #F0997B; }
|
||||
|
||||
.c-pink > rect, .c-pink > circle, .c-pink > ellipse { fill: #72243E; stroke: #ED93B1; }
|
||||
.c-pink > .th, .c-pink > text.th { fill: #F4C0D1; }
|
||||
.c-pink > .ts, .c-pink > text.ts { fill: #ED93B1; }
|
||||
|
||||
.c-gray > rect, .c-gray > circle, .c-gray > ellipse { fill: #444441; stroke: #B4B2A9; }
|
||||
.c-gray > .th, .c-gray > text.th { fill: #D3D1C7; }
|
||||
.c-gray > .ts, .c-gray > text.ts { fill: #B4B2A9; }
|
||||
|
||||
.c-blue > rect, .c-blue > circle, .c-blue > ellipse { fill: #0C447C; stroke: #85B7EB; }
|
||||
.c-blue > .th, .c-blue > text.th { fill: #B5D4F4; }
|
||||
.c-blue > .ts, .c-blue > text.ts { fill: #85B7EB; }
|
||||
|
||||
.c-green > rect, .c-green > circle, .c-green > ellipse { fill: #27500A; stroke: #97C459; }
|
||||
.c-green > .th, .c-green > text.th { fill: #C0DD97; }
|
||||
.c-green > .ts, .c-green > text.ts { fill: #97C459; }
|
||||
|
||||
.c-amber > rect, .c-amber > circle, .c-amber > ellipse { fill: #633806; stroke: #EF9F27; }
|
||||
.c-amber > .th, .c-amber > text.th { fill: #FAC775; }
|
||||
.c-amber > .ts, .c-amber > text.ts { fill: #EF9F27; }
|
||||
|
||||
.c-red > rect, .c-red > circle, .c-red > ellipse { fill: #791F1F; stroke: #F09595; }
|
||||
.c-red > .th, .c-red > text.th { fill: #F7C1C1; }
|
||||
.c-red > .ts, .c-red > text.ts { fill: #F09595; }
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="card">
|
||||
<h1><!-- DIAGRAM TITLE HERE --></h1>
|
||||
<p class="subtitle"><!-- OPTIONAL SUBTITLE HERE --></p>
|
||||
<!-- PASTE SVG HERE -->
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,191 @@
|
||||
---
|
||||
name: hyperframes
|
||||
description: Create HTML-based video compositions, animated title cards, social overlays, captioned talking-head videos, audio-reactive visuals, and shader transitions using HyperFrames. HTML is the source of truth for video. Use when the user wants a rendered MP4/WebM from an HTML composition, wants to animate text/logos/charts over media, needs captions synced to audio, wants TTS narration, or wants to convert a website into a video.
|
||||
version: 1.0.0
|
||||
author: heygen-com
|
||||
license: Apache-2.0
|
||||
platforms: [linux, macos, windows]
|
||||
prerequisites:
|
||||
commands: [node, ffmpeg, npx]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [creative, video, animation, html, gsap, motion-graphics]
|
||||
related_skills: [manim-video, meme-generation]
|
||||
category: creative
|
||||
requires_toolsets: [terminal]
|
||||
---
|
||||
|
||||
# HyperFrames
|
||||
|
||||
HTML is the source of truth for video. A composition is an HTML file with `data-*` attributes for timing, a GSAP timeline for animation, and CSS for appearance. The HyperFrames engine captures the page frame-by-frame and encodes to MP4/WebM with FFmpeg.
|
||||
|
||||
**Complement to `manim-video`:** Use `manim-video` for mathematical/geometric explainers (equations, 3B1B-style). Use `hyperframes` for motion-graphics, talking-head with captions, product tours, social overlays, shader transitions, and anything driven by real video/audio media.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks for a rendered video from text, a script, or a website
|
||||
- Animated title cards, lower thirds, or typographic intros
|
||||
- Captioned narration video (TTS + captions synced to waveform)
|
||||
- Audio-reactive visuals (beat sync, spectrum bars, pulsing glow)
|
||||
- Scene-to-scene transitions (crossfade, wipe, shader warp, flash-through-white)
|
||||
- Social overlays (Instagram/TikTok/YouTube style)
|
||||
- Website-to-video pipeline (capture a URL, produce a promo)
|
||||
- Any HTML/CSS/JS animation that must render deterministically to a video file
|
||||
|
||||
Do **not** use this skill for:
|
||||
- Pure math/equation animation (→ `manim-video`)
|
||||
- Image generation or memes (→ `meme-generation`, image models)
|
||||
- Live video conferencing or streaming
|
||||
|
||||
## Quick Reference
|
||||
|
||||
```bash
|
||||
npx hyperframes init my-video # scaffold a project
|
||||
cd my-video
|
||||
npx hyperframes lint # validate before preview/render
|
||||
npx hyperframes preview # live-reload browser preview (port 3002)
|
||||
npx hyperframes render --output final.mp4 # render to MP4
|
||||
npx hyperframes doctor # diagnose environment issues
|
||||
```
|
||||
|
||||
Render flags: `--quality draft|standard|high` · `--fps 24|30|60` · `--format mp4|webm` · `--docker` (reproducible) · `--strict`.
|
||||
|
||||
Full CLI reference: [references/cli.md](references/cli.md).
|
||||
|
||||
## Setup (one-time)
|
||||
|
||||
```bash
|
||||
bash "$(dirname "$(find ~/.hermes/skills -path '*/hyperframes/SKILL.md' 2>/dev/null | head -1)")/scripts/setup.sh"
|
||||
```
|
||||
|
||||
The script:
|
||||
1. Verifies Node.js >= 22 and FFmpeg are installed (prints fix instructions if not).
|
||||
2. Installs the `hyperframes` CLI globally (`npm install -g hyperframes@>=0.4.2`).
|
||||
3. Pre-caches `chrome-headless-shell` via Puppeteer — **required** for best-quality rendering via Chrome's `HeadlessExperimental.beginFrame` capture path.
|
||||
4. Runs `npx hyperframes doctor` and reports the result.
|
||||
|
||||
See [references/troubleshooting.md](references/troubleshooting.md) if setup fails.
|
||||
|
||||
## Procedure
|
||||
|
||||
### 1. Plan before writing HTML
|
||||
|
||||
Before touching code, articulate at a high level:
|
||||
- **What** — narrative arc, key moments, emotional beats
|
||||
- **Structure** — compositions, tracks (video/audio/overlays), durations
|
||||
- **Visual identity** — colors, fonts, motion character (explosive / cinematic / fluid / technical)
|
||||
- **Hero frame** — for each scene, the moment when the most elements are simultaneously visible. This is the static layout you'll build first.
|
||||
|
||||
**Visual Identity Gate (HARD-GATE).** Before writing ANY composition HTML, a visual identity must be defined. Do NOT write compositions with default or generic colors (`#333`, `#3b82f6`, `Roboto` are tells that this step was skipped). Check in order:
|
||||
|
||||
1. **`DESIGN.md` at project root?** → Use its exact colors, fonts, motion rules, and "What NOT to Do" constraints.
|
||||
2. **User named a style** (e.g. "Swiss Pulse", "dark and techy", "luxury brand")? → Generate a minimal `DESIGN.md` with `## Style Prompt`, `## Colors` (3-5 hex with roles), `## Typography` (1-2 families), `## What NOT to Do` (3-5 anti-patterns).
|
||||
3. **None of the above?** → Ask 3 questions before writing any HTML:
|
||||
- Mood? (explosive / cinematic / fluid / technical / chaotic / warm)
|
||||
- Light or dark canvas?
|
||||
- Any brand colors, fonts, or visual references?
|
||||
|
||||
Then generate a `DESIGN.md` from the answers. Every composition must trace its palette and typography back to `DESIGN.md` or explicit user direction.
|
||||
|
||||
### 2. Scaffold
|
||||
|
||||
```bash
|
||||
npx hyperframes init my-video --non-interactive
|
||||
```
|
||||
|
||||
Templates: `blank`, `warm-grain`, `play-mode`, `swiss-grid`, `vignelli`, `decision-tree`, `kinetic-type`, `product-promo`, `nyt-graph`. Pass `--example <name>` to pick one, `--video clip.mp4` or `--audio track.mp3` to seed with media.
|
||||
|
||||
### 3. Layout before animation
|
||||
|
||||
Write the static HTML+CSS for the **hero frame first** — no GSAP yet. The `.scene-content` container must fill the scene (`width:100%; height:100%; padding:Npx`) with `display:flex` + `gap`. Use padding to push content inward — never `position: absolute; top: Npx` on a content container (content overflows when taller than the remaining space).
|
||||
|
||||
Only after the hero frame looks right, add `gsap.from()` entrances (animate **to** the CSS position) and `gsap.to()` exits (animate **from** it).
|
||||
|
||||
See [references/composition.md](references/composition.md) for the full data-attribute schema and composition rules.
|
||||
|
||||
### 4. Animate with GSAP
|
||||
|
||||
Every composition must:
|
||||
- Register its timeline: `window.__timelines["<composition-id>"] = tl`
|
||||
- Start paused: `gsap.timeline({ paused: true })` — the player controls playback
|
||||
- Use finite `repeat` values (no `repeat: -1` — breaks the capture engine). Calculate: `repeat: Math.ceil(duration / cycleDuration) - 1`.
|
||||
- Be deterministic — no `Math.random()`, `Date.now()`, or wall-clock logic. Use a seeded PRNG if you need pseudo-randomness.
|
||||
- Build synchronously — no `async`/`await`, `setTimeout`, or Promises around timeline construction.
|
||||
|
||||
See [references/gsap.md](references/gsap.md) for the core GSAP API (tweens, eases, stagger, timelines).
|
||||
|
||||
### 5. Transitions between scenes
|
||||
|
||||
Multi-scene compositions require transitions. Rules:
|
||||
1. **Always use a transition between scenes** — no jump cuts.
|
||||
2. **Always use entrance animations** on every scene element (`gsap.from(...)`).
|
||||
3. **Never use exit animations** except on the final scene — the transition IS the exit.
|
||||
4. The final scene may fade out.
|
||||
|
||||
Use `npx hyperframes add <transition-name>` to install shader transitions (`flash-through-white`, `liquid-wipe`, etc.). Full list: `npx hyperframes add --list`.
|
||||
|
||||
### 6. Audio, captions, TTS, audio-reactive, highlighting
|
||||
|
||||
- **Audio:** always a separate `<audio>` element (video is `muted playsinline`).
|
||||
- **TTS:** `npx hyperframes tts "Script text" --voice af_nova --output narration.wav`. List voices with `--list`. Voice ID first letter encodes language (`a`/`b`=English, `e`=Spanish, `f`=French, `j`=Japanese, `z`=Mandarin, etc.) — the CLI auto-infers the phonemizer locale; pass `--lang` only to override. Non-English phonemization requires `espeak-ng` installed system-wide.
|
||||
- **Captions:** `npx hyperframes transcribe narration.wav` → word-level transcript. Pick style from the transcript tone (hype / corporate / tutorial / storytelling / social — see the table in `references/features.md`). **Language rule:** never use `.en` whisper models unless the audio is confirmed English — `.en` translates non-English audio instead of transcribing it. Every caption group MUST have a hard `tl.set(el, { opacity: 0, visibility: "hidden" }, group.end)` kill after its exit tween — otherwise groups leak visible into later ones.
|
||||
- **Audio-reactive visuals:** pre-extract audio bands (bass / mid / treble) and sample per-frame inside the timeline with a `for` loop of `tl.call(draw, [], f / fps)` — a single long tween does NOT react to audio. Map bass → `scale` (pulse), treble → `textShadow`/`boxShadow` (glow), overall amplitude → `opacity`/`y`/`backgroundColor`. Avoid equalizer-bar clichés — let content guide the visual, audio drive its behavior.
|
||||
- **Marker-style highlighting:** highlight, circle, burst, scribble, sketchout effects for text emphasis are deterministic CSS+GSAP — see `references/features.md#marker-highlighting`. Fully seekable, no animated SVG filters.
|
||||
- **Scene transitions:** every multi-scene composition MUST use transitions (no jump cuts). Pick from CSS primitives (push slide, blur crossfade, zoom through, staggered blocks) or shader transitions (`flash-through-white`, `liquid-wipe`, `cross-warp-morph`, `chromatic-split`, etc.) via `npx hyperframes add`. Mood and energy tables live in `references/features.md#transitions`. Do not mix CSS and shader transitions in the same composition.
|
||||
|
||||
### 7. Lint, validate, inspect, preview, render
|
||||
|
||||
```bash
|
||||
npx hyperframes lint # catches missing data-composition-id, overlapping tracks, unregistered timelines
|
||||
npx hyperframes validate # WCAG contrast audit at 5 timestamps
|
||||
npx hyperframes inspect # visual layout audit — overflow, off-frame elements, occluded text
|
||||
npx hyperframes preview # live browser preview
|
||||
npx hyperframes render --quality draft --output draft.mp4 # fast iteration
|
||||
npx hyperframes render --quality high --output final.mp4 # final delivery
|
||||
```
|
||||
|
||||
`hyperframes validate` samples background pixels behind every text element and warns on contrast ratios below 4.5:1 (or 3:1 for large text). `hyperframes inspect` is the layout-side companion — runs the page at multiple timestamps and flags issues that a static lint can't see (a caption that wraps past the safe area only at 4.5s, a card that overflows when its title is the longest variant, an element that ends up behind a transition shader). Run `inspect` especially on compositions with speech bubbles, cards, captions, or tight typography.
|
||||
|
||||
### 8. Website-to-video (if the user gives a URL)
|
||||
|
||||
Use the 7-step capture-to-video workflow in [references/website-to-video.md](references/website-to-video.md): capture → DESIGN.md → SCRIPT.md → storyboard → composition → render → deliver.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **`HeadlessExperimental.beginFrame' wasn't found`** — Chromium 147+ removed this protocol. Ensure you're on `hyperframes@>=0.4.2` (auto-detects and falls back to screenshot mode). Escape hatch: `export PRODUCER_FORCE_SCREENSHOT=true`. See [hyperframes#294](https://github.com/heygen-com/hyperframes/issues/294) and [references/troubleshooting.md](references/troubleshooting.md).
|
||||
- **System Chrome (not `chrome-headless-shell`)** — renders hang for 120s then timeout. Run `npx puppeteer browsers install chrome-headless-shell` (setup.sh does this). `hyperframes doctor` reports which binary will be used.
|
||||
- **`repeat: -1` anywhere** — breaks the capture engine. Always compute a finite repeat count.
|
||||
- **`gsap.set()` on clip elements that enter later** — the element doesn't exist at page load. Use `tl.set(selector, vars, timePosition)` inside the timeline instead, at or after the clip's `data-start`.
|
||||
- **`<br>` inside content text** — forced breaks don't know the rendered font width, so natural wrap + `<br>` double-breaks. Use `max-width` to let text wrap. Exception: short display titles where each word is deliberately on its own line.
|
||||
- **Animating `visibility` or `display`** — GSAP can't tween these. Use `autoAlpha` (handles both visibility and opacity).
|
||||
- **Calling `video.play()` or `audio.play()`** — the framework owns playback. Never call these yourself.
|
||||
- **Building timelines async** — the capture engine reads `window.__timelines` synchronously after page load. Never wrap timeline construction in `async`, `setTimeout`, or a Promise.
|
||||
- **Standalone `index.html` wrapped in `<template>`** — hides all content from the browser. Only **sub-compositions** loaded via `data-composition-src` use `<template>`.
|
||||
- **Using video for audio** — always muted `<video>` + separate `<audio>`.
|
||||
|
||||
## Verification
|
||||
|
||||
Before and after rendering:
|
||||
|
||||
1. **Lint + validate + inspect pass:** `npx hyperframes lint --strict && npx hyperframes validate && npx hyperframes inspect` (lint catches structural issues, validate catches contrast, inspect catches visual layout / overflow issues — see troubleshooting.md if warnings appear).
|
||||
2. **Animation choreography** — for new compositions or significant animation changes, run the animation map. `npx hyperframes init` copies the skill scripts into the project, so the path is project-local:
|
||||
```bash
|
||||
node skills/hyperframes/scripts/animation-map.mjs <composition-dir> \
|
||||
--out <composition-dir>/.hyperframes/anim-map
|
||||
```
|
||||
Outputs a single `animation-map.json` with per-tween summaries, ASCII Gantt timeline, stagger detection, dead zones (>1s with no animation), element lifecycles, and flags (`offscreen`, `collision`, `invisible`, `paced-fast` <0.2s, `paced-slow` >2s). Scan summaries and flags — fix or justify each. Skip on small edits.
|
||||
3. **File exists + non-zero:** `ls -lh final.mp4`.
|
||||
4. **Duration matches `data-duration`:** `ffprobe -v error -show_entries format=duration -of default=nw=1:nk=1 final.mp4`.
|
||||
5. **Visual check:** extract a mid-composition frame: `ffmpeg -i final.mp4 -ss 00:00:05 -vframes 1 preview.png`.
|
||||
6. **Audio present if expected:** `ffprobe -v error -show_streams -select_streams a -of default=nw=1:nk=1 final.mp4 | head -1`.
|
||||
|
||||
If `hyperframes render` fails, run `npx hyperframes doctor` and attach its output when reporting.
|
||||
|
||||
## References
|
||||
|
||||
- [composition.md](references/composition.md) — data attributes, timeline contract, non-negotiable rules, typography/asset rules
|
||||
- [cli.md](references/cli.md) — every CLI command (init, capture, lint, validate, inspect, preview, render, transcribe, tts, doctor, browser, info, upgrade, benchmark)
|
||||
- [gsap.md](references/gsap.md) — GSAP core API for HyperFrames (tweens, eases, stagger, timelines, matchMedia)
|
||||
- [features.md](references/features.md) — captions, TTS, audio-reactive, marker highlighting, transitions (load on demand)
|
||||
- [website-to-video.md](references/website-to-video.md) — 7-step capture-to-video workflow
|
||||
- [troubleshooting.md](references/troubleshooting.md) — OpenClaw fix, env vars, common render errors
|
||||
@@ -0,0 +1,185 @@
|
||||
# HyperFrames CLI
|
||||
|
||||
Everything runs through `npx hyperframes` (or the globally-installed `hyperframes` after `npm install -g hyperframes`). Requires Node.js >= 22 and FFmpeg.
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Scaffold** — `npx hyperframes init my-video` (or `npx hyperframes capture <url>` if starting from a website)
|
||||
2. **Write** — author HTML composition (see `composition.md`)
|
||||
3. **Lint** — `npx hyperframes lint`
|
||||
4. **Validate** — `npx hyperframes validate` (WCAG contrast audit)
|
||||
5. **Inspect** — `npx hyperframes inspect` (visual layout audit)
|
||||
6. **Preview** — `npx hyperframes preview`
|
||||
7. **Render** — `npx hyperframes render`
|
||||
|
||||
Always lint before preview/render — catches missing `data-composition-id`, overlapping tracks, and unregistered timelines.
|
||||
|
||||
## init — Scaffold a Project
|
||||
|
||||
```bash
|
||||
npx hyperframes init my-video # interactive wizard
|
||||
npx hyperframes init my-video --example warm-grain # pick an example template
|
||||
npx hyperframes init my-video --video clip.mp4 # seed with a video file
|
||||
npx hyperframes init my-video --audio track.mp3 # seed with an audio file
|
||||
npx hyperframes init my-video --non-interactive # skip prompts (CI / agent use)
|
||||
```
|
||||
|
||||
Templates: `blank`, `warm-grain`, `play-mode`, `swiss-grid`, `vignelli`, `decision-tree`, `kinetic-type`, `product-promo`, `nyt-graph`.
|
||||
|
||||
`init` creates the correct file structure, copies media, transcribes audio with Whisper, and installs authoring skills. Use it instead of creating files by hand.
|
||||
|
||||
## capture — Website → Editable Components
|
||||
|
||||
```bash
|
||||
npx hyperframes capture https://example.com # → captures/example.com/
|
||||
npx hyperframes capture https://stripe.com -o stripe-video # custom output dir
|
||||
npx hyperframes capture https://example.com --json # machine-readable output
|
||||
npx hyperframes capture https://example.com --skip-assets # skip images/SVGs
|
||||
```
|
||||
|
||||
Captures the site into `captures/<hostname>/capture/` by default, producing `capture/screenshots/`, `capture/assets/`, `capture/extracted/` (tokens.json, visible-text.txt, fonts.json), and a self-contained snapshot.
|
||||
|
||||
All downstream steps (DESIGN.md, SCRIPT.md, STORYBOARD, composition) read from the `capture/` subfolder — see `website-to-video.md`.
|
||||
|
||||
## lint
|
||||
|
||||
```bash
|
||||
npx hyperframes lint # current directory
|
||||
npx hyperframes lint ./my-project # specific project
|
||||
npx hyperframes lint --verbose # include info-level findings
|
||||
npx hyperframes lint --json # machine-readable output
|
||||
```
|
||||
|
||||
Lints `index.html` and all files in `compositions/`. Reports errors (must fix), warnings (should fix), and info (only with `--verbose`).
|
||||
|
||||
## validate
|
||||
|
||||
```bash
|
||||
npx hyperframes validate # WCAG contrast audit at 5 timestamps
|
||||
npx hyperframes validate --no-contrast # skip while iterating
|
||||
```
|
||||
|
||||
Seeks to 5 timestamps, screenshots the page, samples background pixels behind every text element, and warns on contrast ratios below 4.5:1 (normal text) or 3:1 (large text — 24px+, or 19px+ bold). Run before final render.
|
||||
|
||||
## inspect
|
||||
|
||||
```bash
|
||||
npx hyperframes inspect # visual layout audit at 5 timestamps
|
||||
npx hyperframes inspect ./my-project # specific project
|
||||
npx hyperframes inspect --json # agent-readable findings
|
||||
npx hyperframes inspect --samples 15 # denser timeline sweep
|
||||
npx hyperframes inspect --at 1.5,4,7.25 # explicit hero-frame timestamps
|
||||
```
|
||||
|
||||
Use this after `lint` and `validate`, especially for compositions with speech bubbles, cards, captions, or tight typography. Reports overflow, off-frame elements, occluded text, contrast warnings, and per-timestamp layout summaries — catches issues that pure timeline lint can't see (e.g., a caption that wraps past the safe area only at a specific timestamp).
|
||||
|
||||
`npx hyperframes layout` is a compatibility alias for the same visual inspection pass.
|
||||
|
||||
## preview
|
||||
|
||||
```bash
|
||||
npx hyperframes preview # serve current directory (port 3002)
|
||||
npx hyperframes preview --port 4567 # custom port
|
||||
```
|
||||
|
||||
Hot-reloads on file changes. Opens the Studio in your browser automatically.
|
||||
|
||||
## render
|
||||
|
||||
```bash
|
||||
npx hyperframes render # standard MP4
|
||||
npx hyperframes render --output final.mp4 # named output
|
||||
npx hyperframes render --quality draft # fast iteration
|
||||
npx hyperframes render --fps 60 --quality high # final delivery
|
||||
npx hyperframes render --format webm # transparent WebM
|
||||
npx hyperframes render --docker # byte-identical reproducible render
|
||||
```
|
||||
|
||||
| Flag | Options | Default | Notes |
|
||||
| -------------- | ----------------------- | ------------------------------ | --------------------------- |
|
||||
| `--output` | path | `renders/<name>_<timestamp>.mp4` | Output path |
|
||||
| `--fps` | 24, 30, 60 | 30 | 60fps doubles render time |
|
||||
| `--quality` | `draft`, `standard`, `high` | standard | draft for iterating |
|
||||
| `--format` | `mp4`, `webm` | mp4 | WebM supports transparency |
|
||||
| `--workers` | 1–8 or `auto` | auto | Each spawns Chrome |
|
||||
| `--docker` | flag | off | Reproducible output |
|
||||
| `--gpu` | flag | off | GPU-accelerated encoding |
|
||||
| `--strict` | flag | off | Fail on lint errors |
|
||||
| `--strict-all` | flag | off | Fail on errors AND warnings |
|
||||
|
||||
**Quality guidance:** `draft` while iterating, `standard` for review, `high` for final delivery.
|
||||
|
||||
## transcribe
|
||||
|
||||
```bash
|
||||
npx hyperframes transcribe audio.mp3
|
||||
npx hyperframes transcribe video.mp4 --model medium.en --language en
|
||||
npx hyperframes transcribe subtitles.srt # import existing
|
||||
npx hyperframes transcribe subtitles.vtt
|
||||
npx hyperframes transcribe openai-response.json
|
||||
```
|
||||
|
||||
Produces word-level timings suitable for caption components. First run downloads the Whisper model (cached after).
|
||||
|
||||
## tts
|
||||
|
||||
```bash
|
||||
npx hyperframes tts "Text here" --voice af_nova --output narration.wav
|
||||
npx hyperframes tts script.txt --voice bf_emma
|
||||
npx hyperframes tts "La reunión empieza a las nueve" --voice ef_dora --output es.wav
|
||||
npx hyperframes tts "Hello there" --voice af_heart --lang fr-fr --output accented.wav
|
||||
npx hyperframes tts --list # show all voices
|
||||
```
|
||||
|
||||
Uses Kokoro (local, no API key). Voice ID first letter encodes language: `a` American English, `b` British English, `e` Spanish, `f` French, `h` Hindi, `i` Italian, `j` Japanese, `p` Brazilian Portuguese, `z` Mandarin. The CLI auto-infers the phonemizer locale from that prefix — pass `--lang` only to override (e.g. stylized accents). Valid `--lang` codes: `en-us`, `en-gb`, `es`, `fr-fr`, `hi`, `it`, `pt-br`, `ja`, `zh`. Non-English phonemization requires `espeak-ng` installed system-wide (`apt-get install espeak-ng` / `brew install espeak-ng`).
|
||||
|
||||
## doctor
|
||||
|
||||
```bash
|
||||
npx hyperframes doctor
|
||||
```
|
||||
|
||||
Verifies environment:
|
||||
- Node.js >= 22
|
||||
- FFmpeg present on PATH
|
||||
- Available RAM (renders are memory-hungry — 4 GB minimum)
|
||||
- Chrome binary resolution (`chrome-headless-shell` preferred over system Chrome)
|
||||
- Current `hyperframes` version
|
||||
|
||||
Run this **first** when a render fails. See `troubleshooting.md` for interpreting the output.
|
||||
|
||||
## browser
|
||||
|
||||
```bash
|
||||
npx hyperframes browser --install # install the bundled chrome-headless-shell
|
||||
npx hyperframes browser --path # print the resolved browser binary path
|
||||
npx hyperframes browser --clean # clear the bundled browser cache
|
||||
```
|
||||
|
||||
## info
|
||||
|
||||
```bash
|
||||
npx hyperframes info
|
||||
```
|
||||
|
||||
Prints version, Node version, FFmpeg version, OS, and resolved browser path — useful in bug reports.
|
||||
|
||||
## upgrade
|
||||
|
||||
```bash
|
||||
npx hyperframes upgrade -y
|
||||
```
|
||||
|
||||
Check for and install updates. Run this if you hit `HeadlessExperimental.beginFrame` errors — the auto-detect fix shipped in `hyperframes@0.4.2` (commit 4c72ba4, March 2026).
|
||||
|
||||
## Other
|
||||
|
||||
```bash
|
||||
npx hyperframes compositions # list compositions in the project
|
||||
npx hyperframes docs # open documentation in browser
|
||||
npx hyperframes benchmark . # benchmark render performance
|
||||
npx hyperframes add <block> # install a block/component from the catalog
|
||||
npx hyperframes add --list # browse the catalog
|
||||
```
|
||||
|
||||
Popular catalog blocks: `flash-through-white` (shader transition), `instagram-follow` (social overlay), `data-chart` (animated chart), `lower-third` (talking-head overlay). See [hyperframes.heygen.com/catalog](https://hyperframes.heygen.com/catalog).
|
||||
@@ -0,0 +1,129 @@
|
||||
# Composition Authoring
|
||||
|
||||
HTML structure, data attributes, timeline contract, and non-negotiable rules.
|
||||
|
||||
## Root Structure
|
||||
|
||||
Standalone `index.html` — the top-level composition. **Does NOT use `<template>`**. Put the `data-composition-id` div directly in `<body>`.
|
||||
|
||||
```html
|
||||
<!doctype html>
|
||||
<html>
|
||||
<body>
|
||||
<div
|
||||
id="stage"
|
||||
data-composition-id="root"
|
||||
data-start="0"
|
||||
data-duration="10"
|
||||
data-width="1920"
|
||||
data-height="1080"
|
||||
>
|
||||
<!-- clips go here -->
|
||||
<video id="clip-1" data-start="0" data-duration="5" data-track-index="0" src="intro.mp4" muted playsinline></video>
|
||||
<img id="logo" data-start="2" data-duration="3" data-track-index="1" src="logo.png" />
|
||||
<audio id="music" data-start="0" data-duration="10" data-track-index="2" data-volume="0.5" src="music.wav"></audio>
|
||||
</div>
|
||||
|
||||
<script src="https://cdn.jsdelivr.net/npm/gsap@3.14.2/dist/gsap.min.js"></script>
|
||||
<script>
|
||||
window.__timelines = window.__timelines || {};
|
||||
const tl = gsap.timeline({ paused: true });
|
||||
tl.from("#logo", { opacity: 0, y: 40, duration: 0.6 }, 2);
|
||||
window.__timelines["root"] = tl;
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
```
|
||||
|
||||
Sub-compositions loaded via `data-composition-src` **DO** use `<template>`:
|
||||
|
||||
```html
|
||||
<template id="my-comp-template">
|
||||
<div data-composition-id="my-comp" data-width="1920" data-height="1080">
|
||||
<!-- content + scoped <style> + <script> with window.__timelines["my-comp"] -->
|
||||
</div>
|
||||
</template>
|
||||
```
|
||||
|
||||
Load from the root: `<div id="el-1" data-composition-id="my-comp" data-composition-src="compositions/my-comp.html" data-start="0" data-duration="10" data-track-index="1"></div>`
|
||||
|
||||
## Data Attributes
|
||||
|
||||
### All clips
|
||||
|
||||
| Attribute | Required | Values |
|
||||
| ------------------ | --------------------------------- | ------------------------------------------------------ |
|
||||
| `id` | Yes | Unique identifier |
|
||||
| `data-start` | Yes | Seconds, or clip ID reference (`"el-1"`, `"intro + 2"`) |
|
||||
| `data-duration` | Required for img/div/compositions | Seconds. Video/audio defaults to media duration. |
|
||||
| `data-track-index` | Yes | Integer. Same-track clips cannot overlap. |
|
||||
| `data-media-start` | No | Trim offset into source (seconds) |
|
||||
| `data-volume` | No | 0–1 (default 1) |
|
||||
|
||||
`data-track-index` controls timeline layout only — **not** visual layering. Use CSS `z-index` for layering.
|
||||
|
||||
### Composition clips
|
||||
|
||||
| Attribute | Required | Values |
|
||||
| ---------------------------- | -------- | -------------------------------------------- |
|
||||
| `data-composition-id` | Yes | Unique composition ID |
|
||||
| `data-start` | Yes | Start time (root composition: `"0"`) |
|
||||
| `data-duration` | Yes | Takes precedence over GSAP timeline duration |
|
||||
| `data-width` / `data-height` | Yes | Pixel dimensions (1920x1080 or 1080x1920) |
|
||||
| `data-composition-src` | No | Path to external HTML file |
|
||||
|
||||
## Timeline Contract
|
||||
|
||||
- Every timeline starts `{ paused: true }` — the player controls playback.
|
||||
- Register every timeline: `window.__timelines["<composition-id>"] = tl`.
|
||||
- Duration comes from `data-duration`, not from the GSAP timeline length.
|
||||
- Framework auto-nests sub-timelines — do NOT manually add them.
|
||||
- Never create empty tweens just to set duration.
|
||||
|
||||
## Non-Negotiable Rules
|
||||
|
||||
1. **Deterministic.** No `Math.random()`, `Date.now()`, or time-based logic. Use a seeded PRNG (e.g. mulberry32) if you need pseudo-randomness.
|
||||
2. **GSAP only on visual properties.** `opacity`, `x`, `y`, `scale`, `rotation`, `color`, `backgroundColor`, `borderRadius`, transforms. Never animate `visibility`, `display`, or call `video.play()`/`audio.play()`.
|
||||
3. **No property conflicts across timelines.** Never animate the same property on the same element from multiple timelines simultaneously.
|
||||
4. **No `repeat: -1`.** Infinite-repeat tweens break the capture engine. Compute `repeat: Math.ceil(duration / cycleDuration) - 1`.
|
||||
5. **Synchronous timeline construction.** Never build timelines inside `async`/`await`, `setTimeout`, or Promises. The capture engine reads `window.__timelines` synchronously after page load. Fonts are embedded by the compiler — no need to wait for load.
|
||||
6. **Root composition has no `<template>` wrapper.** Only sub-compositions use `<template>`.
|
||||
7. **Video is always `muted playsinline`.** Audio is always a separate `<audio>` element — even if it's the same source file.
|
||||
8. **Content containers use padding, not absolute positioning.** `.scene-content { width: 100%; height: 100%; padding: Npx; display: flex; flex-direction: column; gap: Npx; box-sizing: border-box }`. Absolute-positioned content containers overflow. Reserve `position: absolute` for decoratives only.
|
||||
|
||||
## Scene Transitions
|
||||
|
||||
Multi-scene compositions MUST follow all of these:
|
||||
|
||||
1. **Always use a transition between scenes.** No jump cuts.
|
||||
2. **Always use entrance animations** on every scene element. Every element animates IN via `gsap.from(...)`. No element may appear fully-formed.
|
||||
3. **Never use exit animations** (except on the final scene). This means NO `gsap.to()` that animates `opacity` to 0, `y` offscreen, etc. The transition IS the exit. Outgoing scene content must be fully visible at the moment the transition starts.
|
||||
4. **Final scene only:** may fade elements out. This is the only scene where `gsap.to(..., { opacity: 0 })` is allowed.
|
||||
|
||||
## Typography and Assets
|
||||
|
||||
- **Fonts:** write the `font-family` you want in CSS — the compiler embeds supported fonts automatically. Unsupported fonts produce a compiler warning.
|
||||
- Add `crossorigin="anonymous"` to external media.
|
||||
- For dynamic text sizing, use `window.__hyperframes.fitTextFontSize(text, { maxWidth, fontFamily, fontWeight })`.
|
||||
- All project files live at the project root alongside `index.html`. Sub-compositions reference assets with `../`.
|
||||
- For rendered video: 60px+ headlines, 20px+ body, 16px+ data labels. `font-variant-numeric: tabular-nums` on number columns. Avoid full-screen linear gradients on dark backgrounds (H.264 banding — use radial or solid + localized glow).
|
||||
|
||||
## Animation Guardrails
|
||||
|
||||
- Offset the first animation 0.1–0.3s (not `t=0`).
|
||||
- Vary eases across entrance tweens — at least 3 different eases per scene.
|
||||
- Don't repeat an entrance pattern within a scene.
|
||||
|
||||
## Never Do
|
||||
|
||||
1. Forget `window.__timelines` registration.
|
||||
2. Use video for audio — always muted video + separate `<audio>`.
|
||||
3. Nest video inside a timed div — use a non-timed wrapper.
|
||||
4. Use `data-layer` (use `data-track-index`) or `data-end` (use `data-duration`).
|
||||
5. Animate video element dimensions — animate a wrapper div instead.
|
||||
6. Call `play`/`pause`/`seek` on media — framework owns playback.
|
||||
7. Create a top-level container without `data-composition-id`.
|
||||
8. Use `repeat: -1` on any timeline or tween.
|
||||
9. Build timelines asynchronously.
|
||||
10. Use `gsap.set()` on elements from later scenes — they don't exist in the DOM at page load. Use `tl.set(selector, vars, timePosition)` inside the timeline at or after the clip's `data-start`.
|
||||
11. Use `<br>` in content text — causes unwanted extra breaks when the text wraps naturally. Use `max-width` instead. Exception: short display titles (e.g., "THE\nIMMORTAL\nGAME") where each word is deliberately on its own line.
|
||||
@@ -0,0 +1,289 @@
|
||||
# HyperFrames Feature Reference
|
||||
|
||||
Load this file when a composition needs captions, TTS narration, audio-reactive visuals, marker-style text highlighting, or scene transitions. All patterns here are deterministic (no `Math.random()`, no `Date.now()`, no runtime audio analysis) and live on the same GSAP timeline as the rest of the composition.
|
||||
|
||||
## Captions
|
||||
|
||||
### Language Rule (Non-Negotiable)
|
||||
|
||||
**Never use `.en` whisper models unless the audio is confirmed English.** `.en` models TRANSLATE non-English audio into English instead of transcribing it.
|
||||
|
||||
- User says the language → `npx hyperframes transcribe audio.mp3 --model small --language <code>` (no `.en`)
|
||||
- User confirms English → `--model small.en`
|
||||
- Language unknown → `--model small` (auto-detects)
|
||||
|
||||
### Style Detection
|
||||
|
||||
If the user doesn't specify a caption style, detect it from the transcript tone:
|
||||
|
||||
| Tone | Font mood | Animation | Color | Size |
|
||||
| ------------ | ------------------------ | ---------------------------------- | --------------------------- | ------- |
|
||||
| Hype / launch | Heavy condensed, 800-900 | Scale-pop, `back.out(1.7)`, 0.1-0.2s | Bright on dark | 72-96px |
|
||||
| Corporate | Clean sans, 600-700 | Fade+slide, `power3.out`, 0.3s | White / neutral + muted accent | 56-72px |
|
||||
| Tutorial | Mono / clean sans, 500-600 | Typewriter or fade, 0.4-0.5s | High contrast, minimal | 48-64px |
|
||||
| Storytelling | Serif / elegant, 400-500 | Slow fade, `power2.out`, 0.5-0.6s | Warm muted tones | 44-56px |
|
||||
| Social | Rounded sans, 700-800 | Bounce, `elastic.out`, word-by-word | Playful, colored pills | 56-80px |
|
||||
|
||||
### Word Grouping
|
||||
|
||||
- High energy: 2-3 words, quick turnover.
|
||||
- Conversational: 3-5 words, natural phrases.
|
||||
- Measured / calm: 4-6 words.
|
||||
|
||||
Break on sentence boundaries, 150ms+ pauses, or a max word count.
|
||||
|
||||
### Positioning
|
||||
|
||||
- Landscape (1920x1080): bottom 80-120px, centered.
|
||||
- Portrait (1080x1920): ~600-700px from bottom, centered.
|
||||
- Never cover the subject's face. `position: absolute` (never relative). One caption group visible at a time.
|
||||
|
||||
### Text Overflow Prevention
|
||||
|
||||
Use the runtime helper so captions never overflow:
|
||||
|
||||
```js
|
||||
const result = window.__hyperframes.fitTextFontSize(group.text.toUpperCase(), {
|
||||
fontFamily: "Outfit",
|
||||
fontWeight: 900,
|
||||
maxWidth: 1600, // 1600 landscape, 900 portrait
|
||||
});
|
||||
el.style.fontSize = result.fontSize + "px";
|
||||
```
|
||||
|
||||
When per-word styling uses `scale > 1.0`, compute `maxWidth = safeWidth / maxScale` to leave headroom. Container needs `overflow: visible` (not `hidden` — hidden clips scaled emphasis words and glow).
|
||||
|
||||
### Caption Exit Guarantee
|
||||
|
||||
Every group MUST have a hard kill after its exit tween — otherwise groups leak into later ones:
|
||||
|
||||
```js
|
||||
tl.to(groupEl, { opacity: 0, scale: 0.95, duration: 0.12, ease: "power2.in" }, group.end - 0.12);
|
||||
tl.set(groupEl, { opacity: 0, visibility: "hidden" }, group.end); // deterministic kill
|
||||
```
|
||||
|
||||
### Per-Word Styling
|
||||
|
||||
Scan the transcript for words that deserve distinct treatment:
|
||||
|
||||
- Brand / product names — larger, unique color.
|
||||
- ALL CAPS — scale boost, flash, accent color.
|
||||
- Numbers / statistics — bold weight, accent color.
|
||||
- Emotional keywords — exaggerated animation (overshoot, bounce).
|
||||
- Call-to-action — highlight, underline, color pop.
|
||||
|
||||
## TTS (Kokoro-82M)
|
||||
|
||||
Local, no API key. Runs on CPU. Model downloads on first use (~311 MB + ~27 MB voices, cached in `~/.cache/hyperframes/tts/`).
|
||||
|
||||
### Voice Selection
|
||||
|
||||
| Content type | Voice | Why |
|
||||
| ------------- | ----------------------- | --------------------------- |
|
||||
| Product demo | `af_heart` / `af_nova` | Warm, professional |
|
||||
| Tutorial | `am_adam` / `bf_emma` | Neutral, easy to follow |
|
||||
| Marketing | `af_sky` / `am_michael` | Energetic or authoritative |
|
||||
| Documentation | `bf_emma` / `bm_george` | Clear British English |
|
||||
| Casual | `af_heart` / `af_sky` | Approachable, natural |
|
||||
|
||||
Run `npx hyperframes tts --list` for all 54 voices across 8 languages.
|
||||
|
||||
### Multilingual Phonemization
|
||||
|
||||
Voice ID first letter encodes language: `a`=American English, `b`=British English, `e`=Spanish, `f`=French, `h`=Hindi, `i`=Italian, `j`=Japanese, `p`=Brazilian Portuguese, `z`=Mandarin. The CLI auto-infers the phonemizer locale from that prefix — you don't need `--lang` when voice and text match.
|
||||
|
||||
```bash
|
||||
npx hyperframes tts "La reunión empieza a las nueve" --voice ef_dora --output es.wav
|
||||
npx hyperframes tts "今日はいい天気ですね" --voice jf_alpha --output ja.wav
|
||||
```
|
||||
|
||||
Pass `--lang` only to override auto-detection (e.g. stylized accents):
|
||||
|
||||
```bash
|
||||
npx hyperframes tts "Hello there" --voice af_heart --lang fr-fr --output accented.wav
|
||||
```
|
||||
|
||||
Valid `--lang` codes: `en-us`, `en-gb`, `es`, `fr-fr`, `hi`, `it`, `pt-br`, `ja`, `zh`. Non-English phonemization requires `espeak-ng` installed system-wide (`apt-get install espeak-ng` / `brew install espeak-ng`).
|
||||
|
||||
### Speed
|
||||
|
||||
- `0.7-0.8` — tutorial, complex content
|
||||
- `1.0` — natural (default)
|
||||
- `1.1-1.2` — intros, upbeat content
|
||||
- `1.5+` — rarely appropriate
|
||||
|
||||
### TTS + Captions Workflow
|
||||
|
||||
```bash
|
||||
npx hyperframes tts script.txt --voice af_heart --output narration.wav
|
||||
npx hyperframes transcribe narration.wav # → transcript.json (word-level)
|
||||
```
|
||||
|
||||
## Audio-Reactive Visuals
|
||||
|
||||
Drive visuals from music, voice, or sound. Any GSAP-tweenable property can respond to pre-extracted audio data.
|
||||
|
||||
### Data format
|
||||
|
||||
```js
|
||||
const AUDIO_DATA = {
|
||||
fps: 30,
|
||||
totalFrames: 900,
|
||||
frames: [{ bands: [0.82, 0.45, 0.31, /* ... */] }, /* ... */],
|
||||
};
|
||||
```
|
||||
|
||||
`frames[i].bands[]` are frequency band amplitudes, 0-1. Index 0 = bass, higher indices = treble. Each band is normalized independently across the full track.
|
||||
|
||||
### Mapping audio to visuals
|
||||
|
||||
| Audio signal | Visual property | Effect |
|
||||
| ---------------------- | --------------------------------- | -------------------------- |
|
||||
| Bass (`bands[0]`) | `scale` | Pulse on beat |
|
||||
| Treble (`bands[12-14]`)| `textShadow`, `boxShadow` | Glow intensity |
|
||||
| Overall amplitude | `opacity`, `y`, `backgroundColor` | Breathe, lift, color shift |
|
||||
| Mid-range (`bands[4-8]`)| `borderRadius`, `width` | Shape morphing |
|
||||
|
||||
Any GSAP-tweenable property works — `clipPath`, `filter`, SVG attributes, CSS custom properties. Let content guide the visual and let audio drive its behavior. **Never add** equalizer bars, spectrum analyzers, waveform displays, rainbow cycling, or generic particle systems — they look cheap.
|
||||
|
||||
### Sampling pattern (required)
|
||||
|
||||
Audio reactivity needs per-frame sampling via a `for` loop of `tl.call()`, NOT a single tween. A single long tween does NOT react to audio:
|
||||
|
||||
```js
|
||||
for (let f = 0; f < AUDIO_DATA.totalFrames; f++) {
|
||||
tl.call(
|
||||
((frame) => () => draw(frame))(AUDIO_DATA.frames[f]),
|
||||
[],
|
||||
f / AUDIO_DATA.fps,
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Gotchas
|
||||
|
||||
- **textShadow on a container** with semi-transparent children (e.g. inactive caption words at `rgba(255,255,255,0.3)`) renders a visible glow rectangle behind every child. Apply the glow to active words individually, not to the container.
|
||||
- **Subtlety for text** — 3-6% scale variation, soft glow. Heavy pulsing makes text unreadable.
|
||||
- **Go bigger on non-text** — backgrounds and shapes can handle 10-30% swings.
|
||||
- **Deterministic only** — pre-extracted audio data, no Web Audio API, no runtime analysis.
|
||||
|
||||
## Marker-Style Highlighting
|
||||
|
||||
Deterministic CSS + GSAP implementations of the classic "highlight / circle / burst / scribble / sketchout" drawing modes for emphasizing text. Fully seekable — no animated SVG filters, no JS timers.
|
||||
|
||||
### Highlight (yellow marker sweep)
|
||||
|
||||
```html
|
||||
<span class="mh-highlight-wrap">
|
||||
<span class="mh-highlight-bar" id="hl-1"></span>
|
||||
<span class="mh-highlight-text">highlighted text</span>
|
||||
</span>
|
||||
```
|
||||
|
||||
```css
|
||||
.mh-highlight-wrap { position: relative; display: inline; }
|
||||
.mh-highlight-bar {
|
||||
position: absolute; inset: 0 -6px;
|
||||
background: #fdd835; opacity: 0.35;
|
||||
transform: scaleX(0); transform-origin: left center;
|
||||
border-radius: 3px; z-index: 0;
|
||||
}
|
||||
.mh-highlight-text { position: relative; z-index: 1; }
|
||||
```
|
||||
|
||||
```js
|
||||
tl.to("#hl-1", { scaleX: 1, duration: 0.5, ease: "power2.out" }, 0.6);
|
||||
```
|
||||
|
||||
Multi-line: apply to `.mh-highlight-bar` with `stagger: 0.3`.
|
||||
|
||||
### Circle
|
||||
|
||||
Hand-drawn ellipse around a word. Use a positioned `::before` with `border-radius: 50%`, slight rotation, and `clip-path` to avoid covering the letters. Animate `clip-path` or `stroke-dashoffset` on an inline SVG circle.
|
||||
|
||||
### Burst
|
||||
|
||||
Short radiating lines around a word. Render 6-12 small `<span>` elements positioned in a radial pattern; animate `scaleY` from 0.
|
||||
|
||||
### Scribble
|
||||
|
||||
A chaotic overlay created by animating `stroke-dashoffset` on an inline SVG `<path>` with a `d` attribute describing a zig-zag. Seed values, never `Math.random()`.
|
||||
|
||||
### Sketchout
|
||||
|
||||
A rough rectangle outline. Two `<rect>`s with slight `transform` offsets, animated via `stroke-dashoffset`.
|
||||
|
||||
All five modes tween CSS transforms or `stroke-dashoffset` only — both tween cleanly, are deterministic, and seek correctly.
|
||||
|
||||
## Scene Transitions
|
||||
|
||||
Every multi-scene composition MUST use transitions. No jump cuts.
|
||||
|
||||
### Energy → primary transition
|
||||
|
||||
| Energy | CSS primary | Shader primary | Accent | Duration | Easing |
|
||||
| ------------------------------------ | ---------------------------- | ------------------------------------ | ------------------------------ | --------- | ------------------------ |
|
||||
| **Calm** (wellness, brand, luxury) | Blur crossfade, focus pull | Cross-warp morph, thermal distortion | Light leak, circle iris | 0.5-0.8s | `sine.inOut`, `power1` |
|
||||
| **Medium** (corporate, SaaS) | Push slide, staggered blocks | Whip pan, cinematic zoom | Squeeze, vertical push | 0.3-0.5s | `power2`, `power3` |
|
||||
| **High** (promos, sports, launch) | Zoom through, overexposure | Ridged burn, glitch, chromatic split | Staggered blocks, gravity drop | 0.15-0.3s | `power4`, `expo` |
|
||||
|
||||
Pick ONE primary (60-70% of scene changes) plus 1-2 accents. Never use a different transition for every scene.
|
||||
|
||||
### Mood → transition type
|
||||
|
||||
| Mood | Transitions |
|
||||
| ------------------------ | --------------------------------------------------------------------------- |
|
||||
| Warm / inviting | Light leak, blur crossfade, focus pull, film burn · _Shader:_ thermal distortion, cross-warp morph |
|
||||
| Cold / clinical | Squeeze, zoom out, blinds, shutter, grid dissolve · _Shader:_ gravitational lens |
|
||||
| Editorial / magazine | Push slide, vertical push, diagonal split, shutter · _Shader:_ whip pan |
|
||||
| Tech / futuristic | Grid dissolve, staggered blocks, blinds · _Shader:_ glitch, chromatic split |
|
||||
| Tense / edgy | Glitch, VHS, chromatic aberration, ripple · _Shader:_ ridged burn, domain warp |
|
||||
| Playful / fun | Elastic push, 3D flip, circle iris, morph circle · _Shader:_ swirl vortex, ripple waves |
|
||||
| Dramatic / cinematic | Zoom through, gravity drop, overexposure · _Shader:_ cinematic zoom, gravitational lens |
|
||||
| Premium / luxury | Focus pull, blur crossfade, color dip to black · _Shader:_ cross-warp morph |
|
||||
| Retro / analog | Film burn, light leak, VHS, clock wipe · _Shader:_ light leak |
|
||||
|
||||
### Presets
|
||||
|
||||
| Preset | Duration | Easing |
|
||||
| ---------- | -------- | ----------------- |
|
||||
| `snappy` | 0.2s | `power4.inOut` |
|
||||
| `smooth` | 0.4s | `power2.inOut` |
|
||||
| `gentle` | 0.6s | `sine.inOut` |
|
||||
| `dramatic` | 0.5s | `power3.in` → out |
|
||||
| `instant` | 0.15s | `expo.inOut` |
|
||||
| `luxe` | 0.7s | `power1.inOut` |
|
||||
|
||||
### Install a shader transition
|
||||
|
||||
```bash
|
||||
npx hyperframes add flash-through-white
|
||||
npx hyperframes add --list
|
||||
```
|
||||
|
||||
### CSS vs shader
|
||||
|
||||
- **CSS transitions** animate scene containers with opacity, transforms, `clip-path`, and filters. Simpler to set up.
|
||||
- **Shader transitions** composite both scene textures per-pixel on a WebGL canvas — can warp, dissolve, and morph in ways CSS cannot. Import from `@hyperframes/shader-transitions` instead of writing raw GLSL.
|
||||
|
||||
Don't mix CSS and shader transitions in the same composition — once a composition uses shader transitions, the WebGL canvas replaces DOM-based scene switching for every transition.
|
||||
|
||||
### Shader-compatible CSS rules
|
||||
|
||||
Shader transitions capture DOM scenes to WebGL textures via html2canvas. The canvas 2D pipeline doesn't match CSS exactly:
|
||||
|
||||
1. No `transparent` keyword in gradients — use the target color at zero alpha: `rgba(200,117,51,0)` not `transparent`. (Canvas interpolates `transparent` as `rgba(0,0,0,0)` creating dark fringes.)
|
||||
2. No gradient backgrounds on elements thinner than 4px. Use solid `background-color` on thin accent lines.
|
||||
3. No CSS variables (`var()`) on elements visible during capture — html2canvas doesn't reliably resolve custom properties. Use literal color values.
|
||||
4. Mark uncapturable decoratives with `data-no-capture` — they stay on the live DOM but are absent from the shader texture.
|
||||
5. No gradient opacity below 0.15 — renders differently in canvas vs CSS.
|
||||
6. Every `.scene` div must have explicit `background-color`, AND pass the same color as `bgColor` in the `init()` config. Without either, the texture renders as black.
|
||||
|
||||
These rules only apply to shader transition compositions. CSS-only compositions have no restrictions.
|
||||
|
||||
### Don't
|
||||
|
||||
- Mix CSS and shader transitions in one composition.
|
||||
- Use exit animations on any scene except the final scene — the transition IS the exit.
|
||||
- Introduce a new transition type every scene — pick one primary + 1-2 accents.
|
||||
- Use transitions that create visible geometric repetition (grids, hex cells, uniform dots) — they look artificial regardless of the math behind them. Prefer organic noise (FBM, domain warping).
|
||||
@@ -0,0 +1,136 @@
|
||||
# GSAP for HyperFrames
|
||||
|
||||
GSAP is the animation engine for all HyperFrames compositions. Load from CDN inside the composition:
|
||||
|
||||
```html
|
||||
<script src="https://cdn.jsdelivr.net/npm/gsap@3.14.2/dist/gsap.min.js"></script>
|
||||
```
|
||||
|
||||
## Core Tween Methods
|
||||
|
||||
- **`gsap.to(targets, vars)`** — animate from current state to `vars`. Most common.
|
||||
- **`gsap.from(targets, vars)`** — animate from `vars` to current state (entrances).
|
||||
- **`gsap.fromTo(targets, fromVars, toVars)`** — explicit start and end.
|
||||
- **`gsap.set(targets, vars)`** — apply immediately (duration 0). Don't use on clip elements that enter later — use `tl.set(selector, vars, time)` inside the timeline instead.
|
||||
|
||||
Always use **camelCase** property names (`backgroundColor`, `rotationX`, not `background-color`).
|
||||
|
||||
## Common vars
|
||||
|
||||
- **`duration`** — seconds (default 0.5).
|
||||
- **`delay`** — seconds before start.
|
||||
- **`ease`** — `"power1.out"` (default), `"power3.inOut"`, `"back.out(1.7)"`, `"elastic.out(1, 0.3)"`, `"none"`, `"expo.out"`, `"circ.inOut"`.
|
||||
- **`stagger`** — number `0.1` or object: `{ amount: 0.3, from: "center" }`, `{ each: 0.1, from: "random" }`.
|
||||
- **`overwrite`** — `false` (default), `true`, or `"auto"`.
|
||||
- **`repeat`** — number (never `-1` in HyperFrames). **`yoyo`** — alternates direction with repeat.
|
||||
- **`onComplete`**, **`onStart`**, **`onUpdate`** — callbacks.
|
||||
- **`immediateRender`** — default `true` for `from()`/`fromTo()`. Set `false` on later tweens targeting the same property+element to avoid overwrite surprises.
|
||||
|
||||
## Transforms
|
||||
|
||||
Prefer GSAP's transform aliases over raw CSS `transform`:
|
||||
|
||||
| GSAP property | Equivalent |
|
||||
| --------------------------- | -------------------------- |
|
||||
| `x`, `y`, `z` | translateX/Y/Z (px) |
|
||||
| `xPercent`, `yPercent` | translateX/Y (%) |
|
||||
| `scale`, `scaleX`, `scaleY` | scale |
|
||||
| `rotation` | rotate (deg) |
|
||||
| `rotationX`, `rotationY` | 3D rotate |
|
||||
| `skewX`, `skewY` | skew |
|
||||
| `transformOrigin` | transform-origin |
|
||||
|
||||
- **`autoAlpha`** — prefer over `opacity`. At 0, also sets `visibility: hidden`.
|
||||
- **CSS variables** — `"--hue": 180`.
|
||||
- **Directional rotation** — `"360_cw"`, `"-170_short"`, `"90_ccw"`.
|
||||
- **`clearProps`** — `"all"` or comma-separated; removes inline styles on complete.
|
||||
- **Relative values** — `"+=20"`, `"-=10"`, `"*=2"`.
|
||||
|
||||
## Function-based Values
|
||||
|
||||
```js
|
||||
gsap.to(".item", {
|
||||
x: (i, target, targets) => i * 50,
|
||||
stagger: 0.1,
|
||||
});
|
||||
```
|
||||
|
||||
## Easing
|
||||
|
||||
Built-in eases: `power1` through `power4`, `back`, `bounce`, `circ`, `elastic`, `expo`, `sine`. Each has `.in`, `.out`, `.inOut`.
|
||||
|
||||
Rule of thumb:
|
||||
- Entrances: `power3.out`, `expo.out`, `back.out(1.4)`
|
||||
- Exits: `power2.in`, `expo.in`
|
||||
- Scrubbed sections: `none` (linear)
|
||||
- Vary eases across entrance tweens within a scene — at least 3 different eases.
|
||||
|
||||
## Defaults
|
||||
|
||||
```js
|
||||
gsap.defaults({ duration: 0.6, ease: "power2.out" });
|
||||
```
|
||||
|
||||
## Timelines (HyperFrames primary pattern)
|
||||
|
||||
```js
|
||||
window.__timelines = window.__timelines || {};
|
||||
|
||||
const tl = gsap.timeline({ paused: true, defaults: { duration: 0.6, ease: "power2.out" } });
|
||||
|
||||
tl.from(".title", { y: 50, opacity: 0 }, 0.3);
|
||||
tl.from(".subtitle", { y: 30, opacity: 0 }, 0.5);
|
||||
tl.from(".cta", { scale: 0.8, opacity: 0, ease: "back.out(1.7)" }, 0.8);
|
||||
|
||||
window.__timelines["root"] = tl;
|
||||
```
|
||||
|
||||
### Position parameter
|
||||
|
||||
Third argument to `.from()` / `.to()` / `.add()`:
|
||||
|
||||
- Absolute seconds: `0.5`, `2.1`.
|
||||
- Relative to end: `">+0.2"` (0.2s after previous), `"<"` (same time as previous), `"<+0.3"` (0.3s after previous's start).
|
||||
- Named labels: `tl.addLabel("act2", 5); tl.from(".x", { y: 30 }, "act2");`
|
||||
|
||||
### Nesting
|
||||
|
||||
HyperFrames auto-nests sub-composition timelines. **Do not** manually `tl.add(subTl)` — the framework wires sub-timelines into the parent at the sub-composition's `data-start`.
|
||||
|
||||
### Playback
|
||||
|
||||
The player controls playback. Don't call `tl.play()`, `tl.pause()`, or `tl.reverse()` at construction time. `{ paused: true }` is required.
|
||||
|
||||
## Stagger
|
||||
|
||||
```js
|
||||
// even distribution
|
||||
tl.from(".card", { opacity: 0, y: 40, stagger: 0.1 });
|
||||
|
||||
// control total amount
|
||||
tl.from(".card", { opacity: 0, stagger: { amount: 0.6, from: "center" } });
|
||||
|
||||
// deterministic "random" stagger (HyperFrames compositions must be deterministic)
|
||||
tl.from(".dot", { opacity: 0, stagger: { each: 0.05, from: "random" } });
|
||||
```
|
||||
|
||||
`stagger.from`: `"start"` | `"end"` | `"center"` | `"edges"` | `"random"` | index | `[x, y]` for grid.
|
||||
|
||||
## Performance
|
||||
|
||||
- Animate transforms (`x`, `y`, `scale`, `rotation`, `opacity`) — cheap, GPU-accelerated.
|
||||
- Avoid animating `width`, `height`, `top`, `left`, `margin` — causes layout thrash.
|
||||
- Avoid box-shadow or filter animations on large elements — expensive.
|
||||
- `will-change` is rarely needed; GSAP handles promotion.
|
||||
|
||||
## gsap.matchMedia (rarely needed in HyperFrames)
|
||||
|
||||
Compositions have fixed dimensions (`data-width`/`data-height`), so responsive breakpoints don't apply. You may still use `matchMedia` for `prefers-reduced-motion` when authoring UI previews, but it's not used in rendered video output.
|
||||
|
||||
## Don't Do
|
||||
|
||||
- `repeat: -1` anywhere — breaks the capture engine.
|
||||
- `Math.random()`, `Date.now()`, performance.now()` inside tween values — non-deterministic.
|
||||
- `async` / `setTimeout` / `Promise` around timeline construction — the capture engine reads `window.__timelines` synchronously.
|
||||
- Animate `visibility` or `display` directly — use `autoAlpha`.
|
||||
- `gsap.set()` on clip elements that enter later in the timeline — they don't exist in the DOM at page-load. Use `tl.set(sel, vars, time)` inside the timeline.
|
||||
@@ -0,0 +1,137 @@
|
||||
# Troubleshooting
|
||||
|
||||
## `HeadlessExperimental.beginFrame' wasn't found` (first thing to check)
|
||||
|
||||
**Symptom:** `npx hyperframes render` fails with:
|
||||
|
||||
```
|
||||
✗ Render failed
|
||||
Protocol error (HeadlessExperimental.beginFrame):
|
||||
'HeadlessExperimental.beginFrame' wasn't found
|
||||
```
|
||||
|
||||
**Cause:** Chromium 147+ removed the `HeadlessExperimental.beginFrame` CDP command. This affected sandbox environments (e.g., OpenClaw, some containerized agent hosts) that ship modern Chromium as the system browser. See [hyperframes#294](https://github.com/heygen-com/hyperframes/issues/294).
|
||||
|
||||
**Fix (permanent — preferred):** upgrade.
|
||||
|
||||
```bash
|
||||
npx hyperframes upgrade -y
|
||||
# or
|
||||
npm install -g hyperframes@latest
|
||||
```
|
||||
|
||||
`hyperframes >= 0.4.2` auto-detects whether the resolved browser supports `beginFrame` (checks for `chrome-headless-shell` in the binary path) and falls back to screenshot capture mode when it doesn't. Commit [`4c72ba4`](https://github.com/heygen-com/hyperframes/commit/4c72ba4a36ec2bd6733f7b9cb2a9e63f9fb234b9) (March 2026) shipped this auto-detect.
|
||||
|
||||
**Fix (escape hatch — if you can't upgrade):**
|
||||
|
||||
```bash
|
||||
export PRODUCER_FORCE_SCREENSHOT=true
|
||||
npx hyperframes render
|
||||
```
|
||||
|
||||
This forces screenshot mode regardless of the binary. Screenshot mode is slightly slower but visually identical.
|
||||
|
||||
**Fix (prevent — recommended):** install `chrome-headless-shell` so the engine can use the fast BeginFrame path:
|
||||
|
||||
```bash
|
||||
npx puppeteer browsers install chrome-headless-shell
|
||||
# or let the CLI do it
|
||||
npx hyperframes browser --install
|
||||
```
|
||||
|
||||
`scripts/setup.sh` runs this automatically.
|
||||
|
||||
## `npx hyperframes render` hangs for 120s then times out
|
||||
|
||||
**Cause:** the resolved browser is system Chrome (e.g., `/usr/bin/google-chrome`) and doesn't support the BeginFrame path, but auto-detect also missed it (older `hyperframes` version).
|
||||
|
||||
**Fix:**
|
||||
1. Check which binary is being used: `npx hyperframes browser --path`
|
||||
2. If it's system Chrome, either:
|
||||
- Install `chrome-headless-shell`: `npx hyperframes browser --install`, OR
|
||||
- Set the escape hatch: `export PRODUCER_FORCE_SCREENSHOT=true`, OR
|
||||
- Upgrade: `npx hyperframes upgrade -y`
|
||||
|
||||
## `ffmpeg: command not found`
|
||||
|
||||
Install FFmpeg via your system package manager:
|
||||
|
||||
| OS / distro | Command |
|
||||
| --------------- | ----------------------------------- |
|
||||
| Ubuntu / Debian | `sudo apt-get install -y ffmpeg` |
|
||||
| Fedora / RHEL | `sudo dnf install -y ffmpeg` |
|
||||
| Arch | `sudo pacman -S ffmpeg` |
|
||||
| macOS | `brew install ffmpeg` |
|
||||
| Windows | `winget install Gyan.FFmpeg` |
|
||||
|
||||
Verify: `ffmpeg -version`.
|
||||
|
||||
## `Node version X is not supported`
|
||||
|
||||
HyperFrames requires Node.js >= 22. Check with `node --version`.
|
||||
|
||||
- **nvm:** `nvm install 22 && nvm use 22`
|
||||
- **Homebrew (macOS):** `brew install node@22 && brew link --overwrite node@22`
|
||||
- **apt:** follow [nodesource](https://github.com/nodesource/distributions) for Node 22 LTS.
|
||||
|
||||
## `ENOSPC: no space left on device` or OOM kills during render
|
||||
|
||||
Renders are memory- and disk-hungry. Minimums:
|
||||
|
||||
- **RAM:** 4 GB free (8 GB recommended for 60fps / `--quality high`)
|
||||
- **Disk:** 2 GB free scratch space — frames are written to `/tmp` during capture
|
||||
|
||||
Mitigations:
|
||||
- Lower quality: `--quality draft`.
|
||||
- Lower fps: `--fps 24`.
|
||||
- Lower worker count: `--workers 1`.
|
||||
- Set `TMPDIR` to a volume with more space: `export TMPDIR=/mnt/scratch`.
|
||||
|
||||
## Lint passes but the render is blank / black frames
|
||||
|
||||
Check the browser console in `preview` — usually:
|
||||
- A timeline was registered with the wrong key (`__timelines["typo"]` instead of `__timelines["root"]`).
|
||||
- The root composition was wrapped in `<template>` (only sub-compositions use `<template>`).
|
||||
- A script tag failed to load — check Network tab in preview.
|
||||
|
||||
Run `npx hyperframes lint --verbose` to see info-level findings.
|
||||
|
||||
## Contrast warnings from `hyperframes validate`
|
||||
|
||||
```
|
||||
⚠ WCAG AA contrast warnings (3):
|
||||
· .subtitle "secondary text" — 2.67:1 (need 4.5:1, t=5.3s)
|
||||
```
|
||||
|
||||
- **Dark backgrounds:** brighten the failing color until it clears 4.5:1 (normal text) or 3:1 (large text — 24px+ or 19px+ bold).
|
||||
- **Light backgrounds:** darken it.
|
||||
- Stay within the palette family — don't invent a new color, adjust the existing one.
|
||||
- Skip the check temporarily with `--no-contrast` if iterating rapidly, but clear it before delivery.
|
||||
|
||||
## `Font family 'X' not supported by compiler`
|
||||
|
||||
The compiler embeds a curated set of web-safe + open-source fonts. If a font isn't supported, either:
|
||||
- Swap to a supported alternative from the warning.
|
||||
- Register a custom font via `@font-face` pointing to a `.woff2` in the project directory (the compiler embeds referenced `@font-face` files).
|
||||
|
||||
## Video plays back muted or with no audio
|
||||
|
||||
Check:
|
||||
- The `<video>` element has `muted playsinline` (required — browser autoplay policy).
|
||||
- Audio is a **separate** `<audio>` element, not the video element.
|
||||
- Audio `data-volume` is set (defaults to 1).
|
||||
- The audio file is at the expected path — compositions load relative to their own directory.
|
||||
|
||||
## Docker render fails on Linux with rootless Docker
|
||||
|
||||
Add `--privileged` or pass `--cap-add=SYS_ADMIN`:
|
||||
|
||||
```bash
|
||||
npx hyperframes render --docker --docker-args "--cap-add=SYS_ADMIN"
|
||||
```
|
||||
|
||||
The headless browser needs namespace permissions for sandboxing.
|
||||
|
||||
## Bug reports
|
||||
|
||||
Include `npx hyperframes info` output + the full error log. File at [github.com/heygen-com/hyperframes](https://github.com/heygen-com/hyperframes/issues).
|
||||
@@ -0,0 +1,145 @@
|
||||
# Website to Video
|
||||
|
||||
Capture a website, produce a professional video from it. Use when the user provides a URL and wants a video — social ad, product tour, 30-second promo, etc.
|
||||
|
||||
The workflow has 7 steps. Each produces an artifact that gates the next. **Do not skip steps** — each artifact prevents a downstream failure mode.
|
||||
|
||||
## Step 1: Capture & Understand
|
||||
|
||||
```bash
|
||||
npx hyperframes capture https://example.com -o example-video
|
||||
```
|
||||
|
||||
Produces `example-video/capture/` with:
|
||||
- `capture/screenshots/` — above-the-fold + section screenshots (up to `--max-screenshots`)
|
||||
- `capture/assets/` — logos, hero images, background video (if any)
|
||||
- `capture/extracted/tokens.json` — colors, fonts, and spacing tokens
|
||||
- `capture/extracted/visible-text.txt` — extracted headings, paragraphs, CTAs
|
||||
- `capture/extracted/fonts.json` — font families and stacks detected in computed styles
|
||||
- `capture/asset-descriptions.md` — auto-generated asset catalog
|
||||
|
||||
All subsequent steps read from the `capture/` subfolder — `capture/extracted/tokens.json`, `capture/assets/hero.png`, etc. Never strip the `capture/` prefix when referencing these files.
|
||||
|
||||
**Gate:** Print a site summary — name, top 3 colors, primary + display fonts, hero asset path, one-sentence vibe. Keep it in your context — don't re-capture.
|
||||
|
||||
## Step 2: Write DESIGN.md
|
||||
|
||||
Small brand reference at the project root. 6 sections, ~90 lines. This is the cheat sheet — not the creative plan.
|
||||
|
||||
```markdown
|
||||
# DESIGN
|
||||
|
||||
## Brand
|
||||
- Name: Example Co.
|
||||
- One-line mission: "…"
|
||||
|
||||
## Colors
|
||||
- Background: #0B0F14
|
||||
- Primary: #00E0A4 (accent, CTA)
|
||||
- Secondary: #7A8B9B (body text)
|
||||
- Text: #FFFFFF
|
||||
|
||||
## Typography
|
||||
- Display: "Inter Tight", 700, tight letter-spacing
|
||||
- Body: "Inter", 400
|
||||
|
||||
## Motion
|
||||
- Mood: precise, technical, confident
|
||||
- Eases: `power3.out` for entrances, `expo.in` for exits
|
||||
|
||||
## Assets
|
||||
- Logo: `capture/assets/logo.svg`
|
||||
- Hero image: `capture/assets/hero.png`
|
||||
|
||||
## What NOT to Do
|
||||
- No purple, no pastels, no serif body
|
||||
- No playful/bubbly eases (`elastic`, `bounce`)
|
||||
- No drop shadows on text
|
||||
```
|
||||
|
||||
**Gate:** `DESIGN.md` exists in the project directory.
|
||||
|
||||
## Step 3: Write SCRIPT.md
|
||||
|
||||
Narration script. Story backbone. **Scene durations come from the narration, not from guessing.**
|
||||
|
||||
```markdown
|
||||
# SCRIPT
|
||||
|
||||
## Scene 1 — Hook (0:00–0:04)
|
||||
"What if your dashboards wrote themselves?"
|
||||
|
||||
## Scene 2 — Problem (0:04–0:11)
|
||||
"Teams spend hours stitching together queries, charts, and callouts — every Monday."
|
||||
|
||||
## Scene 3 — Solution (0:11–0:22)
|
||||
"Example Co. watches your data streams and proposes the dashboard you'd have built — in seconds."
|
||||
|
||||
## Scene 4 — CTA (0:22–0:28)
|
||||
"Try it free at example.com."
|
||||
```
|
||||
|
||||
Run `npx hyperframes tts SCRIPT.md --voice af_nova --output narration.wav` to generate TTS audio. Note the exact duration — that's the video's duration.
|
||||
|
||||
**Gate:** `SCRIPT.md` + `narration.wav` exist and durations match the plan (±0.3s).
|
||||
|
||||
## Step 4: Storyboard
|
||||
|
||||
Text-only scene plan: for each scene, describe the hero frame — what's on screen at the scene's most-visible moment.
|
||||
|
||||
```markdown
|
||||
# STORYBOARD
|
||||
|
||||
## Scene 1 (0:00–0:04) — Hook
|
||||
Hero frame: giant "WHAT IF YOUR DASHBOARDS WROTE THEMSELVES?" in display font, centered, on near-black. Logo top-left at 40% opacity.
|
||||
Entrance: each word staggers in, 0.08s apart.
|
||||
Transition out: flash-through-white into Scene 2.
|
||||
```
|
||||
|
||||
One paragraph per scene. Do NOT skip this step — it's where you catch narrative gaps before writing HTML.
|
||||
|
||||
**Gate:** `STORYBOARD.md` exists. Each scene has: hero frame, entrance, transition.
|
||||
|
||||
## Step 5: Composition
|
||||
|
||||
Write `index.html` scene-by-scene:
|
||||
- Each scene is a `<div class="scene scene-N">` positioned absolutely, full-bleed.
|
||||
- Static HTML+CSS for the hero frame first (no GSAP).
|
||||
- Layer the narration `<audio>` at `data-start="0"` on a high track index.
|
||||
- Add a transitions component (`flash-through-white`, `liquid-wipe`, etc.) between each scene.
|
||||
- THEN add GSAP entrances (`gsap.from()`), no exits — transitions own the exit.
|
||||
- Register `window.__timelines["root"] = tl`.
|
||||
|
||||
Install transitions as needed:
|
||||
|
||||
```bash
|
||||
npx hyperframes add flash-through-white
|
||||
```
|
||||
|
||||
## Step 6: Render
|
||||
|
||||
```bash
|
||||
npx hyperframes lint --strict # must pass
|
||||
npx hyperframes validate # WCAG contrast audit
|
||||
npx hyperframes render --quality draft --output draft.mp4
|
||||
```
|
||||
|
||||
Watch the draft. Note issues in a `REVIEW.md` bullet list (scene, timestamp, issue). Fix, re-render.
|
||||
|
||||
When happy:
|
||||
|
||||
```bash
|
||||
npx hyperframes render --quality high --output final.mp4
|
||||
```
|
||||
|
||||
## Step 7: Deliver
|
||||
|
||||
- Report file path + duration + file size to the user.
|
||||
- If the user wants a vertical cut, re-render with a 9:16 composition (`data-width="1080" data-height="1920"`) — typically requires a separate `index-vertical.html` with tighter typography and re-stacked scene layout.
|
||||
|
||||
## Common Failure Modes
|
||||
|
||||
- **Skipped DESIGN.md** → colors drift scene-to-scene; output feels like "AI slides."
|
||||
- **Skipped STORYBOARD.md** → scenes overlap or hero frames collide with transitions.
|
||||
- **Exit animations** before transitions → empty frames when the transition fires.
|
||||
- **Narration longer than `data-duration`** → audio clips mid-sentence. Update the composition's `data-duration` to match the TTS output length + 0.5s buffer.
|
||||
+135
@@ -0,0 +1,135 @@
|
||||
#!/usr/bin/env bash
|
||||
# HyperFrames setup for Hermes.
|
||||
#
|
||||
# Verifies Node >= 22 and FFmpeg, installs the `hyperframes` CLI globally,
|
||||
# pre-caches `chrome-headless-shell`, and runs `hyperframes doctor`.
|
||||
#
|
||||
# Pins `hyperframes@>=0.4.2` so the OpenClaw/Chromium-147 fix from
|
||||
# https://github.com/heygen-com/hyperframes/issues/294 (commit 4c72ba4)
|
||||
# is always present — the engine auto-detects `HeadlessExperimental.beginFrame`
|
||||
# support and falls back to screenshot capture otherwise.
|
||||
#
|
||||
# Idempotent: safe to re-run.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
MIN_NODE_MAJOR=22
|
||||
MIN_HYPERFRAMES_VERSION="0.4.2"
|
||||
|
||||
red() { printf '\033[31m%s\033[0m\n' "$*"; }
|
||||
green() { printf '\033[32m%s\033[0m\n' "$*"; }
|
||||
yellow() { printf '\033[33m%s\033[0m\n' "$*"; }
|
||||
bold() { printf '\033[1m%s\033[0m\n' "$*"; }
|
||||
|
||||
bold "==> HyperFrames setup"
|
||||
|
||||
# --- 1. Node.js --------------------------------------------------------------
|
||||
|
||||
if ! command -v node >/dev/null 2>&1; then
|
||||
red "✗ Node.js is not installed."
|
||||
echo " Install Node.js >= ${MIN_NODE_MAJOR} (nvm, Homebrew, or your package manager) and re-run."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
node_version="$(node --version | sed 's/^v//')"
|
||||
node_major="$(echo "$node_version" | cut -d. -f1)"
|
||||
if [ "$node_major" -lt "$MIN_NODE_MAJOR" ]; then
|
||||
red "✗ Node.js ${node_version} is too old. HyperFrames requires Node.js >= ${MIN_NODE_MAJOR}."
|
||||
echo " Upgrade with 'nvm install ${MIN_NODE_MAJOR} && nvm use ${MIN_NODE_MAJOR}' or your package manager."
|
||||
exit 1
|
||||
fi
|
||||
green "✓ Node.js ${node_version}"
|
||||
|
||||
# --- 2. FFmpeg ---------------------------------------------------------------
|
||||
|
||||
if ! command -v ffmpeg >/dev/null 2>&1; then
|
||||
red "✗ FFmpeg is not installed."
|
||||
case "$(uname -s)" in
|
||||
Linux*) echo " sudo apt-get install -y ffmpeg # Debian/Ubuntu"
|
||||
echo " sudo dnf install -y ffmpeg # Fedora/RHEL";;
|
||||
Darwin*) echo " brew install ffmpeg";;
|
||||
MINGW*|MSYS*|CYGWIN*) echo " winget install Gyan.FFmpeg";;
|
||||
*) echo " See https://ffmpeg.org/download.html";;
|
||||
esac
|
||||
exit 1
|
||||
fi
|
||||
green "✓ FFmpeg $(ffmpeg -version 2>&1 | head -1 | awk '{print $3}')"
|
||||
|
||||
# --- 3. npm ------------------------------------------------------------------
|
||||
|
||||
if ! command -v npm >/dev/null 2>&1; then
|
||||
red "✗ npm is not installed (should ship with Node.js)."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- 4. Install / upgrade hyperframes CLI -----------------------------------
|
||||
|
||||
bold "==> Installing hyperframes CLI (>= ${MIN_HYPERFRAMES_VERSION})"
|
||||
|
||||
current_hyperframes=""
|
||||
if command -v hyperframes >/dev/null 2>&1; then
|
||||
current_hyperframes="$(hyperframes --version 2>/dev/null | tail -1 | sed 's/^v//')"
|
||||
fi
|
||||
|
||||
if [ -n "$current_hyperframes" ]; then
|
||||
yellow " Found hyperframes ${current_hyperframes}"
|
||||
fi
|
||||
|
||||
# Always install/upgrade to >= MIN version.
|
||||
# Using 'latest' so we pick up any newer auto-detect/capture fixes.
|
||||
if ! npm install -g "hyperframes@latest" >/dev/null 2>&1; then
|
||||
red "✗ npm install -g hyperframes@latest failed."
|
||||
echo " Try: sudo npm install -g hyperframes@latest"
|
||||
echo " Or use a user-scoped npm prefix: npm config set prefix ~/.npm-global && export PATH=\"\$HOME/.npm-global/bin:\$PATH\""
|
||||
exit 1
|
||||
fi
|
||||
|
||||
installed_version="$(hyperframes --version 2>/dev/null | tail -1 | sed 's/^v//')"
|
||||
green "✓ hyperframes ${installed_version} installed globally"
|
||||
|
||||
# Sanity-check minimum version.
|
||||
version_ge() {
|
||||
# version_ge A B → true if A >= B
|
||||
[ "$(printf '%s\n%s\n' "$1" "$2" | sort -V | head -1)" = "$2" ]
|
||||
}
|
||||
if ! version_ge "$installed_version" "$MIN_HYPERFRAMES_VERSION"; then
|
||||
red "✗ hyperframes ${installed_version} is below required minimum ${MIN_HYPERFRAMES_VERSION}."
|
||||
echo " Try 'npm install -g hyperframes@latest' or 'sudo npm install -g hyperframes@latest'."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# --- 5. Pre-cache chrome-headless-shell --------------------------------------
|
||||
#
|
||||
# Chromium 147+ removed HeadlessExperimental.beginFrame. System Chrome (e.g.
|
||||
# /usr/bin/google-chrome) can't render with the fast path, so the engine
|
||||
# auto-detects and falls back to screenshot mode — but BeginFrame mode is
|
||||
# faster and produces higher-quality output. Install chrome-headless-shell
|
||||
# up front so the engine picks it over system Chrome.
|
||||
|
||||
bold "==> Pre-caching chrome-headless-shell (for best render quality)"
|
||||
|
||||
if ! npx --yes puppeteer browsers install chrome-headless-shell >/dev/null 2>&1; then
|
||||
yellow "⚠ Could not pre-install chrome-headless-shell."
|
||||
yellow " Rendering will still work via screenshot-mode fallback (slower)."
|
||||
yellow " If you hit HeadlessExperimental.beginFrame errors:"
|
||||
yellow " export PRODUCER_FORCE_SCREENSHOT=true"
|
||||
yellow " See references/troubleshooting.md."
|
||||
else
|
||||
green "✓ chrome-headless-shell installed"
|
||||
fi
|
||||
|
||||
# --- 6. Doctor ---------------------------------------------------------------
|
||||
|
||||
bold "==> Running hyperframes doctor"
|
||||
|
||||
if hyperframes doctor; then
|
||||
green "✓ HyperFrames is ready"
|
||||
echo
|
||||
echo " Scaffold a project: npx hyperframes init my-video"
|
||||
echo " Preview: npx hyperframes preview"
|
||||
echo " Render: npx hyperframes render"
|
||||
else
|
||||
yellow "⚠ hyperframes doctor reported issues."
|
||||
yellow " See references/troubleshooting.md or re-run 'hyperframes doctor'."
|
||||
exit 1
|
||||
fi
|
||||
@@ -0,0 +1,207 @@
|
||||
---
|
||||
name: kanban-video-orchestrator
|
||||
description: Plan, set up, and monitor a multi-agent video production pipeline backed by Hermes Kanban. Use when the user wants to make ANY video — narrative film, product/marketing, music video, explainer, ASCII/terminal art, abstract/generative loop, comic, 3D, real-time/installation — and the work warrants decomposition into specialized profiles (writer, designer, animator, renderer, voice, editor, etc.) coordinated through a kanban board. Performs adaptive discovery to scope the brief, designs an appropriate team for the requested style, generates the setup script that creates Hermes profiles + initial kanban task, then helps monitor execution and intervene when tasks stall or fail. Routes scenes to whichever Hermes rendering / audio / design skill fits each beat (`ascii-video`, `manim-video`, `p5js`, `comfyui`, `touchdesigner-mcp`, `blender-mcp`, `pixel-art`, `baoyu-comic`, `claude-design`, `excalidraw`, `songsee`, `heartmula`, …) plus external APIs for TTS, image-gen, and image-to-video as needed.
|
||||
version: 1.0.0
|
||||
author: [SHL0MS, alt-glitch]
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [video, kanban, multi-agent, orchestration, production-pipeline]
|
||||
related_skills: [kanban-orchestrator, kanban-worker, ascii-video, manim-video, p5js, comfyui, touchdesigner-mcp, blender-mcp, pixel-art, ascii-art, songwriting-and-ai-music, heartmula, songsee, spotify, youtube-content, claude-design, excalidraw, architecture-diagram, concept-diagrams, baoyu-comic, baoyu-infographic, humanizer, gif-search, meme-generation]
|
||||
credits: |
|
||||
The single-project workspace layout, profile-config patching pattern,
|
||||
SOUL.md-per-profile model, TEAM.md task-graph convention, and
|
||||
`--workspace dir:<path>` discipline are adapted from alt-glitch's
|
||||
original multi-agent video pipeline at
|
||||
https://github.com/NousResearch/kanban-video-pipeline.
|
||||
---
|
||||
|
||||
# Kanban Video Orchestrator
|
||||
|
||||
Wrap any video request — from a 15-second product teaser to a 5-minute narrative
|
||||
short to a music video to an ASCII loop — in a Hermes Kanban pipeline that
|
||||
decomposes the work to specialized agent profiles.
|
||||
|
||||
This skill does **not** render anything itself. It is a meta-pipeline that:
|
||||
|
||||
1. **Scopes** the request through targeted discovery
|
||||
2. **Designs** an appropriate team (which roles, which tools per role) based on the style
|
||||
3. **Generates** a setup script that creates Hermes profiles, project workspace, and the initial kanban task
|
||||
4. **Hands off** to the director profile, which decomposes via the kanban
|
||||
5. **Monitors** execution, helps intervene when tasks stall or fail
|
||||
|
||||
The actual rendering happens inside the kanban once it's running, via whichever
|
||||
existing skills + tools fit the scenes — `ascii-video`, `manim-video`, `p5js`,
|
||||
`comfyui`, `touchdesigner-mcp`, `blender-mcp`, `songwriting-and-ai-music`,
|
||||
`heartmula`, external APIs, or plain Python with PIL + ffmpeg.
|
||||
|
||||
## When NOT to use this skill
|
||||
|
||||
- The video is one continuous procedural project that needs no specialists. Just write the code directly.
|
||||
- The user wants a quick one-shot conversion (e.g. "convert this mp4 to a GIF") — use ffmpeg directly.
|
||||
- The output is a static image, GIF, or audio-only artifact — use the matching specific skill (`ascii-art`, `gifs`, `meme-generation`, `songwriting-and-ai-music`).
|
||||
- The work fits a single existing skill cleanly (e.g. a pure ASCII video — just use `ascii-video`).
|
||||
|
||||
## Workflow
|
||||
|
||||
```
|
||||
DISCOVER → BRIEF → TEAM DESIGN → SETUP → EXECUTE → MONITOR
|
||||
```
|
||||
|
||||
### Step 1 — Discover (ask the right questions)
|
||||
|
||||
The discovery process is **adaptive**: ask only what is actually needed. Always
|
||||
start with three questions to identify the broad shape:
|
||||
|
||||
- **What is the video?** (one-sentence brief)
|
||||
- **How long?** (5-30s teaser / 30-90s short / 90s-3min explainer / 3-10min film / longer)
|
||||
- **What aspect ratio + target platform?** (1:1 / 9:16 / 16:9; X, IG, YouTube, internal, etc.)
|
||||
|
||||
From the answer, classify the style category. The style determines which
|
||||
follow-up questions to ask. **Do not ask all questions at once.** Ask 2-4 at a
|
||||
time, listen, then proceed. Make reasonable assumptions whenever the user
|
||||
implies an answer.
|
||||
|
||||
For complete intake patterns and per-style question banks, see
|
||||
**[references/intake.md](references/intake.md)**.
|
||||
|
||||
### Step 2 — Brief
|
||||
|
||||
Once enough is known, produce a structured `brief.md` using the template in
|
||||
`assets/brief.md.tmpl`. Stages:
|
||||
|
||||
1. **Concept** — the one-sentence pitch + emotional north star
|
||||
2. **Scope** — duration, aspect, platform, deadline
|
||||
3. **Style** — visual references, brand constraints, tone
|
||||
4. **Scenes** — beat-by-beat breakdown (durations, content, target tool)
|
||||
5. **Audio** — narration / music / SFX / silent (per scene if needed)
|
||||
6. **Deliverables** — file format, resolution, optional alternates (vertical cut, GIF, etc.)
|
||||
|
||||
Show the brief to the user for confirmation before designing the team. **The
|
||||
brief is the contract** — every downstream task references it.
|
||||
|
||||
### Step 3 — Team design
|
||||
|
||||
Pick role archetypes from the library that fit this video. **Compose, don't
|
||||
clone.** Most videos need 4-7 profiles. The director is always present; the
|
||||
rest are picked by what the brief actually requires.
|
||||
|
||||
For the role library and per-style team compositions, see
|
||||
**[references/role-archetypes.md](references/role-archetypes.md)**.
|
||||
|
||||
For mapping role → which Hermes skills + toolsets it loads, see
|
||||
**[references/tool-matrix.md](references/tool-matrix.md)**.
|
||||
|
||||
### Step 4 — Setup
|
||||
|
||||
Generate a setup script (`setup.sh`) and run it. The script:
|
||||
|
||||
1. Creates the project workspace (`~/projects/video-pipeline/<slug>/`)
|
||||
2. Copies any provided assets into `taste/`, `audio/`, `assets/`
|
||||
3. Creates each Hermes profile via `hermes profile create --clone`
|
||||
4. Writes per-profile `SOUL.md` (personality + role definition)
|
||||
5. Configures profile YAML (toolsets, always_load skills, cwd)
|
||||
6. Writes `brief.md`, `TEAM.md`, and `taste/` content
|
||||
7. Fires the initial `hermes kanban create` task assigned to the director
|
||||
|
||||
Use `scripts/bootstrap_pipeline.py` to generate setup.sh from a brief +
|
||||
team-design JSON. See **[references/kanban-setup.md](references/kanban-setup.md)**
|
||||
for the setup script structure, profile config patterns, and the critical
|
||||
"shared workspace" rule.
|
||||
|
||||
### Step 5 — Execute
|
||||
|
||||
Run `setup.sh`. Then provide the user with monitoring commands:
|
||||
|
||||
```bash
|
||||
hermes kanban watch --tenant <project-tenant> # live events
|
||||
hermes kanban list --tenant <project-tenant> # board snapshot
|
||||
hermes dashboard # visual board UI
|
||||
```
|
||||
|
||||
The director profile takes over from here, decomposing the work and routing
|
||||
tasks to specialist profiles via the kanban toolset.
|
||||
|
||||
### Step 6 — Monitor and intervene
|
||||
|
||||
Stay engaged — the kanban runs autonomously but a stuck task or bad output
|
||||
needs human (or AI) judgment.
|
||||
|
||||
Monitoring patterns: poll `kanban list` periodically, inspect any RUNNING task
|
||||
that exceeds its expected duration with `kanban show <id>`, and check
|
||||
heartbeats. When a worker's output fails review, the standard interventions are:
|
||||
|
||||
1. Comment on the worker's task with specific feedback (`kanban_comment`)
|
||||
2. Create a re-run task with the original as parent
|
||||
3. Adjust the brief's scope and let the director re-decompose
|
||||
|
||||
For diagnostic patterns, intervention recipes, and the "task is stuck"
|
||||
playbook, see **[references/monitoring.md](references/monitoring.md)**.
|
||||
|
||||
## Reference: worked examples
|
||||
|
||||
Six concrete pipelines covering very different video styles — narrative film,
|
||||
product/marketing, music video, math/algorithm explainer, ASCII video, real-time
|
||||
installation — showing how the same workflow yields very different teams and
|
||||
task graphs. See **[references/examples.md](references/examples.md)**.
|
||||
|
||||
## Critical rules
|
||||
|
||||
1. **Discovery before action.** Never start generating a brief or team without
|
||||
asking at least the three baseline questions. A bad brief cascades through
|
||||
the entire pipeline.
|
||||
|
||||
2. **Match the team to the video.** Don't reuse the same 4-profile setup for
|
||||
every job. A music video that doesn't have a beat-analysis profile will
|
||||
misfire. A narrative film that doesn't have a writer profile will produce
|
||||
incoherent scenes. See `references/role-archetypes.md`.
|
||||
|
||||
3. **One workspace per project.** All profiles for a given video share the same
|
||||
`dir:` workspace. Tasks pass artifacts via shared filesystem and structured
|
||||
handoffs. **Every** `kanban_create` call passes
|
||||
`workspace_kind="dir"` + `workspace_path="<absolute project path>"`.
|
||||
|
||||
4. **Tenant every project.** Use a project-specific tenant
|
||||
(`--tenant <project-slug>`). Keeps the dashboard scoped and prevents
|
||||
cross-pollination with other ongoing kanbans.
|
||||
|
||||
5. **Respect existing skills.** When a scene fits an existing skill, the
|
||||
relevant renderer should load that skill via `--skill <name>` on its task
|
||||
or `always_load` in its profile. Do not re-derive what a skill already
|
||||
provides.
|
||||
|
||||
6. **The director never executes.** Even with the full `kanban + terminal +
|
||||
file` toolset, the director's `SOUL.md` rules forbid it from executing
|
||||
work itself. It decomposes and routes only — every concrete task becomes
|
||||
a `hermes kanban create` call to a specialist profile. The
|
||||
`kanban-orchestrator` skill spells this out further.
|
||||
|
||||
7. **Don't over-decompose.** A 30-second product video does NOT need 20 tasks.
|
||||
Aim for the smallest task graph that still parallelizes well and exposes the
|
||||
right human-review gates.
|
||||
|
||||
8. **Verify API keys BEFORE firing.** External APIs (TTS, image-gen,
|
||||
image-to-video) need keys in `~/.hermes/.env` or the user's secret store.
|
||||
A worker that hits a missing-key error wastes a task slot. The setup
|
||||
script's `check_key` helper aborts cleanly if a required key is missing.
|
||||
|
||||
## File map
|
||||
|
||||
```
|
||||
SKILL.md ← this file (workflow + rules)
|
||||
references/
|
||||
intake.md ← discovery question banks per style
|
||||
role-archetypes.md ← role library (writer, designer, animator, …)
|
||||
tool-matrix.md ← skill + toolset mapping per role
|
||||
kanban-setup.md ← setup script structure & profile config
|
||||
monitoring.md ← watch + intervene patterns
|
||||
examples.md ← six worked pipelines
|
||||
assets/
|
||||
brief.md.tmpl ← brief skeleton
|
||||
setup.sh.tmpl ← setup script skeleton
|
||||
soul.md.tmpl ← profile personality skeleton
|
||||
scripts/
|
||||
bootstrap_pipeline.py ← generate setup.sh from brief + team JSON
|
||||
monitor.py ← polling + intervention helpers
|
||||
```
|
||||
@@ -0,0 +1,79 @@
|
||||
# Video Brief — {{TITLE}}
|
||||
|
||||
> Slug: `{{SLUG}}` · Tenant: `{{TENANT}}` · Project workspace: `{{WORKSPACE}}`
|
||||
|
||||
## 1. Concept
|
||||
|
||||
**One-line pitch.** {{ONE_LINE_PITCH}}
|
||||
|
||||
**Emotional north star.** {{EMOTIONAL_NORTH_STAR}}
|
||||
*(What should the viewer feel walking away?)*
|
||||
|
||||
## 2. Scope
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| Duration | {{DURATION_S}} seconds |
|
||||
| Aspect ratio | {{ASPECT}} |
|
||||
| Resolution | {{RESOLUTION}} |
|
||||
| Frame rate | {{FPS}} fps |
|
||||
| Target platforms | {{PLATFORMS}} |
|
||||
| Deadline | {{DEADLINE}} |
|
||||
| Quality bar | {{QUALITY_BAR}} *(rough draft / polished / archival)* |
|
||||
|
||||
## 3. Style
|
||||
|
||||
**Visual references.** {{VISUAL_REFS}}
|
||||
|
||||
**Tone.** {{TONE}}
|
||||
|
||||
**Brand constraints.** {{BRAND_CONSTRAINTS}}
|
||||
*(colors, typography, motion language; or "n/a")*
|
||||
|
||||
**Aesthetic rules.**
|
||||
{{AESTHETIC_RULES}}
|
||||
|
||||
## 4. Scenes
|
||||
|
||||
Beat-by-beat breakdown. Each scene gets a row.
|
||||
|
||||
| # | Time | Content | Target tool / skill | Audio | Notes |
|
||||
|---|------|---------|---------------------|-------|-------|
|
||||
| 1 | 0:00–0:0X | {{SCENE_1_CONTENT}} | {{SCENE_1_TOOL}} | {{SCENE_1_AUDIO}} | {{SCENE_1_NOTES}} |
|
||||
| 2 | 0:0X–0:0Y | ... | ... | ... | ... |
|
||||
|
||||
## 5. Audio
|
||||
|
||||
**Approach.** {{AUDIO_APPROACH}}
|
||||
*(narration / music-only / synced to track / silent / mixed)*
|
||||
|
||||
**Voiceover.** {{VO_DETAILS}}
|
||||
*(provider, voice, language, script source — "n/a" if no VO)*
|
||||
|
||||
**Music.** {{MUSIC_DETAILS}}
|
||||
*(provided track path / commission via Suno / commission via heartmula /
|
||||
license-free / "n/a")*
|
||||
|
||||
**SFX.** {{SFX_DETAILS}}
|
||||
*(generated, library, or "n/a")*
|
||||
|
||||
## 6. Deliverables
|
||||
|
||||
| Format | Resolution | Notes |
|
||||
|--------|-----------|-------|
|
||||
| {{PRIMARY_FORMAT}} | {{PRIMARY_RES}} | The main output |
|
||||
| {{ALT_FORMAT_1}} | {{ALT_RES_1}} | {{ALT_NOTES_1}} |
|
||||
|
||||
**Final filename.** `output/final.mp4`
|
||||
*(plus optional `output/final-9x16.mp4`, `output/captions.srt`, etc.)*
|
||||
|
||||
## 7. Constraints
|
||||
|
||||
- API keys required: {{API_KEYS_REQUIRED}}
|
||||
- External dependencies: {{EXT_DEPS}}
|
||||
- Source assets to incorporate: {{SOURCE_ASSETS}}
|
||||
|
||||
---
|
||||
|
||||
**This brief is the contract. The director and every downstream profile read
|
||||
it. If the brief changes, the kanban must be re-fired — don't edit live.**
|
||||
@@ -0,0 +1,185 @@
|
||||
#!/usr/bin/env bash
|
||||
# ═══════════════════════════════════════════════════════════════════════
|
||||
# Video Pipeline Setup — {{TITLE}}
|
||||
#
|
||||
# Generated by kanban-video-orchestrator skill.
|
||||
#
|
||||
# Slug: {{SLUG}}
|
||||
# Workspace: {{WORKSPACE}}
|
||||
# Tenant: {{TENANT}}
|
||||
# ═══════════════════════════════════════════════════════════════════════
|
||||
set -euo pipefail
|
||||
|
||||
PROJECT_SLUG="{{SLUG}}"
|
||||
WORKSPACE="$HOME/projects/video-pipeline/${PROJECT_SLUG}"
|
||||
TENANT="{{TENANT}}"
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# 1. Verify required API keys
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
echo "═══ Checking required API keys ═══"
|
||||
|
||||
check_key() {
|
||||
local var="$1"
|
||||
local kc_account="${2:-hermes}"
|
||||
local kc_service="${3:-$1}"
|
||||
if grep -q "^${var}=" "$HOME/.hermes/.env" 2>/dev/null && \
|
||||
[ -n "$(grep "^${var}=" "$HOME/.hermes/.env" | cut -d= -f2-)" ]; then
|
||||
echo " ✓ ${var} (env)"
|
||||
return 0
|
||||
fi
|
||||
if command -v security >/dev/null 2>&1 && \
|
||||
security find-generic-password -a "${kc_account}" -s "${kc_service}" -w >/dev/null 2>&1; then
|
||||
echo " ✓ ${var} (Keychain ${kc_account}/${kc_service})"
|
||||
return 0
|
||||
fi
|
||||
echo " ✗ ${var} not set in ~/.hermes/.env or Keychain (${kc_account}/${kc_service})"
|
||||
return 1
|
||||
}
|
||||
|
||||
# Customize this list per project — only check keys actually used:
|
||||
{{KEY_CHECKS}}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# 2. Create project workspace
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
echo "═══ Creating project workspace ═══"
|
||||
mkdir -p "$WORKSPACE"/{taste,audio/{voiceover,sfx},assets,scenes,checkpoints,tools,output}
|
||||
{{SCENE_DIRS}}
|
||||
echo " ✓ $WORKSPACE"
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# 3. Create Hermes profiles
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
echo "═══ Creating Hermes profiles ═══"
|
||||
|
||||
{{PROFILE_CREATE_COMMANDS}}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# 4. Configure profiles (toolsets, skills, cwd)
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
echo "═══ Configuring profiles ═══"
|
||||
|
||||
configure_profile() {
|
||||
local profile="$1"
|
||||
local toolsets_json="$2" # JSON array string, e.g. '["kanban","terminal","file"]'
|
||||
local skills_json="$3" # JSON array string, e.g. '["kanban-worker","ascii-video"]'
|
||||
python3 - "$profile" "$toolsets_json" "$skills_json" "$WORKSPACE" <<'PY'
|
||||
"""Patch a Hermes profile config.yaml using PyYAML so we don't depend on the
|
||||
exact default-config string format. Validates the patch took effect and exits
|
||||
non-zero if anything's off."""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
try:
|
||||
import yaml
|
||||
except ImportError:
|
||||
print("ERROR: PyYAML required. pip install pyyaml", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
profile, toolsets_json, skills_json, workspace = sys.argv[1:5]
|
||||
toolsets = json.loads(toolsets_json)
|
||||
skills = json.loads(skills_json)
|
||||
|
||||
p = os.path.expanduser(f"~/.hermes/profiles/{profile}/config.yaml")
|
||||
if not os.path.exists(p):
|
||||
print(f" ✗ profile config not found: {p}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
with open(p) as f:
|
||||
cfg = yaml.safe_load(f) or {}
|
||||
|
||||
# Apply our changes — only the keys we actually want to set.
|
||||
cfg["toolsets"] = toolsets
|
||||
cfg.setdefault("skills", {})
|
||||
cfg["skills"]["always_load"] = skills
|
||||
|
||||
# Note: we do NOT touch cfg["approvals"] — that's a security-sensitive
|
||||
# setting (manual confirmation of tool calls). Workspace cwd is overridden
|
||||
# per-task by `--workspace dir:<path>` on `hermes kanban create`, so we
|
||||
# don't need to mutate cfg["terminal"]["cwd"] either.
|
||||
|
||||
with open(p, "w") as f:
|
||||
yaml.safe_dump(cfg, f, sort_keys=False)
|
||||
|
||||
# Validate
|
||||
with open(p) as f:
|
||||
after = yaml.safe_load(f)
|
||||
errors = []
|
||||
if after.get("toolsets") != toolsets:
|
||||
errors.append(f"toolsets mismatch: {after.get('toolsets')!r}")
|
||||
if after.get("skills", {}).get("always_load") != skills:
|
||||
errors.append(f"skills.always_load mismatch: {after.get('skills', {}).get('always_load')!r}")
|
||||
if errors:
|
||||
print(f" ✗ {profile}: " + "; ".join(errors), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
PY
|
||||
if [ $? -ne 0 ]; then
|
||||
echo " ✗ failed to configure ${profile}" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo " ✓ ${profile}"
|
||||
}
|
||||
|
||||
{{PROFILE_CONFIG_COMMANDS}}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# 5. Write SOUL.md per profile
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
echo "═══ Writing profile personalities ═══"
|
||||
|
||||
{{SOUL_WRITES}}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# 6. Copy brief, TEAM.md, and any provided assets
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
echo "═══ Writing brief + taste ═══"
|
||||
|
||||
cat > "$WORKSPACE/brief.md" <<'BRIEF_EOF'
|
||||
{{BRIEF_CONTENTS}}
|
||||
BRIEF_EOF
|
||||
|
||||
cat > "$WORKSPACE/TEAM.md" <<'TEAM_EOF'
|
||||
{{TEAM_CONTENTS}}
|
||||
TEAM_EOF
|
||||
|
||||
{{TASTE_WRITES}}
|
||||
|
||||
{{ASSET_COPIES}}
|
||||
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
# 7. Fire the initial kanban task
|
||||
# ─────────────────────────────────────────────────────────────────────
|
||||
echo "═══ Firing initial kanban task ═══"
|
||||
|
||||
hermes kanban create "Direct production of {{TITLE}}" \
|
||||
--assignee director \
|
||||
--workspace dir:"$WORKSPACE" \
|
||||
--tenant "$TENANT" \
|
||||
--priority 2 \
|
||||
--max-runtime 4h \
|
||||
--body "$(cat <<EOF
|
||||
Read brief.md, TEAM.md, and taste/.
|
||||
|
||||
Decompose into the team graph defined in TEAM.md.
|
||||
|
||||
All child tasks MUST use:
|
||||
workspace_kind="dir"
|
||||
workspace_path="$WORKSPACE"
|
||||
tenant="$TENANT"
|
||||
|
||||
Do not execute the work yourself — route every concrete subtask to the
|
||||
appropriate profile via kanban_create.
|
||||
EOF
|
||||
)"
|
||||
|
||||
echo ""
|
||||
echo "═══ Setup complete ═══"
|
||||
echo ""
|
||||
echo "Monitor with:"
|
||||
echo " hermes kanban watch --tenant $TENANT"
|
||||
echo " hermes kanban list --tenant $TENANT"
|
||||
echo " hermes dashboard"
|
||||
echo ""
|
||||
echo "Workspace: $WORKSPACE"
|
||||
@@ -0,0 +1,38 @@
|
||||
# {{ROLE_NAME}}
|
||||
|
||||
You are the **{{ROLE_NAME}}** for this video production.
|
||||
|
||||
## Project context
|
||||
|
||||
- **Brief:** read `brief.md` in your CWD
|
||||
- **Team graph:** read `TEAM.md` in your CWD
|
||||
- **Style spec:** read `taste/brand-guide.md` and `taste/emotional-dna.md` in
|
||||
your CWD
|
||||
|
||||
## What you do
|
||||
|
||||
{{ROLE_RESPONSIBILITIES}}
|
||||
|
||||
## Inputs you read
|
||||
|
||||
{{INPUTS_READ}}
|
||||
|
||||
## Outputs you produce
|
||||
|
||||
{{OUTPUTS_PRODUCED}}
|
||||
|
||||
## Tools and skills available
|
||||
|
||||
- **Toolsets:** {{TOOLSETS}}
|
||||
- **Skills loaded:** {{SKILLS}}
|
||||
- **External APIs / CLIs:** {{EXTERNAL_TOOLS}}
|
||||
|
||||
## Rules
|
||||
|
||||
{{ROLE_RULES}}
|
||||
|
||||
{{COMMON_RULES}}
|
||||
|
||||
## Common reference commands
|
||||
|
||||
{{COMMON_COMMANDS}}
|
||||
@@ -0,0 +1,227 @@
|
||||
# Worked Examples
|
||||
|
||||
Six concrete pipelines covering different video styles. Each shows the team
|
||||
composition, task graph, and skill/tool choices the orchestrator would make
|
||||
for that brief. **These are illustrative, not templates** — adapt to the
|
||||
actual brief.
|
||||
|
||||
## Example 1 — Narrative short film (text-to-image → image-to-video → cut)
|
||||
|
||||
**Brief:** A 90-second noir-style short. A detective walks through a rainy
|
||||
city. Voiceover narration. AI-generated visuals.
|
||||
|
||||
**Team:**
|
||||
- `director` — vision, decomposition, approval
|
||||
- `writer` — script + voiceover copy (loads `humanizer` for natural voice)
|
||||
- `storyboarder` — beat-by-beat shot list (loads `excalidraw`)
|
||||
- `image-generator` — generates each shot's still via local ComfyUI workflows
|
||||
(loads `comfyui`)
|
||||
- `image-to-video-generator` — animates each still (Runway/Kling, OR
|
||||
ComfyUI's AnimateDiff/WAN workflows via `comfyui`)
|
||||
- `voice-talent` — narration via ElevenLabs
|
||||
- `audio-mixer` — VO + ambient pad
|
||||
- `editor` — assembly + transitions
|
||||
- `reviewer` — final QA
|
||||
|
||||
**Task graph:**
|
||||
```
|
||||
T0 director decompose
|
||||
T1 writer script + voiceover.md (parent: T0)
|
||||
T2 storyboarder shot list with framing per beat (parent: T1)
|
||||
T3 image-generator one still per shot (~12 shots) (parent: T2)
|
||||
T4 image-to-video animate each still (parent: T3)
|
||||
T5 voice-talent generate narration audio (parent: T1)
|
||||
T6 audio-mixer mix VO + ambient (parent: T5)
|
||||
T7 editor cut + transitions + audio mux (parents: T4, T6)
|
||||
T8 reviewer final QA (parent: T7)
|
||||
```
|
||||
|
||||
**Key choices:**
|
||||
- Local ComfyUI via `comfyui` skill is preferred over external API for
|
||||
cost/control — but external APIs are fine if ComfyUI isn't installed
|
||||
- `editor` profile is ffmpeg-only, no Hermes skill required beyond
|
||||
`kanban-worker`
|
||||
- Storyboarder produces `storyboard.excalidraw` alongside the markdown
|
||||
|
||||
## Example 2 — Product / marketing teaser
|
||||
|
||||
**Brief:** A 30-second product teaser for a developer tool. Shows code +
|
||||
terminal + UI screen recordings, voiceover, CTA at end. Square 1:1.
|
||||
|
||||
**Team:**
|
||||
- `director`
|
||||
- `copywriter` — taglines, voiceover script, CTA (loads `humanizer`)
|
||||
- `concept-artist` — style frames (loads `claude-design` for UI mockups)
|
||||
- `renderer-motion-graphics` — animated UI sequences (Remotion CLI)
|
||||
- `renderer-ascii` — terminal-style demo scenes (loads `ascii-video`)
|
||||
- `voice-talent` — VO via ElevenLabs
|
||||
- `editor` — assembly + brand-color treatment
|
||||
- `audio-mixer` — VO + light music bed
|
||||
- `captioner` — burned subtitles for muted-autoplay platforms
|
||||
- `masterer` — produces 1:1 + 9:16 + 16:9 variants
|
||||
|
||||
**Task graph:**
|
||||
```
|
||||
T0 director decompose
|
||||
T1 copywriter copy.md + cta + vo script (parent: T0)
|
||||
T2 concept-artist visual-spec.md + style frames (parent: T1)
|
||||
T3a renderer-motion-graphics scene 1: UI sequence (parent: T2)
|
||||
T3b renderer-ascii scene 2: terminal demo (parent: T2)
|
||||
T3c renderer-motion-graphics scene 3: feature highlight (parent: T2)
|
||||
T3d renderer-motion-graphics scene 4: CTA card (parent: T2)
|
||||
T4 voice-talent narration (parent: T1)
|
||||
T5 audio-mixer VO + music bed (parent: T4)
|
||||
T6 editor cut + transitions (parents: T3*, T5)
|
||||
T7 captioner SRT + burned subtitles (parent: T6)
|
||||
T8 masterer 1:1, 9:16, 16:9 variants (parent: T7)
|
||||
```
|
||||
|
||||
**Key choices:**
|
||||
- Multiple specialized renderers (motion-graphics + ASCII) coexist
|
||||
- Captioner is included because muted autoplay is the norm on social
|
||||
- `claude-design` skill for UI mockups maps directly to the product video idiom
|
||||
|
||||
## Example 3 — Music video (synced to provided track)
|
||||
|
||||
**Brief:** A 3-minute music video for a provided lo-fi hip-hop track. Visuals
|
||||
should pulse with the beat. Generative + ASCII hybrid. Vertical 9:16.
|
||||
|
||||
**Team:**
|
||||
- `director`
|
||||
- `music-supervisor` — analyze track, emit `audio/beats.json` (loads `songsee`)
|
||||
- `storyboarder` — beat-aligned shot list (loads `excalidraw`)
|
||||
- `renderer-ascii` — ASCII scenes synced to bass kicks (loads `ascii-video`)
|
||||
- `renderer-p5js` — generative particle scenes synced to highs (loads `p5js`)
|
||||
- `editor` — beat-cut assembly using `beats.json`
|
||||
- `reviewer` — sync QA
|
||||
|
||||
**Task graph:**
|
||||
```
|
||||
T0 director decompose
|
||||
T1 music-supervisor analyze track → beats.json + spectrogram (parent: T0)
|
||||
T2 storyboarder shot list aligned to beats (parents: T1, T0)
|
||||
T3a renderer-ascii scene 1: bass-driven ASCII (parent: T2)
|
||||
T3b renderer-p5js scene 2: high-end particle field (parent: T2)
|
||||
... (more scenes)
|
||||
T4 editor cut to beats + mux track (parents: T3*, T1)
|
||||
T5 reviewer sync QA + final approval (parent: T4)
|
||||
```
|
||||
|
||||
**Key choices:**
|
||||
- `music-supervisor` runs FIRST — `beats.json` gates the renderers
|
||||
- `editor` uses `beats.json` directly to align cuts to bass kicks
|
||||
- No voice-talent — music is the audio
|
||||
- Two specialized renderers (`ascii-video` + `p5js`) for visual variety
|
||||
|
||||
## Example 4 — Math/algorithm explainer
|
||||
|
||||
**Brief:** A 2-minute explainer of an algorithm. 3Blue1Brown-style. Animated
|
||||
diagrams, equations, narration. Square 1:1.
|
||||
|
||||
**Team:**
|
||||
- `director`
|
||||
- `writer` — narration script (loads `humanizer`)
|
||||
- `cinematographer` — visual spec (loads `manim-video`)
|
||||
- `renderer-manim` — all animated scenes (loads `manim-video`)
|
||||
- `voice-talent` — narration via ElevenLabs
|
||||
- `editor` — assembly + audio mux
|
||||
- `captioner` — burned subtitles
|
||||
|
||||
**Task graph:**
|
||||
```
|
||||
T0 director decompose
|
||||
T1 writer script + narration (parent: T0)
|
||||
T2 cinematographer visual spec for all scenes (parent: T1)
|
||||
T3a-Tn renderer-manim scenes 1..N (parents: T2)
|
||||
T4 voice-talent narration audio (parent: T1)
|
||||
T5 editor cut + mux (parents: T3*, T4)
|
||||
T6 captioner SRT + burn (parent: T5)
|
||||
```
|
||||
|
||||
**Key choices:**
|
||||
- `manim-video` skill drives both the cinematographer (visual language) and
|
||||
the renderer (actual scene production)
|
||||
- The `manim-video` skill's reference docs (animation-design-thinking,
|
||||
scene-planning, equations) auto-load when needed via the renderer's pinned skill
|
||||
|
||||
## Example 5 — ASCII video, music-track-only
|
||||
|
||||
**Brief:** A 60-second pure-ASCII video reactive to an existing track. No
|
||||
voiceover, no other tools. Square 1:1.
|
||||
|
||||
**Team:**
|
||||
- `director`
|
||||
- `music-supervisor` — track analysis (loads `songsee`)
|
||||
- `renderer-ascii` — all visuals (loads `ascii-video`)
|
||||
- `editor` — assembly + audio mux
|
||||
|
||||
**Task graph:**
|
||||
```
|
||||
T0 director decompose
|
||||
T1 music-supervisor analyze track (parent: T0)
|
||||
T2a renderer-ascii scene 1 (parents: T1, T0)
|
||||
T2b renderer-ascii scene 2 (parents: T1, T0)
|
||||
T2c renderer-ascii scene 3 (parents: T1, T0)
|
||||
T3 editor stitch + mux audio (parents: T2*)
|
||||
```
|
||||
|
||||
**Key choices:**
|
||||
- Minimal team (4 profiles) for a focused single-tool project
|
||||
- No reviewer — short experimental piece, director approves directly
|
||||
- All scenes run through one `renderer-ascii` profile because the `ascii-video`
|
||||
skill covers everything
|
||||
|
||||
This example illustrates the rule: **don't over-decompose**. Three scenes
|
||||
through one renderer is fine. Don't spawn three renderer profiles.
|
||||
|
||||
## Example 6 — Real-time / installation art
|
||||
|
||||
**Brief:** A 2-minute audio-reactive visual for a gallery installation. Driven
|
||||
by an audio input feed. TouchDesigner-based. 16:9 4K.
|
||||
|
||||
**Team:**
|
||||
- `director`
|
||||
- `cinematographer` — visual language spec (loads `touchdesigner-mcp`)
|
||||
- `renderer-touchdesigner` — all visuals + record-to-disk
|
||||
(loads `touchdesigner-mcp`)
|
||||
- `audio-mixer` — final loudness pass on the captured audio (optional if
|
||||
pre-mixed source)
|
||||
- `editor` — assemble final clip from TouchDesigner recording
|
||||
- `reviewer` — visual QA
|
||||
|
||||
**Task graph:**
|
||||
```
|
||||
T0 director decompose
|
||||
T1 cinematographer TD operator graph spec (parent: T0)
|
||||
T2 renderer-touchdesigner build TD network + record output (parent: T1)
|
||||
T3 editor trim + audio mux (parent: T2)
|
||||
T4 reviewer final QA (parent: T3)
|
||||
```
|
||||
|
||||
**Key choices:**
|
||||
- `touchdesigner-mcp` controls a running TouchDesigner instance — the
|
||||
cinematographer designs the operator graph, renderer builds it
|
||||
- Output is a recording from the running TD network, not a render-to-frames
|
||||
process; editor mostly just trims
|
||||
|
||||
## Pattern recognition
|
||||
|
||||
When the user describes a video, look for these signals to map to an example:
|
||||
|
||||
- **Plot, characters, scripted dialogue** → Example 1 (narrative)
|
||||
- **Specific product, CTA, brand colors, voiceover** → Example 2 (marketing)
|
||||
- **Track file provided, "synced to music"** → Example 3 (music video)
|
||||
- **"Explain how X works", math/algorithm/concept walkthrough** → Example 4 (manim explainer)
|
||||
- **Terminal aesthetic, ASCII, retro pixel** → Example 5 (ASCII)
|
||||
- **"Audio-reactive", "real-time", "installation"** → Example 6 (TouchDesigner)
|
||||
- **Comic-style narrative** → use `renderer-comic` (`baoyu-comic` skill)
|
||||
- **Retro game / pixel-art aesthetic** → use `renderer-pixel` (`pixel-art` skill)
|
||||
- **3D scene, photoreal environment** → use `renderer-3d` (`blender-mcp`)
|
||||
- **Generative art, particle system, shader** → use `renderer-p5js` (`p5js`)
|
||||
- **AI-generated photoreal stills + animation** → use `renderer-comfyui`
|
||||
(`comfyui`) for both stills and image-to-video
|
||||
- **"video about how the system works", recursive demo** → composable from
|
||||
any of the above; the recursion is a rendering technique, not a style
|
||||
|
||||
The actual team should be derived from the specific brief — these examples are
|
||||
starting points, not endpoints.
|
||||
@@ -0,0 +1,166 @@
|
||||
# Intake — Discovery Question Banks
|
||||
|
||||
The discovery process is **adaptive**. Always start with three baseline
|
||||
questions to identify the broad style category, then drill into a per-style
|
||||
question bank. Ask 2-4 questions at a time, listen, then proceed. Make
|
||||
reasonable assumptions whenever the user implies an answer.
|
||||
|
||||
## Tier 0 — Baseline (always ask)
|
||||
|
||||
1. **What is the video?** — One-sentence pitch
|
||||
2. **How long?** — Approximate duration
|
||||
3. **Aspect ratio + target platform?** — 16:9 / 9:16 / 1:1 / 4:5; X, IG, YouTube, internal, etc.
|
||||
|
||||
From these answers, classify the style category and pick the relevant Tier 1
|
||||
follow-ups. **Do not** continue asking until you have at least these three.
|
||||
|
||||
## Style classification
|
||||
|
||||
Map the brief to one of these archetypes (or a hybrid):
|
||||
|
||||
| Archetype | Tells |
|
||||
|-----------|-------|
|
||||
| **Narrative film** | Plot, characters, scenes-with-events, dialogue, location |
|
||||
| **Product / marketing** | A specific product or feature being shown / sold; CTA at end |
|
||||
| **Music video** | A specific track exists; visuals sync to music |
|
||||
| **Explainer / educational** | A concept being taught; voiceover-driven |
|
||||
| **Tutorial / changelog** | Software demo, terminal-heavy, technical |
|
||||
| **ASCII / terminal art** | Retro terminal aesthetic explicit, character-grid |
|
||||
| **Abstract / loop** | Generative, no plot, often perfect-loop |
|
||||
| **Documentary / interview cut** | Real footage, transcription-driven |
|
||||
| **Real-time / installation** | Audio-reactive, gallery installation, VJ output |
|
||||
|
||||
If ambiguous, **ask** which category fits — don't guess. Hybrids are common
|
||||
(e.g., a product video with a narrative arc); decompose into the dominant
|
||||
mode + secondary modifiers.
|
||||
|
||||
**Recursive / meta** ("a video that shows its own production") is a
|
||||
*rendering technique*, not a separate style — compose it from any of the
|
||||
above by adding a two-pass render step where pass 2 uses pass 1's output as
|
||||
texture inside the final scene.
|
||||
|
||||
## Tier 1 — Per-style follow-ups
|
||||
|
||||
### Narrative film
|
||||
|
||||
- **Setting / world?** — When and where the story takes place
|
||||
- **Characters?** — How many, archetypes, who carries dialogue
|
||||
- **Beat list or full script?** — Has the user written the story or do we draft it
|
||||
- **Dialogue language?** — Spoken lines, on-screen subs only, silent
|
||||
- **Visual generation approach?** — Text-to-image (FAL/Midjourney/Imagen) →
|
||||
image-to-video (Runway/Kling), 3D animation (Blender), 2D animation,
|
||||
procedural, or hybrid
|
||||
- **Voice approach?** — TTS (which voice), recorded VO, no dialogue
|
||||
- **Music / score?** — Commissioned (via `songwriting-and-ai-music` Suno
|
||||
prompts, or local `heartmula`), licensed track provided, silent
|
||||
|
||||
### Product / marketing
|
||||
|
||||
- **Product?** — Name, what it does, key feature being shown
|
||||
- **Target audience?** — Who's watching, what they care about
|
||||
- **CTA?** — Visit URL, install, sign up, etc.
|
||||
- **Tone?** — Serious, playful, technical, premium, edgy
|
||||
- **Brand assets available?** — Logo files, color palette, fonts, existing footage
|
||||
- **Animation style?** — Motion graphics (Remotion / AE-style), screen recording,
|
||||
generative, illustrated
|
||||
- **Voiceover?** — Yes (which voice / language) or text-only
|
||||
- **Music?** — Track provided, license-free needed, custom-composed
|
||||
|
||||
### Music video
|
||||
|
||||
- **Track file?** — Path to the audio (essential — we'll analyze BPM + beats)
|
||||
- **Track length to use?** — Full song or a section
|
||||
- **Genre / energy?** — Tells what visual rhythm and density to use
|
||||
- **Lyric / narrative content?** — Are there lyrics to render on screen,
|
||||
or is it purely visual?
|
||||
- **Visual reference style?** — Existing music videos / artists for reference
|
||||
- **Performer footage?** — None, has clips, will provide
|
||||
- **Visual generation approach?** — Per-beat generative, edit-driven cuts of stock
|
||||
footage, illustrated, hybrid
|
||||
|
||||
### Explainer / educational
|
||||
|
||||
- **What concept is being taught?** — One-sentence concept, key takeaway
|
||||
- **Audience expertise?** — Beginner / intermediate / expert
|
||||
- **Diagram density?** — Heavy math / formulas / code / abstract concepts
|
||||
- **Voiceover?** — TTS / recorded / on-screen text only
|
||||
- **Tool preference?** — `manim-video` (math), `p5js` (generative),
|
||||
Remotion (UI motion graphics), `comfyui` (AI-generated visuals),
|
||||
`ascii-video` (technical/retro), hybrid
|
||||
- **Pacing?** — Fast and dense (3Blue1Brown) or slow and contemplative
|
||||
|
||||
### Tutorial / changelog / software demo
|
||||
|
||||
- **Software being demonstrated?** — Name, what it does
|
||||
- **Demo script?** — Sequence of commands / screens to show
|
||||
- **Terminal-only or with GUI?**
|
||||
- **Voiceover for narration?**
|
||||
- **Diagram support needed?** — Often these benefit from a diagram skill
|
||||
alongside the screen-capture/render step (`excalidraw`,
|
||||
`architecture-diagram`, `concept-diagrams`)
|
||||
|
||||
### ASCII / terminal art
|
||||
|
||||
- **Source material?** — Generative / driven by audio / converting existing
|
||||
video / static image starting point
|
||||
- **Color palette?** — Brand-driven (gold/black/blue), Matrix green, full
|
||||
rainbow, monochrome
|
||||
- **Audio reactivity?** — None / loose mood / tight beat sync / FFT-driven
|
||||
- **Character set?** — ASCII only / Unicode block-drawing / mystic glyphs
|
||||
- **Loop or narrative?** — Perfect loop or one-shot
|
||||
|
||||
### Abstract / loop
|
||||
|
||||
- **Mood / emotion?** — One word that captures the feel
|
||||
- **Motion type?** — Zoom-into-itself, particle drift, wave, geometric, organic
|
||||
- **Loop required?** — Perfect loop (Droste-style) or just satisfying ending
|
||||
- **Audio?** — Silent, ambient pad, beat-synced
|
||||
|
||||
### Documentary / interview cut
|
||||
|
||||
- **Source footage?** — Provided clips, length per clip
|
||||
- **Transcript / subtitles?** — Provided or to be generated
|
||||
- **Story structure?** — Chronological / thematic / arc
|
||||
- **B-roll approach?** — Generated, stock library, none
|
||||
|
||||
### Real-time / installation
|
||||
|
||||
- **Output environment?** — Gallery wall, projector, screen, web embed
|
||||
- **Audio source?** — Live audio input, pre-recorded track, both
|
||||
- **Reactivity tightness?** — Mood-level (loose) vs. tight beat-sync vs. live
|
||||
parameter control
|
||||
- **Tool preference?** — `touchdesigner-mcp` for full TD operator graphs;
|
||||
`p5js` for web-canvas; `comfyui` for generative-AI fed by audio features
|
||||
|
||||
## Tier 2 — Always ask near the end
|
||||
|
||||
- **Brand assets path?** — Where logo / color palette / fonts / music library lives
|
||||
- **Output format requirements?** — Codec preference, target file size, accepted
|
||||
alternates (vertical cut, GIF, audio-only)
|
||||
- **Deadline?** — Affects task `max_runtime_seconds` and acceptable scope
|
||||
- **Quality bar?** — Rough draft for review / polished final / archival
|
||||
- **Existing footage / assets to reuse?** — Anything that should appear, not just inform
|
||||
|
||||
## Reasonable assumption defaults
|
||||
|
||||
When the user under-specifies, fill in these defaults rather than asking:
|
||||
|
||||
| Question | Default |
|
||||
|----------|---------|
|
||||
| Frame rate | 30 fps for X / IG; 60 fps for tutorials/explainers; 24 fps for narrative film |
|
||||
| Resolution | 1080×1080 for square, 1920×1080 for 16:9, 1080×1920 for 9:16 |
|
||||
| Codec | H.264 / yuv420p, CRF 18 |
|
||||
| Audio codec | AAC 192 kbps |
|
||||
| Voice | Provider's mid-range neutral voice unless brand calls for distinctive timbre |
|
||||
| Music | Silent (require user to specify if music is wanted) |
|
||||
| Captions | On for explainer/tutorial; off for narrative/abstract unless requested |
|
||||
| Quality bar | Polished final unless user says draft |
|
||||
|
||||
State the assumption explicitly: *"Assuming 30fps and AAC audio unless you say otherwise — proceed?"*
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
- **Asking 10 questions at once.** Maximum 4 per turn.
|
||||
- **Asking for things the brief already implies.** If the user said "music video for my track," do not ask "is there a track?"
|
||||
- **Failing to classify before drilling in.** Tier-1 questions depend on classification; mixing them up wastes turns.
|
||||
- **Treating "make a video" as enough to proceed.** Always confirm the three baseline questions.
|
||||
@@ -0,0 +1,276 @@
|
||||
# Kanban Setup — Project Bootstrap & Profile Configuration
|
||||
|
||||
Once the brief is locked and the team is designed, the next step is producing
|
||||
the actual `setup.sh` that creates the project workspace, configures Hermes
|
||||
profiles, and fires the initial kanban task.
|
||||
|
||||
This file documents the patterns. The companion script
|
||||
`scripts/bootstrap_pipeline.py` automates most of it from a structured input
|
||||
JSON.
|
||||
|
||||
> **Credit:** the single-project-workspace layout, profile-config patching
|
||||
> approach, SOUL.md-per-profile convention, and `--workspace dir:<path>` rule
|
||||
> are adapted from alt-glitch's original multi-agent video pipeline:
|
||||
> [NousResearch/kanban-video-pipeline](https://github.com/NousResearch/kanban-video-pipeline).
|
||||
> This skill generalizes those patterns across video styles and replaces the
|
||||
> string-replacement config patcher with a PyYAML-based one.
|
||||
|
||||
## Project workspace structure
|
||||
|
||||
Every video project gets one workspace under `~/projects/video-pipeline/<slug>/`:
|
||||
|
||||
```
|
||||
~/projects/video-pipeline/<slug>/
|
||||
├── brief.md ← the contract; all tasks reference
|
||||
├── TEAM.md ← team composition + task graph (director reads this)
|
||||
├── taste/
|
||||
│ ├── brand-guide.md ← color, typography, motion rules
|
||||
│ ├── emotional-dna.md ← what the piece should FEEL like
|
||||
│ └── style-frames/ ← optional: visual references
|
||||
├── audio/
|
||||
│ ├── track.mp3 ← provided music (if any)
|
||||
│ ├── voiceover/ ← per-line TTS clips
|
||||
│ └── sfx/ ← sound effects
|
||||
├── assets/
|
||||
│ ├── logos/
|
||||
│ ├── fonts/
|
||||
│ └── existing-footage/ ← reusable provided clips
|
||||
├── scenes/
|
||||
│ ├── scene-01/
|
||||
│ │ ├── VISUAL_SPEC.md ← cinematographer's per-scene spec
|
||||
│ │ ├── render.py ← renderer's code (or sketch.html, etc.)
|
||||
│ │ ├── checkpoints/ ← preview frames for QA
|
||||
│ │ └── clip.mp4 ← the deliverable for this scene
|
||||
│ ├── scene-02/...
|
||||
│ └── ...
|
||||
├── checkpoints/ ← global review frames
|
||||
├── tools/ ← optional project-local helpers
|
||||
└── output/
|
||||
├── final.mp4 ← stitched + audio
|
||||
├── final-noaudio.mp4
|
||||
├── final-9x16.mp4 ← optional: vertical alternate
|
||||
└── captions.srt ← optional: subtitle file
|
||||
```
|
||||
|
||||
**The slug** is derived from the brief title: lowercase, hyphen-separated.
|
||||
Example: `q3-product-teaser`, `ascii-mood-loop`, `interview-cut-2026-q1`.
|
||||
|
||||
## The setup.sh script
|
||||
|
||||
The setup script does six things in order:
|
||||
|
||||
1. **Create workspace tree** — all directories above
|
||||
2. **Create profiles** — `hermes profile create <name> --clone`
|
||||
3. **Configure profiles** — patch each profile's
|
||||
`~/.hermes/profiles/<name>/config.yaml` to set toolsets, always_load skills,
|
||||
and `cwd`
|
||||
4. **Write SOUL.md per profile** — the personality + role definition
|
||||
5. **Copy any provided assets + write `brief.md`, `TEAM.md`, and `taste/`**
|
||||
6. **Fire the initial kanban task** — `hermes kanban create` assigned to the director
|
||||
|
||||
See `assets/setup.sh.tmpl` for the skeleton.
|
||||
|
||||
### Profile creation pattern
|
||||
|
||||
```bash
|
||||
hermes profile create director --clone 2>/dev/null || true
|
||||
```
|
||||
|
||||
The `--clone` flag clones from the active profile (preserving model, base
|
||||
config). The `|| true` makes the script idempotent — re-running won't error if
|
||||
the profile already exists.
|
||||
|
||||
### Profile config patching
|
||||
|
||||
Each profile has a YAML config at `~/.hermes/profiles/<name>/config.yaml`. The
|
||||
setup script edits exactly two keys:
|
||||
|
||||
1. `toolsets:` — replace the default with the role's required toolsets
|
||||
2. `skills.always_load:` — list the role's must-load skills (may be empty)
|
||||
|
||||
**Do NOT** modify `approvals.mode` (controls user-confirmation of tool calls
|
||||
— a security setting that must stay as the user configured it). **Do NOT**
|
||||
modify `terminal.cwd` — the kanban dispatcher overrides cwd per-task via
|
||||
`--workspace dir:<path>`, so the profile's cwd is irrelevant to the kanban
|
||||
work and changing it could break the user's interactive use of the profile.
|
||||
|
||||
Use **PyYAML**, not string replacement, so the patch is robust against
|
||||
default-config schema drift:
|
||||
|
||||
```bash
|
||||
configure_profile() {
|
||||
local profile="$1"
|
||||
local toolsets_json="$2" # JSON array, e.g. '["kanban","terminal","file"]'
|
||||
local skills_json="$3" # JSON array, e.g. '["kanban-worker","ascii-video"]'
|
||||
python3 - "$profile" "$toolsets_json" "$skills_json" <<'PY'
|
||||
import json, os, sys, yaml
|
||||
profile, ts_json, sk_json = sys.argv[1:4]
|
||||
p = os.path.expanduser(f"~/.hermes/profiles/{profile}/config.yaml")
|
||||
with open(p) as f:
|
||||
cfg = yaml.safe_load(f) or {}
|
||||
cfg["toolsets"] = json.loads(ts_json)
|
||||
cfg.setdefault("skills", {})["always_load"] = json.loads(sk_json)
|
||||
with open(p, "w") as f:
|
||||
yaml.safe_dump(cfg, f, sort_keys=False)
|
||||
PY
|
||||
}
|
||||
```
|
||||
|
||||
PyYAML must be installed in the user's Python (it ships with most Hermes
|
||||
installs). If absent: `pip install pyyaml`.
|
||||
|
||||
The setup script should also **validate** the patch by re-reading the file
|
||||
and comparing — see `assets/setup.sh.tmpl` for the validation pattern.
|
||||
|
||||
### SOUL.md per profile
|
||||
|
||||
Each profile gets a `SOUL.md` at `~/.hermes/profiles/<name>/SOUL.md` that
|
||||
defines its role, voice, and rules. See `assets/soul.md.tmpl` for the
|
||||
template. Customize per role and per project.
|
||||
|
||||
The director's SOUL.md should be the most opinionated — its voice flavors
|
||||
the entire production. **Critical content for the director's SOUL.md:**
|
||||
|
||||
- **Anti-temptation rules:** "Do not execute the work yourself. For every
|
||||
concrete task, create a kanban task and assign it. Decompose, route, comment,
|
||||
approve — that's the whole job." (The `kanban-orchestrator` skill provides
|
||||
the deeper playbook; load it.)
|
||||
- **Decomposition steps:** Read `brief.md`, `TEAM.md`, `taste/`. Use the team
|
||||
graph in `TEAM.md` to fan out tasks.
|
||||
- **The workspace_path rule** (see below).
|
||||
|
||||
Other profiles' SOUL.md is briefer; mostly mechanical: who you are, what you
|
||||
read, what you produce, what skills/tools to use, where to write outputs.
|
||||
Most non-director profiles should `always_load: kanban-worker` for the
|
||||
deeper-than-baseline kanban guidance.
|
||||
|
||||
### Initial kanban task
|
||||
|
||||
The final action of setup.sh is firing the kanban:
|
||||
|
||||
```bash
|
||||
hermes kanban create "Direct production of <video title>" \
|
||||
--assignee director \
|
||||
--workspace dir:"$HOME/projects/video-pipeline/${PROJECT_SLUG}" \
|
||||
--tenant ${PROJECT_SLUG} \
|
||||
--priority 2 \
|
||||
--max-runtime 4h \
|
||||
--body "$(cat <<EOF
|
||||
Read brief.md, TEAM.md, and taste/.
|
||||
Decompose into the team graph defined in TEAM.md.
|
||||
All child tasks MUST use:
|
||||
workspace_kind="dir"
|
||||
workspace_path="$HOME/projects/video-pipeline/${PROJECT_SLUG}"
|
||||
tenant="${PROJECT_SLUG}"
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
The `--workspace dir:<path>` flag is **critical** — it tells the kanban that
|
||||
all child tasks share this workspace. Skipping or using `worktree` will
|
||||
isolate profiles and break artifact sharing.
|
||||
|
||||
## The TEAM.md file
|
||||
|
||||
Alongside `brief.md`, write a `TEAM.md` that the director reads. It documents
|
||||
the team composition + task graph the orchestrator should follow. This
|
||||
removes ambiguity and prevents the director from inventing extra steps.
|
||||
|
||||
Example structure (for an ASCII video with a music supervisor and editor):
|
||||
|
||||
```markdown
|
||||
# Team & Task Graph — <video title>
|
||||
|
||||
## Team
|
||||
|
||||
- `director` (this profile) — vision, decomposition, approval
|
||||
- `cinematographer` — visual spec, quality review (loads `ascii-video`)
|
||||
- `renderer-ascii` — ASCII scenes (loads `ascii-video`)
|
||||
- `music-supervisor` — track analysis (loads `songsee`)
|
||||
- `voice-talent` — narration (uses ElevenLabs API)
|
||||
- `audio-mixer` — final mix (ffmpeg)
|
||||
- `editor` — assembly (ffmpeg)
|
||||
- `reviewer` — final QA gate
|
||||
|
||||
## Task Graph
|
||||
|
||||
T0: this task — decompose
|
||||
│
|
||||
├── T1: cinematographer "Design visual language" (parent: T0)
|
||||
│ │
|
||||
│ ├── T2a: renderer-ascii "Scene 1 — title card" (parent: T1)
|
||||
│ ├── T2b: renderer-ascii "Scene 2 — main beat" (parent: T1)
|
||||
│ ├── T2c: renderer-ascii "Scene 3 — outro" (parent: T1)
|
||||
│
|
||||
├── T3: music-supervisor "Analyze track + emit beats.json" (parent: T0)
|
||||
│
|
||||
├── T4: voice-talent "Generate narration" (parent: T0)
|
||||
│
|
||||
├── T5: audio-mixer "Mix VO + bg music" (parents: T3, T4)
|
||||
│
|
||||
├── T6: editor "Assemble cut + mux audio" (parents: T2*, T5)
|
||||
│
|
||||
└── T7: reviewer "Final QA" (parent: T6)
|
||||
```
|
||||
|
||||
The director turns this into actual `kanban_create` calls.
|
||||
|
||||
## API-key prerequisites check
|
||||
|
||||
Before firing the kanban, verify required keys are available. Check both
|
||||
`~/.hermes/.env` and macOS Keychain (if on macOS):
|
||||
|
||||
```bash
|
||||
check_key() {
|
||||
local var="$1"
|
||||
local kc_account="$2"
|
||||
local kc_service="$3"
|
||||
if grep -q "^${var}=" ~/.hermes/.env 2>/dev/null && \
|
||||
[ -n "$(grep "^${var}=" ~/.hermes/.env | cut -d= -f2-)" ]; then
|
||||
return 0
|
||||
fi
|
||||
if command -v security >/dev/null 2>&1 && \
|
||||
security find-generic-password -a "${kc_account}" -s "${kc_service}" -w >/dev/null 2>&1; then
|
||||
return 0
|
||||
fi
|
||||
echo "ERROR: ${var} not set in ~/.hermes/.env or Keychain (${kc_account}/${kc_service})"
|
||||
return 1
|
||||
}
|
||||
|
||||
check_key ELEVENLABS_API_KEY hermes ELEVENLABS_API_KEY || exit 1
|
||||
check_key OPENROUTER_API_KEY hermes OPENROUTER_API_KEY || exit 1
|
||||
# ...
|
||||
```
|
||||
|
||||
If a key is missing, the script aborts with a clear message rather than
|
||||
firing a kanban that will hit credential errors mid-execution.
|
||||
|
||||
## Critical rules
|
||||
|
||||
1. **`workspace_kind="dir"` + `workspace_path="<absolute>"` on every kanban_create.** Otherwise profiles can't share artifacts.
|
||||
|
||||
2. **Tenant every task.** `--tenant <project-slug>` keeps the dashboard scoped
|
||||
and prevents cross-pollination with other ongoing kanbans.
|
||||
|
||||
3. **Idempotency keys.** For tasks that should not duplicate on re-run (e.g.,
|
||||
setup creating profiles), use the `idempotency_key` argument or check
|
||||
existence first.
|
||||
|
||||
4. **`max_runtime_seconds` per task.** Renderers that get stuck eat compute.
|
||||
Standard defaults:
|
||||
- Renderer task: 1800s (30min)
|
||||
- Editor task: 600s (10min)
|
||||
- Voice-talent task: 300s (5min)
|
||||
- Image-generator task: 600s (10min)
|
||||
- Image-to-video-generator task: 900s (15min)
|
||||
|
||||
5. **Heartbeats for long renders.** Tasks expected to run >5min should emit
|
||||
`kanban_heartbeat` periodically with progress. Renderers should report
|
||||
frame counts; the editor should report assembly progress.
|
||||
|
||||
6. **The `audio/` and `taste/` dirs are populated BEFORE firing the kanban.**
|
||||
Don't ask the director's pipeline to source these — copy at setup time.
|
||||
|
||||
7. **`brief.md` is read-only after setup.** If the brief changes during
|
||||
execution, that's a significant pivot — re-fire the kanban rather than edit
|
||||
live.
|
||||
@@ -0,0 +1,180 @@
|
||||
# Monitoring — Watch the Pipeline + Intervene
|
||||
|
||||
After `setup.sh` fires the kanban, the work runs autonomously. The role of
|
||||
this skill in the execution phase is to help the user (and the AI overseeing
|
||||
the session) detect problems early and intervene effectively.
|
||||
|
||||
## Live monitoring commands
|
||||
|
||||
```bash
|
||||
# Live event stream — task spawns, status changes, heartbeats, completions
|
||||
hermes kanban watch --tenant <project-slug>
|
||||
|
||||
# Snapshot of the board
|
||||
hermes kanban list --tenant <project-slug>
|
||||
hermes kanban list --tenant <project-slug> --json # machine-readable
|
||||
|
||||
# Per-status counts + oldest-ready age
|
||||
hermes kanban stats --tenant <project-slug>
|
||||
|
||||
# Visual dashboard (browser)
|
||||
hermes dashboard
|
||||
|
||||
# Inspect a specific task (includes comments + events)
|
||||
hermes kanban show <task-id>
|
||||
|
||||
# Follow a single task's event stream
|
||||
hermes kanban tail <task-id>
|
||||
```
|
||||
|
||||
Verify available subcommands with `hermes kanban --help` — the kanban CLI
|
||||
ships with `init / create / list / show / assign / link / unlink / claim /
|
||||
comment / complete / block / unblock / archive / tail / dispatch / watch /
|
||||
stats / heartbeat / log / runs / context / gc`.
|
||||
|
||||
The companion `scripts/monitor.py` polls the kanban via the CLI and surfaces
|
||||
common issues (stuck tasks, missing heartbeats, repeated retries, dependency
|
||||
deadlocks).
|
||||
|
||||
## What to watch for
|
||||
|
||||
### Healthy pipeline indicators
|
||||
|
||||
- Tasks transition `READY → RUNNING → DONE` in roughly the expected order
|
||||
- Renderers emit periodic `kanban_heartbeat` events with progress (e.g. "frame
|
||||
240/720")
|
||||
- Each task's runtime is well under its `max_runtime_seconds` cap
|
||||
- No task accumulates more than 1 retry
|
||||
- Dependency arrows resolve (children unblock as parents complete)
|
||||
|
||||
### Warning signs
|
||||
|
||||
| Symptom | Likely cause | Action |
|
||||
|---------|--------------|--------|
|
||||
| Task RUNNING but no heartbeat in 2+ min | Worker stuck, infinite loop, blocked on input | `hermes kanban show <id>` — read the worker's last events. The dispatcher SIGTERMs tasks that exceed their `max-runtime`; if you need to stop one earlier, `hermes kanban block <id>` then `hermes kanban archive <id>`, and create a re-run task. |
|
||||
| Same task retried 2+ times | Reproducible failure (missing key, bad spec, broken tool) | `hermes kanban show <id>` to read failure events. Fix root cause before re-running. |
|
||||
| RUNNING longer than max_runtime | Task is slow but progressing OR genuinely stuck | Check heartbeats with `hermes kanban tail <id>`. If progressing, the dispatcher will SIGTERM eventually anyway — raise `max-runtime` on a re-created task. |
|
||||
| Child task READY but parents still RUNNING for >2× expected | Cascade slow, dependency miswired | Check the dependency graph. Inspect the parent: sometimes it completed but its handoff fields (summary, metadata) were empty so the child has nothing to consume. |
|
||||
| New tasks not appearing | Director is hung in decomposition | Inspect director task with `kanban show`. Often a malformed `kanban_create` call. |
|
||||
| Specialist tasks completing instantly | Decomposition created tasks without bodies | Director didn't pass enough context. Re-create with explicit body content. |
|
||||
| Tasks created but never picked up | Profile not running, or tenant mismatch, or dispatcher not running | Check `hermes profile list` (profile exists?), `hermes status` (gateway/dispatcher up?), and verify tenant. |
|
||||
| Specific renderer task fails → review note → renderer redoes → fails again | Brief is asking for the impossible | Pivot the brief, not the renderer. |
|
||||
|
||||
## Intervention recipes
|
||||
|
||||
### Rejecting bad output
|
||||
|
||||
When a renderer ships a clip that doesn't pass review:
|
||||
|
||||
```bash
|
||||
# 1. Comment on the renderer's task with specific feedback
|
||||
hermes kanban comment <renderer-task-id> "Scene 3 looks too sparse \
|
||||
— increase visual density. Tighten color palette to brand spec."
|
||||
|
||||
# 2. Create a re-render task with the original as parent
|
||||
hermes kanban create "Scene 3 — re-render with feedback" \
|
||||
--assignee renderer-ascii \
|
||||
--parent <renderer-task-id> \
|
||||
--workspace dir:"$HOME/projects/video-pipeline/<slug>" \
|
||||
--tenant <slug> \
|
||||
--skill ascii-video \
|
||||
--max-runtime 30m
|
||||
```
|
||||
|
||||
### Adding a new dependency mid-flight
|
||||
|
||||
When the editor needs an asset that wasn't originally planned (e.g., a captions
|
||||
file):
|
||||
|
||||
```bash
|
||||
# 1. Create the new task and capture its id
|
||||
NEW_TASK_ID=$(hermes kanban create "Generate SRT captions from voiceover" \
|
||||
--assignee captioner \
|
||||
--workspace dir:"$HOME/projects/video-pipeline/<slug>" \
|
||||
--tenant <slug> \
|
||||
--json | python3 -c "import json,sys;print(json.load(sys.stdin)['id'])")
|
||||
|
||||
# 2. Wire it as a parent of the editor's task with `kanban link`
|
||||
hermes kanban link "$NEW_TASK_ID" <editor-task-id>
|
||||
```
|
||||
|
||||
`kanban link` takes `parent_id child_id` (parent first). Use `kanban unlink`
|
||||
to remove a dependency.
|
||||
|
||||
### Stopping a worker that's stuck
|
||||
|
||||
The kanban dispatcher will SIGTERM (then SIGKILL) any task that exceeds its
|
||||
`--max-runtime` automatically. To stop one sooner:
|
||||
|
||||
```bash
|
||||
# Mark blocked so the dispatcher leaves it alone, then archive
|
||||
hermes kanban block <task-id>
|
||||
hermes kanban archive <task-id>
|
||||
|
||||
# Diagnose what happened
|
||||
hermes kanban show <task-id> # task body, comments, recent events
|
||||
hermes kanban tail <task-id> # follow the live event stream
|
||||
hermes kanban log <task-id> # worker process log
|
||||
```
|
||||
|
||||
After stopping, decide: fix root cause + re-create the task, or skip and
|
||||
adjust dependent tasks.
|
||||
|
||||
### Pivoting the brief
|
||||
|
||||
If during execution the user wants something fundamentally different:
|
||||
|
||||
1. Cancel the active director task and all RUNNING children
|
||||
2. Edit `brief.md` and `TEAM.md`
|
||||
3. Re-fire the initial `hermes kanban create` for the director
|
||||
|
||||
Don't try to "edit while running" — the kanban's audit trail makes a clean
|
||||
pivot more legible than mid-stream changes.
|
||||
|
||||
## Periodic check-in script
|
||||
|
||||
A simple polling pattern for hands-off monitoring:
|
||||
|
||||
```bash
|
||||
while true; do
|
||||
clear
|
||||
hermes kanban list --tenant <slug>
|
||||
echo "---"
|
||||
hermes kanban stats --tenant <slug>
|
||||
sleep 30
|
||||
done
|
||||
```
|
||||
|
||||
For a live event feed, run `hermes kanban watch --tenant <slug>` in a
|
||||
separate terminal — it streams task lifecycle events as they happen.
|
||||
|
||||
For automated intervention (auto-restart stuck tasks, auto-create re-render on
|
||||
review failure), see the `scripts/monitor.py` patterns.
|
||||
|
||||
## When to call it done
|
||||
|
||||
The pipeline is finished when:
|
||||
|
||||
1. All RENDER tasks complete and pass review
|
||||
2. The editor's `output/final.mp4` exists and `ffprobe` confirms expected
|
||||
duration + streams
|
||||
3. The reviewer (if present) has approved
|
||||
4. Optional masterer variants exist
|
||||
|
||||
At this point, present the final.mp4 path to the user along with any review
|
||||
notes. Do NOT delete the workspace — the user may want to iterate on a single
|
||||
scene without re-running the whole pipeline.
|
||||
|
||||
## Common gotchas
|
||||
|
||||
- **Tenant mismatches.** A task created with the wrong tenant won't appear in
|
||||
monitoring. Always pass `--tenant <slug>` consistently.
|
||||
- **Profile process not running.** Tasks queue indefinitely in READY if no
|
||||
worker for that profile is online. Check `hermes profile list` and start
|
||||
any missing profiles.
|
||||
- **Workspace permissions.** All profiles need read+write to the workspace
|
||||
directory. `chmod -R u+rw <workspace>` if any worker reports permission
|
||||
errors.
|
||||
- **Audio/visual sync.** The editor's clip stitching must match the
|
||||
renderer's actual output durations. Don't hardcode scene durations in
|
||||
the editor — read from the renderer's handoff metadata.
|
||||
@@ -0,0 +1,298 @@
|
||||
# Role Archetypes
|
||||
|
||||
The library of role archetypes for video production. **Compose a team from this
|
||||
list, don't clone a fixed roster.** Most videos need 4-7 profiles. The director
|
||||
is always present; everything else is conditional on the brief.
|
||||
|
||||
Each role's profile name is by convention `kebab-case` (e.g. `creative-director`,
|
||||
`image-generator`). Multiple instances of the same role get descriptive suffixes
|
||||
when they need different focus (e.g., `renderer-ascii`, `renderer-3d`).
|
||||
|
||||
For toolset + skill mapping per role, see [tool-matrix.md](tool-matrix.md).
|
||||
|
||||
## Always present
|
||||
|
||||
### director
|
||||
|
||||
The vision-holder. Reads the brief and brand guide, decomposes into a task
|
||||
graph, comments to steer creative direction, approves the final cut.
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** `kanban-orchestrator`. The kanban plugin auto-injects baseline
|
||||
orchestration guidance for free; `kanban-orchestrator` is the deeper
|
||||
decomposition playbook. Add `creative-ideation` if the brief is wide-open
|
||||
and needs framing help.
|
||||
- **Personality:** Tied to the brand voice — see `assets/soul.md.tmpl`
|
||||
|
||||
The director has the same toolset as everyone else, but its `SOUL.md` rules
|
||||
**forbid** execution. The "decompose, don't execute" discipline is enforced
|
||||
by personality + the kanban-orchestrator skill, not by missing tools.
|
||||
|
||||
## Pre-production roles
|
||||
|
||||
Pick based on what the brief needs.
|
||||
|
||||
### writer / screenwriter
|
||||
|
||||
Writes scripts, dialogue, voiceover copy, narration. Use for any video with
|
||||
spoken or written words beyond a tagline.
|
||||
|
||||
- **Toolsets:** kanban, file
|
||||
- **Skills:** `kanban-worker`, `humanizer` (post-process to strip AI-tells)
|
||||
- **Outputs:** `script.md`, `narration.md`, `dialogue/scene-NN.md`
|
||||
|
||||
### copywriter
|
||||
|
||||
Like `writer` but specifically for marketing copy: taglines, CTAs, voiceover
|
||||
scripts for product videos.
|
||||
|
||||
- **Toolsets:** kanban, file
|
||||
- **Skills:** `kanban-worker`, `humanizer`
|
||||
- **Outputs:** `copy.md`
|
||||
|
||||
### concept-artist / visual-designer
|
||||
|
||||
Develops the visual identity: mood board, style frames, color palette
|
||||
rationale, typography choices. Produces a `visual-spec.md` that all generators
|
||||
follow. Often produces still reference frames using image-generation APIs or
|
||||
local skills.
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** `kanban-worker` plus any project-specific design skill —
|
||||
`claude-design` (UI/web), `sketch` (quick mockup variants),
|
||||
`popular-web-designs` (matching known web aesthetic), `pixel-art` (retro),
|
||||
`ascii-art` (terminal/retro), `excalidraw` (hand-drawn frames),
|
||||
`design-md` (text-based design docs)
|
||||
- **Outputs:** `visual-spec.md`, `taste/style-frames/*.png`
|
||||
|
||||
### storyboarder
|
||||
|
||||
Maps the brief to a beat-by-beat shot list with timing. Critical for narrative
|
||||
film and music video. Often pairs with a diagramming tool.
|
||||
|
||||
- **Toolsets:** kanban, file
|
||||
- **Skills:** `kanban-worker` plus a diagram skill — `excalidraw` (sketch),
|
||||
`architecture-diagram` (technical/system), `concept-diagrams` (educational/
|
||||
scientific)
|
||||
- **Outputs:** `storyboard.md` with one row per scene/shot, optional
|
||||
storyboard sketches
|
||||
|
||||
### cinematographer / dp
|
||||
|
||||
Designs the visual language: framing, color, motion, transitions. Reviews
|
||||
generator output for visual consistency. Hands off per-scene `VISUAL_SPEC.md`.
|
||||
|
||||
- **Toolsets:** kanban, terminal, file, video, vision
|
||||
- **Skills:** `kanban-worker` plus the visual skill that matches the project
|
||||
(e.g., `ascii-video` for ASCII work, `manim-video` for explainers,
|
||||
`touchdesigner-mcp` for real-time visuals, etc.)
|
||||
- **Outputs:** `scenes/scene-NN/VISUAL_SPEC.md`, review comments on renderer
|
||||
tasks
|
||||
- **Reviews via:** `video_analyze` (sends full clip to multimodal LLM for
|
||||
native review), `vision_analyze` for spot-checking frames, ffprobe summaries
|
||||
|
||||
## Production roles
|
||||
|
||||
### renderer (generic)
|
||||
|
||||
A worker that produces visual content for one or more scenes. Loaded with
|
||||
whichever creative skill fits the scene's style. Multiple renderers can run in
|
||||
parallel, each pinned to a different skill via `always_load` in their profile
|
||||
or `--skill` on the task.
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** one creative skill (see specialized variants below)
|
||||
- **Outputs:** `scenes/scene-NN/clip.mp4`
|
||||
|
||||
### Specialized renderer variants
|
||||
|
||||
When scenes need very different tools, create specialized renderer profiles
|
||||
instead of overloading one. Each loads a different creative skill.
|
||||
|
||||
| Variant | Skill | Best for |
|
||||
|---------|-------|----------|
|
||||
| `renderer-ascii` | `ascii-video` | Terminal aesthetic, retro pixel, audio-reactive grid, video-to-ASCII conversion |
|
||||
| `renderer-manim` | `manim-video` | Math, algorithms, 3Blue1Brown-style explainers, equation derivations |
|
||||
| `renderer-p5js` | `p5js` | Generative art, particles, shaders, organic motion, web-canvas content |
|
||||
| `renderer-comfyui` | `comfyui` | AI-generated stills + video using local ComfyUI workflows (img-to-img, img-to-video, etc.) |
|
||||
| `renderer-touchdesigner` | `touchdesigner-mcp` | Real-time, audio-reactive, installation art, VJ-style content |
|
||||
| `renderer-3d` | `blender-mcp` *(optional)* | 3D modeling, animation, photoreal environments, character animation |
|
||||
| `renderer-pixel` | `pixel-art` | Retro game aesthetic with era-correct palettes |
|
||||
| `renderer-comic` | `baoyu-comic` | Knowledge-comic style narrative scenes |
|
||||
| `renderer-meme` | `meme-generation` *(optional)* | Meme-style stills for satirical/social content |
|
||||
| `renderer-procedural` | (none — Python with PIL + ffmpeg directly) | Custom procedural content where no skill fits |
|
||||
| `renderer-video` | (external image-to-video API: Runway / Kling / Luma) | Animating still images in narrative film |
|
||||
| `renderer-motion-graphics` | (external — Remotion CLI) | Motion graphics, kinetic typography, UI animations |
|
||||
|
||||
For external-API renderers, the profile holds the API client logic; only
|
||||
`kanban-worker` is loaded, plus the terminal toolset and the API key.
|
||||
|
||||
### image-generator
|
||||
|
||||
Specifically for text-to-image generation. Often produces stills that go to
|
||||
`renderer-video` for animation.
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** `kanban-worker`, optionally `comfyui` (drives a local
|
||||
ComfyUI install for image generation)
|
||||
- **External APIs (alternative to local ComfyUI):** FAL, Replicate, OpenAI
|
||||
Images, Midjourney
|
||||
- **Outputs:** `scenes/scene-NN/stills/*.png`
|
||||
|
||||
### image-to-video-generator
|
||||
|
||||
Takes still images and animates them via Runway/Kling/Luma APIs, or via
|
||||
ComfyUI's image-to-video workflows locally. Almost always follows
|
||||
`image-generator` in narrative film pipelines.
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** `kanban-worker`, optionally `comfyui` (for local image-to-video
|
||||
workflows like AnimateDiff or WAN)
|
||||
- **External APIs:** Runway, Kling, Luma, Pika
|
||||
- **Outputs:** `scenes/scene-NN/clip.mp4`
|
||||
|
||||
### music-supervisor
|
||||
|
||||
Sources, analyzes, and prepares the music track. For music videos, also
|
||||
produces a beat/BPM map and key-moment timestamps. Uses `songsee` for
|
||||
spectrograms when the editor or renderer needs a visual reference of the
|
||||
audio's energy.
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** `kanban-worker`, `songsee` (audio visualization), plus one of:
|
||||
- `songwriting-and-ai-music` — when commissioning lyrics + Suno prompts
|
||||
- `heartmula` — when generating music with the open-source local model
|
||||
- `spotify` — when sourcing existing tracks
|
||||
- **Outputs:** `audio/track.mp3`, `audio/beats.json`, optional
|
||||
`audio/track-spectrogram.png`
|
||||
|
||||
### voice-talent / narrator
|
||||
|
||||
Generates voiceover audio. Calls a TTS API directly; no Hermes skill required
|
||||
beyond `kanban-worker`. The user can also supply pre-recorded VO instead of
|
||||
generation.
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** `kanban-worker`
|
||||
- **External APIs:** ElevenLabs, OpenAI TTS, etc.
|
||||
- **Outputs:** `audio/voiceover/line-NN.mp3`, `audio/voiceover/timeline.mp3`
|
||||
|
||||
### foley / sfx-designer
|
||||
|
||||
Sound effects and ambient design. Often optional unless the brief calls for
|
||||
sound design specifically.
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** `kanban-worker`, `songsee` for audio-feature visualization when
|
||||
designing to a track
|
||||
- **Outputs:** `audio/sfx/*.mp3`
|
||||
|
||||
## Post-production roles
|
||||
|
||||
### editor
|
||||
|
||||
Assembles the final cut from clips. Uses ffmpeg for stitching, fades,
|
||||
transitions. Reviews each clip for pacing and quality before assembly.
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** `kanban-worker`
|
||||
- **External tools:** ffmpeg, ffprobe
|
||||
- **Outputs:** `output/final.mp4`, `output/final-noaudio.mp4`
|
||||
|
||||
### colorist
|
||||
|
||||
Color grading. Usually optional — if the renderers already produce
|
||||
brand-consistent output and the editor just stitches, the colorist is overkill.
|
||||
Worth including for narrative film with hero shots.
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** `kanban-worker`
|
||||
- **Outputs:** `output/final-graded.mp4`
|
||||
|
||||
### audio-mixer
|
||||
|
||||
Mixes voiceover + music + SFX into a final audio track. Sets levels, ducks
|
||||
music under VO, normalizes loudness (LUFS).
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** `kanban-worker`
|
||||
- **External tools:** ffmpeg with `loudnorm` filter, optional `sox`
|
||||
- **Outputs:** `audio/final-mix.mp3`
|
||||
|
||||
### captioner
|
||||
|
||||
Burns subtitles into the video, generates SRT, handles accessibility. Can also
|
||||
generate captions from audio via Whisper.
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** `kanban-worker`
|
||||
- **External tools:** Whisper (CLI or API), ffmpeg subtitle filters
|
||||
- **Outputs:** `output/captions.srt`, `output/final-captioned.mp4`
|
||||
|
||||
### masterer
|
||||
|
||||
Final encode + format variants. Produces deliverables for each platform target
|
||||
(square for IG, vertical for TikTok, full HD for YouTube, etc.).
|
||||
|
||||
- **Toolsets:** kanban, terminal, file
|
||||
- **Skills:** `kanban-worker`
|
||||
- **Outputs:** `output/final-1080.mp4`, `output/final-9x16.mp4`, etc.
|
||||
|
||||
## QA roles
|
||||
|
||||
### reviewer
|
||||
|
||||
A neutral quality gate. Reads the brief, watches the cut, comments
|
||||
specifically on what's off (pacing, sync, brand alignment, technical
|
||||
quality). Distinct from the cinematographer (who reviews visuals during
|
||||
production) and the editor (who reviews for assembly).
|
||||
|
||||
- **Toolsets:** kanban, terminal, file, video, vision
|
||||
- **Skills:** `kanban-worker`
|
||||
- **Review tools:** `video_analyze` (native clip review via multimodal LLM),
|
||||
`vision_analyze` (frame/thumbnail review), ffprobe
|
||||
- **Outputs:** `review-notes.md`, comments on tasks
|
||||
|
||||
### brand-cop
|
||||
|
||||
Reviews specifically for brand compliance — colors, typography, voice. Use
|
||||
when the brand guidelines are detailed and a generic reviewer might miss
|
||||
violations.
|
||||
|
||||
- **Toolsets:** kanban, file
|
||||
- **Skills:** `kanban-worker`
|
||||
- **Outputs:** comments + `brand-review.md`
|
||||
|
||||
## Composing teams — heuristics
|
||||
|
||||
- **Always:** director + at least one renderer + editor.
|
||||
- **Add writer** if scripted dialogue / narration / on-screen text exceeds a
|
||||
tagline.
|
||||
- **Add storyboarder** if the brief has more than 5 distinct beats and the
|
||||
director hasn't already laid out a beat list.
|
||||
- **Add cinematographer** if multiple renderer instances need consistent
|
||||
visual language. (For a single-tool video, the renderer's own skill spec
|
||||
is enough.)
|
||||
- **Add image-generator + image-to-video-generator pair** for narrative film
|
||||
with photorealistic visuals.
|
||||
- **Add music-supervisor** when music is provided and rhythm matters
|
||||
(music videos always; explainers sometimes).
|
||||
- **Add voice-talent** for any voiceover / narrative dialogue.
|
||||
- **Add audio-mixer** when there are 2+ audio sources (VO + music, music + SFX).
|
||||
- **Add captioner** for accessibility-priority projects (explainer, tutorial,
|
||||
any platform that defaults to muted playback).
|
||||
- **Add reviewer** for high-stakes projects. Skip for quick experimental loops.
|
||||
- **Add masterer** when multiple platform deliverables are needed.
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
- **One renderer doing everything.** If scenes use very different tools
|
||||
(ASCII + 3D + motion graphics), use specialized renderer variants. The
|
||||
renderer loads ONE creative skill at a time; mixing styles in a single
|
||||
renderer causes thrashing.
|
||||
- **A separate profile per scene.** No. Profiles are per-role, not per-scene.
|
||||
Eight scenes use one or two renderer profiles, not eight.
|
||||
- **A "general" profile that does everything.** Worse than no specialization.
|
||||
The kanban routing breaks down if every task fits every profile.
|
||||
- **No reviewer for important deliverables.** Saves an hour of pipeline time
|
||||
but ships flaws.
|
||||
@@ -0,0 +1,317 @@
|
||||
# Tool Matrix — Skills + Toolsets per Role
|
||||
|
||||
Maps each role archetype to the Hermes skills it should `always_load` and the
|
||||
toolsets it needs. Only references skills that ship in the public hermes-agent
|
||||
repository (under `skills/` or `optional-skills/`). External APIs and CLIs are
|
||||
called from the terminal toolset; they don't appear in `always_load`.
|
||||
|
||||
## Hermes skills relevant to video production
|
||||
|
||||
### Visual / rendering skills (`hermes-agent/skills/creative/`)
|
||||
|
||||
| Skill | What it does | Best fit for |
|
||||
|-------|--------------|--------------|
|
||||
| `ascii-video` | Production pipeline for ASCII art video — generative, audio-reactive, video-to-ASCII | Renderer for ASCII / terminal / retro pixel content; cinematographer for ASCII projects |
|
||||
| `ascii-art` | Static ASCII art generation | Concept artist for ASCII style frames; secondary tool for ASCII renderer |
|
||||
| `manim-video` | Manim CE animations — math, algorithms, 3Blue1Brown-style explainers | Renderer for math, algorithm walkthroughs, technical concept explainers |
|
||||
| `p5js` | p5.js sketches — generative art, shaders, interactive, 3D | Renderer for generative art, particle systems, organic motion, web-canvas content |
|
||||
| `comfyui` | Generate images, video, audio with ComfyUI workflows (image-to-image, image-to-video, etc.) | image-generator, image-to-video-generator, or general renderer for AI-generated content |
|
||||
| `touchdesigner-mcp` | Control a running TouchDesigner instance — real-time visuals, audio-reactive installation art, VJ | Renderer for real-time/audio-reactive content; installation art; live performance |
|
||||
| `blender-mcp` *(optional)* | Control Blender 4.3+ via MCP — 3D modeling, animation, rendering | Renderer for 3D scenes, photoreal environments, character animation |
|
||||
| `pixel-art` | Pixel art with era palettes (NES, Game Boy, PICO-8) | Renderer for retro game aesthetic; concept artist for pixel-style frames |
|
||||
| `baoyu-comic` | Knowledge-comic generation (educational, biography, tutorial) | Renderer for comic-style narrative; explainer in panel form |
|
||||
| `baoyu-infographic` | Infographic generation | Renderer for data-driven explainer scenes |
|
||||
| `meme-generation` *(optional)* | Generate meme images by overlaying text on templates | Generator for satirical/social content; meme-style stills |
|
||||
|
||||
### Design / pre-production skills (`hermes-agent/skills/creative/`)
|
||||
|
||||
| Skill | What it does | Best fit for |
|
||||
|-------|--------------|--------------|
|
||||
| `claude-design` | Design one-off HTML artifacts (landing, deck, prototype) | Concept artist for product video style frames; storyboarder for UI-heavy content |
|
||||
| `design-md` | Design markdown docs | Concept artist documenting visual specs |
|
||||
| `popular-web-designs` | Reference patterns for popular web designs | Concept artist; cinematographer when matching a known UI aesthetic |
|
||||
| `sketch` | Throwaway HTML mockups (2-3 design variants to compare) | Concept artist exploring directions; storyboarder for UI flows |
|
||||
| `excalidraw` | Excalidraw-style hand-drawn diagrams | Storyboarder; concept artist for sketch-style frames |
|
||||
| `architecture-diagram` | Software architecture diagrams | Storyboarder for technical content; explainer scenes about systems |
|
||||
| `concept-diagrams` *(optional)* | Flat, minimal SVG diagrams (educational visual language; physics, chemistry, math, anatomy, etc.) | Renderer / storyboarder for explainer scenes with clean educational diagrams |
|
||||
| `pretext` | Mathematical/scientific content authoring | Writer / cinematographer for technical-explainer pretexts |
|
||||
| `creative-ideation` | Constraint-driven project ideation | Director / cinematographer when the brief is wide-open and needs framing |
|
||||
| `humanizer` | Strip AI-isms from text, add real voice | Writer / copywriter post-process to avoid AI-tells in scripts and VO copy |
|
||||
|
||||
### Audio / media skills (`hermes-agent/skills/creative/` + `skills/media/`)
|
||||
|
||||
| Skill | What it does | Best fit for |
|
||||
|-------|--------------|--------------|
|
||||
| `songwriting-and-ai-music` | Songwriting craft + Suno prompt patterns | Music supervisor when commissioning a track via Suno |
|
||||
| `heartmula` | Open-source music generation (Apache-2.0, Suno-like) | Music supervisor generating bespoke tracks without external APIs |
|
||||
| `songsee` | Spectrograms, mel/chroma/MFCC of audio files | Music supervisor analyzing tracks; foley-designer designing to a beat; editor visualizing a mix |
|
||||
| `spotify` | Spotify control — play, search, queue, manage playlists | Music supervisor sourcing existing tracks; reference research |
|
||||
| `youtube-content` | Fetch transcripts + transform to chapters/summaries/posts | Documentary cut, content adaptation, research for explainers |
|
||||
| `gif-search` | Find existing GIFs | Editor / concept artist sourcing references |
|
||||
| `gifs` | GIF tooling | Masterer producing GIF deliverables |
|
||||
|
||||
### Kanban infrastructure (`hermes-agent/skills/devops/`)
|
||||
|
||||
| Skill | What it does | When to load |
|
||||
|-------|--------------|--------------|
|
||||
| `kanban-orchestrator` | Decomposition playbook + anti-temptation rules for orchestrator profiles | Director only |
|
||||
| `kanban-worker` | Pitfalls, examples, edge cases for kanban workers (deeper than auto-injected guidance) | Any profile — load when handling tricky multi-step workflows |
|
||||
|
||||
The kanban plugin auto-injects baseline orchestration guidance into every
|
||||
worker's system prompt — the `kanban_create` fan-out pattern, claim/handoff
|
||||
lifecycle, and the "decompose, don't execute" rule for orchestrators.
|
||||
`kanban-orchestrator` and `kanban-worker` are deeper playbooks loaded when a
|
||||
profile needs them.
|
||||
|
||||
## External tools (called from terminal toolset)
|
||||
|
||||
These are **not** Hermes skills but external CLIs / APIs that profiles invoke.
|
||||
They don't appear in `always_load`; instead the role's terminal commands hit
|
||||
them directly.
|
||||
|
||||
| Tool | What it does | Profile that uses it |
|
||||
|------|--------------|----------------------|
|
||||
| `ffmpeg` | Video / audio encode, splice, mux | renderer, editor, audio-mixer, masterer |
|
||||
| `ffprobe` | Inspect media | All media-touching profiles |
|
||||
| Whisper (CLI or API) | Speech-to-text for captions | captioner |
|
||||
| Text-to-image API (FAL / Replicate / OpenAI / Midjourney) | Stills generation | image-generator (alternative to local `comfyui`) |
|
||||
| Image-to-video API (Runway / Kling / Luma / Pika) | Animate stills | image-to-video-generator |
|
||||
| Text-to-speech API (ElevenLabs / OpenAI TTS / etc.) | Voiceover generation | voice-talent |
|
||||
| Suno API or web | Track composition (paired with `songwriting-and-ai-music`) | music-supervisor |
|
||||
| Remotion CLI (`npx remotion render`) | React-based motion graphics | renderer-motion-graphics |
|
||||
| Manim CE (`manim`) | Math animation render (driven by `manim-video` skill's recipes) | renderer-manim |
|
||||
| Blender (`blender -b`) | 3D rendering (alternative to `blender-mcp`) | renderer-3d |
|
||||
|
||||
## Built-in Hermes tools for media review
|
||||
|
||||
These are native Hermes tools — not invoked via terminal but through their own
|
||||
toolsets. Enable them per-profile by adding the toolset to the profile config.
|
||||
|
||||
| Tool | Toolset | What it does | Profile that uses it |
|
||||
|------|---------|--------------|----------------------|
|
||||
| `video_analyze` | `video` (opt-in — `hermes tools enable video`) | Native video understanding — sends full clip to a multimodal LLM (Gemini via OpenRouter) for review without frame extraction. Supports mp4, webm, mov, avi, mkv. 50 MB cap. Model: `AUXILIARY_VIDEO_MODEL` env → `AUXILIARY_VISION_MODEL` fallback. | reviewer, cinematographer, editor |
|
||||
| `vision_analyze` | `vision` (core — enabled by default) | Image/frame analysis — review stills, thumbnails, exported frames. Already available to all profiles without opt-in. | reviewer, cinematographer, concept-artist |
|
||||
|
||||
## Standard toolset configurations per role
|
||||
|
||||
### director
|
||||
|
||||
```yaml
|
||||
toolsets:
|
||||
- kanban
|
||||
- terminal
|
||||
- file
|
||||
skills:
|
||||
always_load:
|
||||
- kanban-orchestrator
|
||||
```
|
||||
|
||||
The director's terminal access is conventional but the SOUL.md rules forbid
|
||||
execution. Audit logs catch violations.
|
||||
|
||||
### writer / copywriter
|
||||
|
||||
```yaml
|
||||
toolsets:
|
||||
- kanban
|
||||
- file
|
||||
skills:
|
||||
always_load:
|
||||
- kanban-worker
|
||||
- humanizer # post-process scripts to strip AI-tells
|
||||
```
|
||||
|
||||
No terminal — writers don't need it.
|
||||
|
||||
### concept-artist
|
||||
|
||||
```yaml
|
||||
toolsets:
|
||||
- kanban
|
||||
- terminal
|
||||
- file
|
||||
skills:
|
||||
always_load:
|
||||
- kanban-worker
|
||||
# plus one or more (style-dependent):
|
||||
# - claude-design (UI / web product video)
|
||||
# - sketch (quick mockup variants)
|
||||
# - excalidraw (hand-drawn frames)
|
||||
# - ascii-art (ASCII style frames)
|
||||
# - pixel-art (retro/game aesthetic)
|
||||
# - popular-web-designs (matching known web aesthetic)
|
||||
# - design-md (text-based design docs)
|
||||
```
|
||||
|
||||
### storyboarder
|
||||
|
||||
```yaml
|
||||
toolsets:
|
||||
- kanban
|
||||
- file
|
||||
skills:
|
||||
always_load:
|
||||
- kanban-worker
|
||||
# one of:
|
||||
# - excalidraw (sketch storyboards)
|
||||
# - architecture-diagram (technical/system content)
|
||||
# - concept-diagrams (educational / scientific content)
|
||||
```
|
||||
|
||||
### cinematographer
|
||||
|
||||
```yaml
|
||||
toolsets:
|
||||
- kanban
|
||||
- terminal
|
||||
- file
|
||||
- video # video_analyze — review full clips natively
|
||||
- vision # vision_analyze — review stills / exported frames
|
||||
skills:
|
||||
always_load:
|
||||
- kanban-worker
|
||||
# the visual skill that matches the project, e.g.:
|
||||
# - ascii-video (ASCII projects)
|
||||
# - manim-video (math/explainer)
|
||||
# - p5js (generative)
|
||||
# - comfyui (AI-generated visuals)
|
||||
# - blender-mcp (3D)
|
||||
# - touchdesigner-mcp (real-time/installation)
|
||||
```
|
||||
|
||||
### renderer (specialized variants)
|
||||
|
||||
```yaml
|
||||
toolsets:
|
||||
- kanban
|
||||
- terminal
|
||||
- file
|
||||
skills:
|
||||
always_load:
|
||||
- kanban-worker
|
||||
# ONE skill per renderer variant (or empty for external-API renderers):
|
||||
# - ascii-video (renderer-ascii)
|
||||
# - manim-video (renderer-manim)
|
||||
# - p5js (renderer-p5js)
|
||||
# - comfyui (renderer-comfyui — img/video AI gen)
|
||||
# - touchdesigner-mcp (renderer-touchdesigner)
|
||||
# - blender-mcp (renderer-3d)
|
||||
# - pixel-art (renderer-pixel)
|
||||
# - baoyu-comic (renderer-comic)
|
||||
# - meme-generation (renderer-meme)
|
||||
```
|
||||
|
||||
For external-API renderers (image-to-video-generator using Runway, voice-talent
|
||||
using ElevenLabs, renderer-motion-graphics using Remotion), `always_load` only
|
||||
contains `kanban-worker` — the role's work is API-driven and the API key +
|
||||
terminal commands suffice.
|
||||
|
||||
For multi-skill renderer setups (rare — usually one variant per skill is
|
||||
cleaner) use `--skill <name>` on individual `kanban_create` calls to override
|
||||
which skill loads for that specific task.
|
||||
|
||||
### image-generator / image-to-video-generator / voice-talent
|
||||
|
||||
```yaml
|
||||
toolsets:
|
||||
- kanban
|
||||
- terminal
|
||||
- file
|
||||
skills:
|
||||
always_load:
|
||||
- kanban-worker
|
||||
# for image-generator that drives ComfyUI locally:
|
||||
# - comfyui
|
||||
env_required:
|
||||
# populate based on the chosen API:
|
||||
- FAL_KEY # or REPLICATE_API_TOKEN, OPENAI_API_KEY for image-gen
|
||||
- RUNWAY_API_KEY # or KLING_API_KEY, LUMA_API_KEY for image-to-video
|
||||
- ELEVENLABS_API_KEY # or OPENAI_API_KEY for TTS
|
||||
```
|
||||
|
||||
If the user's setup has ComfyUI installed locally, the `comfyui` skill can
|
||||
replace the external image-gen API entirely (cheaper, more control, supports
|
||||
custom workflows for image-to-video too).
|
||||
|
||||
### music-supervisor
|
||||
|
||||
```yaml
|
||||
toolsets:
|
||||
- kanban
|
||||
- terminal
|
||||
- file
|
||||
skills:
|
||||
always_load:
|
||||
- kanban-worker
|
||||
- songsee # spectrograms / audio analysis
|
||||
# plus (depending on what the project needs):
|
||||
# - songwriting-and-ai-music (commissioning Suno tracks)
|
||||
# - heartmula (commissioning open-source local generation)
|
||||
# - spotify (sourcing existing tracks)
|
||||
```
|
||||
|
||||
### editor / audio-mixer / captioner / masterer
|
||||
|
||||
```yaml
|
||||
toolsets:
|
||||
- kanban
|
||||
- terminal
|
||||
- file
|
||||
- video # video_analyze — editor reviews assembled cuts natively
|
||||
- vision # vision_analyze — spot-check frames
|
||||
skills:
|
||||
always_load:
|
||||
- kanban-worker
|
||||
```
|
||||
|
||||
These are mostly ffmpeg-driven; no special skill needed beyond `kanban-worker`.
|
||||
For captioner add Whisper invocation patterns to the SOUL.md.
|
||||
|
||||
### reviewer / brand-cop
|
||||
|
||||
```yaml
|
||||
toolsets:
|
||||
- kanban
|
||||
- terminal # for media inspection (ffprobe, etc.)
|
||||
- file
|
||||
- video # video_analyze — review full clips natively
|
||||
- vision # vision_analyze — review stills / exported frames
|
||||
skills:
|
||||
always_load:
|
||||
- kanban-worker
|
||||
```
|
||||
|
||||
## API key requirements
|
||||
|
||||
Track these in the project setup. The setup script should verify each required
|
||||
key is present in `~/.hermes/.env` (or macOS Keychain) before firing the kanban.
|
||||
|
||||
| Service | Env var | Used by |
|
||||
|---------|---------|---------|
|
||||
| ElevenLabs | `ELEVENLABS_API_KEY` | voice-talent |
|
||||
| OpenAI | `OPENAI_API_KEY` | image-generator (DALL-E), voice-talent (TTS) |
|
||||
| OpenRouter | `OPENROUTER_API_KEY` | reviewer, cinematographer, editor (`video_analyze` routes through `AUXILIARY_VIDEO_MODEL` → OpenRouter) |
|
||||
| FAL | `FAL_KEY` | image-generator (FAL flux models) |
|
||||
| Replicate | `REPLICATE_API_TOKEN` | image-generator (alternate provider) |
|
||||
| Runway | `RUNWAY_API_KEY` | image-to-video-generator |
|
||||
| Kling | `KLING_API_KEY` | image-to-video-generator (alternate) |
|
||||
| Luma | `LUMA_API_KEY` | image-to-video-generator (alternate) |
|
||||
| Suno | `SUNO_API_KEY` | music-supervisor (paired with `songwriting-and-ai-music`) |
|
||||
| Spotify | `SPOTIFY_CLIENT_ID` + `SPOTIFY_CLIENT_SECRET` | music-supervisor (paired with `spotify` skill) |
|
||||
| Anthropic | `ANTHROPIC_API_KEY` | every Hermes profile (Claude) |
|
||||
|
||||
If a key is missing, prompt the user to add it. Storage methods, in order of
|
||||
preference: macOS Keychain → `~/.hermes/.env` → environment variable.
|
||||
|
||||
## Skill version pinning
|
||||
|
||||
If a specific skill version is desired, pass it via the per-task
|
||||
`--skill <name>=<version>` flag. The default is whatever's installed.
|
||||
|
||||
## Adding a new skill to the matrix
|
||||
|
||||
When a new Hermes-public video skill ships:
|
||||
|
||||
1. Add a row to the relevant table at the top of this file
|
||||
2. If it warrants a specialized renderer variant, add to `role-archetypes.md`
|
||||
3. Update relevant per-style examples in `examples.md`
|
||||
+501
@@ -0,0 +1,501 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Bootstrap a video production kanban from a structured plan JSON.
|
||||
|
||||
Reads a plan.json describing the team + brief, expands templates from
|
||||
../assets/, and writes a setup.sh that creates Hermes profiles and fires the
|
||||
initial kanban task.
|
||||
|
||||
Profile-config patching, SOUL.md-per-profile, TEAM.md task-graph convention,
|
||||
and the `hermes kanban create --workspace dir:` initial-task pattern are
|
||||
adapted from alt-glitch's NousResearch/kanban-video-pipeline.
|
||||
|
||||
Usage:
|
||||
bootstrap_pipeline.py plan.json [--out setup.sh]
|
||||
|
||||
The plan.json schema is documented inline below — see the `validate_plan`
|
||||
function. A minimal example:
|
||||
|
||||
{
|
||||
"title": "Q3 Product Teaser",
|
||||
"slug": "q3-product-teaser",
|
||||
"tenant": "q3-product-teaser",
|
||||
"duration_s": 30,
|
||||
"aspect": "1:1",
|
||||
"resolution": "1080x1080",
|
||||
"fps": 30,
|
||||
"team": [
|
||||
{
|
||||
"profile": "director",
|
||||
"role": "director",
|
||||
"toolsets": ["kanban", "terminal", "file"],
|
||||
"skills": [],
|
||||
"responsibilities": "...",
|
||||
"inputs": "brief.md, TEAM.md, taste/",
|
||||
"outputs": "kanban tasks for the team"
|
||||
},
|
||||
...
|
||||
],
|
||||
"scenes": [
|
||||
{"n": 1, "time": "0:00-0:08", "content": "...", "tool": "renderer-ascii"},
|
||||
...
|
||||
],
|
||||
"audio": {"approach": "voiceover + music bed", "vo": "ElevenLabs Lily",
|
||||
"music": "license-free", "sfx": "n/a"},
|
||||
"deliverables": [
|
||||
{"format": "mp4", "resolution": "1080x1080", "notes": "primary"}
|
||||
],
|
||||
"api_keys_required": ["ELEVENLABS_API_KEY", "OPENROUTER_API_KEY"],
|
||||
"brief_extra": {
|
||||
"concept_one_liner": "...",
|
||||
"emotional_north_star": "...",
|
||||
"visual_refs": "...",
|
||||
"tone": "...",
|
||||
"brand_constraints": "..."
|
||||
}
|
||||
}
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
ASSETS_DIR = Path(__file__).resolve().parent.parent / "assets"
|
||||
|
||||
|
||||
def load_template(name: str) -> str:
|
||||
return (ASSETS_DIR / name).read_text()
|
||||
|
||||
|
||||
PROFILE_NAME_RE = re.compile(r"^[a-z0-9][a-z0-9_-]{0,63}$")
|
||||
SLUG_RE = re.compile(r"^[a-z0-9][a-z0-9-]+$")
|
||||
|
||||
|
||||
def validate_plan(plan: dict) -> list[str]:
|
||||
"""Return a list of validation error strings; empty list = valid."""
|
||||
errors = []
|
||||
required_top = ["title", "slug", "tenant", "duration_s", "aspect",
|
||||
"resolution", "fps", "team", "scenes", "audio",
|
||||
"deliverables"]
|
||||
for k in required_top:
|
||||
if k not in plan:
|
||||
errors.append(f"missing required key: {k}")
|
||||
|
||||
if "team" in plan:
|
||||
if not isinstance(plan["team"], list) or not plan["team"]:
|
||||
errors.append("team must be a non-empty list")
|
||||
else:
|
||||
roles = [t.get("role") for t in plan["team"]]
|
||||
if "director" not in roles:
|
||||
errors.append("team must include a director role")
|
||||
seen_profiles = set()
|
||||
for i, t in enumerate(plan["team"]):
|
||||
for k in ["profile", "role", "toolsets", "skills",
|
||||
"responsibilities"]:
|
||||
if k not in t:
|
||||
errors.append(f"team[{i}] missing {k}")
|
||||
# Profile name must match Hermes's regex (lowercase
|
||||
# alphanumeric + hyphens + underscores, up to 64 chars).
|
||||
if "profile" in t:
|
||||
if not PROFILE_NAME_RE.match(t["profile"]):
|
||||
errors.append(
|
||||
f"team[{i}].profile {t['profile']!r} must match "
|
||||
f"[a-z0-9][a-z0-9_-]{{0,63}} per Hermes profile rules"
|
||||
)
|
||||
if t["profile"] in seen_profiles:
|
||||
errors.append(
|
||||
f"team[{i}].profile {t['profile']!r} is duplicated"
|
||||
)
|
||||
seen_profiles.add(t["profile"])
|
||||
# Toolsets / skills must be lists, not strings.
|
||||
if "toolsets" in t and not isinstance(t["toolsets"], list):
|
||||
errors.append(
|
||||
f"team[{i}].toolsets must be a list of strings"
|
||||
)
|
||||
if "skills" in t and not isinstance(t["skills"], list):
|
||||
errors.append(
|
||||
f"team[{i}].skills must be a list of strings"
|
||||
)
|
||||
|
||||
if "slug" in plan:
|
||||
if not SLUG_RE.match(plan["slug"]):
|
||||
errors.append("slug must be lowercase, hyphenated, "
|
||||
"starting with [a-z0-9]")
|
||||
|
||||
return errors
|
||||
|
||||
|
||||
def render_brief(plan: dict) -> str:
|
||||
"""Render brief.md from the plan."""
|
||||
tmpl = load_template("brief.md.tmpl")
|
||||
extra = plan.get("brief_extra", {})
|
||||
|
||||
# Scene table rows
|
||||
scene_rows = []
|
||||
for s in plan["scenes"]:
|
||||
scene_rows.append(
|
||||
f"| {s.get('n', '?')} | {s.get('time', '?')} | "
|
||||
f"{s.get('content', '')} | {s.get('tool', '')} | "
|
||||
f"{s.get('audio', '')} | {s.get('notes', '')} |"
|
||||
)
|
||||
scene_table = "\n".join(scene_rows) if scene_rows else "_(none yet)_"
|
||||
|
||||
# Deliverable rows
|
||||
deliv_rows = []
|
||||
for d in plan["deliverables"]:
|
||||
deliv_rows.append(
|
||||
f"| {d.get('format', '?')} | {d.get('resolution', '?')} | "
|
||||
f"{d.get('notes', '')} |"
|
||||
)
|
||||
deliv_table = "\n".join(deliv_rows) if deliv_rows else "_(none)_"
|
||||
|
||||
# Replacements (single-pass)
|
||||
replacements = {
|
||||
"TITLE": plan["title"],
|
||||
"SLUG": plan["slug"],
|
||||
"TENANT": plan["tenant"],
|
||||
"WORKSPACE": f"~/projects/video-pipeline/{plan['slug']}",
|
||||
"ONE_LINE_PITCH": extra.get("concept_one_liner", "_(TBD)_"),
|
||||
"EMOTIONAL_NORTH_STAR": extra.get("emotional_north_star", "_(TBD)_"),
|
||||
"DURATION_S": str(plan["duration_s"]),
|
||||
"ASPECT": plan["aspect"],
|
||||
"RESOLUTION": plan["resolution"],
|
||||
"FPS": str(plan["fps"]),
|
||||
"PLATFORMS": extra.get("platforms", "_(TBD)_"),
|
||||
"DEADLINE": extra.get("deadline", "_(none)_"),
|
||||
"QUALITY_BAR": extra.get("quality_bar", "polished"),
|
||||
"VISUAL_REFS": extra.get("visual_refs", "_(none)_"),
|
||||
"TONE": extra.get("tone", "_(TBD)_"),
|
||||
"BRAND_CONSTRAINTS": extra.get("brand_constraints", "_(none)_"),
|
||||
"AESTHETIC_RULES": extra.get("aesthetic_rules", "_(TBD)_"),
|
||||
"AUDIO_APPROACH": plan["audio"].get("approach", "_(TBD)_"),
|
||||
"VO_DETAILS": plan["audio"].get("vo", "_(n/a)_"),
|
||||
"MUSIC_DETAILS": plan["audio"].get("music", "_(n/a)_"),
|
||||
"SFX_DETAILS": plan["audio"].get("sfx", "_(n/a)_"),
|
||||
"PRIMARY_FORMAT": plan["deliverables"][0]["format"],
|
||||
"PRIMARY_RES": plan["deliverables"][0]["resolution"],
|
||||
"ALT_FORMAT_1": (plan["deliverables"][1]["format"]
|
||||
if len(plan["deliverables"]) > 1 else "_(none)_"),
|
||||
"ALT_RES_1": (plan["deliverables"][1]["resolution"]
|
||||
if len(plan["deliverables"]) > 1 else ""),
|
||||
"ALT_NOTES_1": (plan["deliverables"][1].get("notes", "")
|
||||
if len(plan["deliverables"]) > 1 else ""),
|
||||
"API_KEYS_REQUIRED": ", ".join(plan.get("api_keys_required", [])) or "none",
|
||||
"EXT_DEPS": extra.get("ext_deps", "ffmpeg, Python 3.11+"),
|
||||
"SOURCE_ASSETS": extra.get("source_assets", "_(none)_"),
|
||||
}
|
||||
out = tmpl
|
||||
for k, v in replacements.items():
|
||||
out = out.replace("{{" + k + "}}", str(v))
|
||||
|
||||
# Scene + deliv tables: replace the placeholder row in the template
|
||||
out = re.sub(
|
||||
r"\|\s*1\s*\|\s*0:00–0:0X.+?\n\|\s*2\s*\|.+?\n",
|
||||
scene_table + "\n",
|
||||
out, flags=re.DOTALL,
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def render_team_md(plan: dict) -> str:
|
||||
"""Render TEAM.md from the team list + scene → tool mapping."""
|
||||
lines = [f"# Team & Task Graph — {plan['title']}", "", "## Team", ""]
|
||||
for t in plan["team"]:
|
||||
skills = (
|
||||
f"loads `{', '.join(t['skills'])}`"
|
||||
if t["skills"] else "no skills required"
|
||||
)
|
||||
lines.append(
|
||||
f"- `{t['profile']}` — {t['responsibilities']} ({skills})"
|
||||
)
|
||||
lines.extend(["", "## Task Graph", "", "```"])
|
||||
|
||||
# Build a simple task graph based on conventions
|
||||
profiles_by_role = {t["role"]: t["profile"] for t in plan["team"]}
|
||||
director = profiles_by_role.get("director", "director")
|
||||
lines.append(f"T0 {director} — decompose")
|
||||
|
||||
next_id = 1
|
||||
parents_for_renderer: list[str] = ["T0"]
|
||||
|
||||
if "cinematographer" in profiles_by_role:
|
||||
cid = f"T{next_id}"
|
||||
lines.append(
|
||||
f"{cid:5} {profiles_by_role['cinematographer']} — visual spec for all scenes (parent: T0)"
|
||||
)
|
||||
parents_for_renderer = [cid]
|
||||
next_id += 1
|
||||
|
||||
if "music-supervisor" in profiles_by_role:
|
||||
cid = f"T{next_id}"
|
||||
lines.append(
|
||||
f"{cid:5} {profiles_by_role['music-supervisor']} — track analysis + beats.json (parent: T0)"
|
||||
)
|
||||
next_id += 1
|
||||
ms_id = cid
|
||||
else:
|
||||
ms_id = None
|
||||
|
||||
# Scenes
|
||||
scene_ids = []
|
||||
for s in plan["scenes"]:
|
||||
cid = f"T{next_id}"
|
||||
renderer_profile = s.get("tool") or "renderer"
|
||||
# Lookup the actual profile name
|
||||
for t in plan["team"]:
|
||||
if t["role"] == renderer_profile or t["profile"] == renderer_profile:
|
||||
renderer_profile = t["profile"]
|
||||
break
|
||||
parents = parents_for_renderer + ([ms_id] if ms_id else [])
|
||||
parent_str = ", ".join(parents)
|
||||
lines.append(
|
||||
f"{cid:5} {renderer_profile} — scene {s.get('n', '?')}: "
|
||||
f"{s.get('content', '')[:50]} (parents: {parent_str})"
|
||||
)
|
||||
scene_ids.append(cid)
|
||||
next_id += 1
|
||||
|
||||
# VO + audio mix
|
||||
if "voice-talent" in profiles_by_role:
|
||||
vo_id = f"T{next_id}"
|
||||
lines.append(f"{vo_id:5} {profiles_by_role['voice-talent']} — narration (parent: T0)")
|
||||
next_id += 1
|
||||
else:
|
||||
vo_id = None
|
||||
|
||||
if "audio-mixer" in profiles_by_role:
|
||||
am_id = f"T{next_id}"
|
||||
am_parents = [p for p in [ms_id, vo_id] if p]
|
||||
lines.append(
|
||||
f"{am_id:5} {profiles_by_role['audio-mixer']} — mix audio (parents: {', '.join(am_parents)})"
|
||||
)
|
||||
next_id += 1
|
||||
else:
|
||||
am_id = None
|
||||
|
||||
# Editor
|
||||
if "editor" in profiles_by_role:
|
||||
ed_id = f"T{next_id}"
|
||||
ed_parents = scene_ids + [p for p in [am_id, vo_id, ms_id] if p and p not in scene_ids]
|
||||
lines.append(
|
||||
f"{ed_id:5} {profiles_by_role['editor']} — assemble + mux (parents: {', '.join(ed_parents)})"
|
||||
)
|
||||
next_id += 1
|
||||
else:
|
||||
ed_id = None
|
||||
|
||||
# Captioner
|
||||
if "captioner" in profiles_by_role and ed_id:
|
||||
cap_id = f"T{next_id}"
|
||||
lines.append(
|
||||
f"{cap_id:5} {profiles_by_role['captioner']} — SRT + burn (parent: {ed_id})"
|
||||
)
|
||||
next_id += 1
|
||||
last = cap_id
|
||||
else:
|
||||
last = ed_id
|
||||
|
||||
# Reviewer
|
||||
if "reviewer" in profiles_by_role and last:
|
||||
rv_id = f"T{next_id}"
|
||||
lines.append(
|
||||
f"{rv_id:5} {profiles_by_role['reviewer']} — final QA (parent: {last})"
|
||||
)
|
||||
|
||||
lines.append("```")
|
||||
lines.extend([
|
||||
"",
|
||||
"## Per-task workspace requirement",
|
||||
"",
|
||||
f"All `kanban_create` calls MUST pass:",
|
||||
f"```",
|
||||
f'workspace_kind="dir"',
|
||||
f'workspace_path="$HOME/projects/video-pipeline/{plan["slug"]}"',
|
||||
f'tenant="{plan["tenant"]}"',
|
||||
f"```",
|
||||
])
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def render_setup_sh(plan: dict, brief_md: str, team_md: str) -> str:
|
||||
"""Render setup.sh from the plan."""
|
||||
tmpl = load_template("setup.sh.tmpl")
|
||||
|
||||
# API key checks
|
||||
key_checks = []
|
||||
for key in plan.get("api_keys_required", []):
|
||||
key_checks.append(f'check_key {key} hermes {key} || exit 1')
|
||||
key_checks_str = "\n".join(key_checks) if key_checks else "# (no API keys required)"
|
||||
|
||||
# Scene dirs
|
||||
scene_dir_lines = []
|
||||
for s in plan["scenes"]:
|
||||
n = s.get("n", "?")
|
||||
scene_dir_lines.append(f'mkdir -p "$WORKSPACE/scenes/scene-{n:02d}"/checkpoints')
|
||||
scene_dirs = "\n".join(scene_dir_lines) if scene_dir_lines else ""
|
||||
|
||||
# Profile create
|
||||
profile_creates = []
|
||||
for t in plan["team"]:
|
||||
profile_creates.append(
|
||||
f'hermes profile create {t["profile"]} --clone 2>/dev/null || true'
|
||||
)
|
||||
|
||||
# Profile config — emit JSON arrays so the bash function can pass them
|
||||
# safely through to the Python YAML patcher.
|
||||
profile_configs = []
|
||||
for t in plan["team"]:
|
||||
ts_json = json.dumps(t["toolsets"])
|
||||
sk_json = json.dumps(t["skills"])
|
||||
# Use single-quoted bash strings; JSON only contains "/[/], no single
|
||||
# quotes, so this is safe.
|
||||
profile_configs.append(
|
||||
f"configure_profile {t['profile']!r} {ts_json!r} {sk_json!r}"
|
||||
)
|
||||
|
||||
# SOUL writes — uses heredocs per profile
|
||||
soul_writes = []
|
||||
for t in plan["team"]:
|
||||
soul_writes.append(
|
||||
f'cat > "$HOME/.hermes/profiles/{t["profile"]}/SOUL.md" <<\'SOUL_EOF\'\n'
|
||||
f"{render_soul_md(t, plan)}\n"
|
||||
f"SOUL_EOF\n"
|
||||
f'echo " ✓ SOUL.md for {t["profile"]}"'
|
||||
)
|
||||
|
||||
# Taste writes (placeholder; real content optional)
|
||||
taste_writes = (
|
||||
'cat > "$WORKSPACE/taste/brand-guide.md" <<\'TASTE_EOF\'\n'
|
||||
'# Brand Guide\n\n'
|
||||
'_(Populate with project-specific colors, typography, motion rules)_\n'
|
||||
'TASTE_EOF\n'
|
||||
'cat > "$WORKSPACE/taste/emotional-dna.md" <<\'DNA_EOF\'\n'
|
||||
'# Emotional DNA\n\n'
|
||||
'_(What this piece should FEEL like — populate from the brief.)_\n'
|
||||
'DNA_EOF'
|
||||
)
|
||||
|
||||
# Asset copies — leave empty by default; user fills in
|
||||
asset_copies = "# Add cp/rsync commands here for any provided assets"
|
||||
|
||||
out = tmpl
|
||||
out = out.replace("{{TITLE}}", plan["title"])
|
||||
out = out.replace("{{SLUG}}", plan["slug"])
|
||||
out = out.replace("{{TENANT}}", plan["tenant"])
|
||||
out = out.replace("{{WORKSPACE}}", f"~/projects/video-pipeline/{plan['slug']}")
|
||||
out = out.replace("{{KEY_CHECKS}}", key_checks_str)
|
||||
out = out.replace("{{SCENE_DIRS}}", scene_dirs)
|
||||
out = out.replace("{{PROFILE_CREATE_COMMANDS}}", "\n".join(profile_creates))
|
||||
out = out.replace("{{PROFILE_CONFIG_COMMANDS}}", "\n".join(profile_configs))
|
||||
out = out.replace("{{SOUL_WRITES}}", "\n".join(soul_writes))
|
||||
out = out.replace("{{BRIEF_CONTENTS}}", brief_md)
|
||||
out = out.replace("{{TEAM_CONTENTS}}", team_md)
|
||||
out = out.replace("{{TASTE_WRITES}}", taste_writes)
|
||||
out = out.replace("{{ASSET_COPIES}}", asset_copies)
|
||||
|
||||
return out
|
||||
|
||||
|
||||
def render_soul_md(team_member: dict, plan: dict) -> str:
|
||||
"""Render a profile's SOUL.md from a team member dict + plan context."""
|
||||
tmpl = load_template("soul.md.tmpl")
|
||||
role = team_member["role"]
|
||||
|
||||
common_rules = (
|
||||
"- **Read the brief and team graph** before doing anything else.\n"
|
||||
"- **Pass `workspace_kind=\"dir\"` and `workspace_path` on every "
|
||||
"`kanban_create` call.** This keeps the team in one shared workspace.\n"
|
||||
f"- **Use tenant `{plan['tenant']}`** on every kanban call.\n"
|
||||
"- **Write outputs to predictable paths.** Other profiles depend on "
|
||||
"your filename conventions.\n"
|
||||
"- **Emit heartbeats** during long-running work. Renderers should "
|
||||
"report frame counts; editors should report assembly progress.\n"
|
||||
)
|
||||
|
||||
if role == "director":
|
||||
common_rules += (
|
||||
"- **Do not execute the work yourself.** For every concrete task, "
|
||||
"create a kanban task and assign it to the appropriate profile.\n"
|
||||
"- **Decompose, route, comment, approve — that's the whole job.**\n"
|
||||
"- **Read TEAM.md** for the canonical task graph. Do not invent "
|
||||
"new roles unless the brief truly demands it.\n"
|
||||
"- **Load the `kanban-orchestrator` skill** for the deeper "
|
||||
"decomposition playbook beyond the auto-injected baseline.\n"
|
||||
)
|
||||
|
||||
common_commands = (
|
||||
"```bash\n"
|
||||
"# Inspect a clip\n"
|
||||
"ffprobe -v quiet -show_entries format=duration -show_entries "
|
||||
"stream=codec_name,width,height,r_frame_rate <file.mp4>\n"
|
||||
"\n"
|
||||
"# Extract a frame for QA\n"
|
||||
"ffmpeg -y -i <input.mp4> -vf \"select='eq(n,30)'\" -vsync vfr <out.png>\n"
|
||||
"```"
|
||||
)
|
||||
|
||||
out = tmpl
|
||||
out = out.replace("{{ROLE_NAME}}", role)
|
||||
out = out.replace("{{ROLE_RESPONSIBILITIES}}", team_member["responsibilities"])
|
||||
out = out.replace("{{INPUTS_READ}}", team_member.get("inputs", "_(see brief)_"))
|
||||
out = out.replace("{{OUTPUTS_PRODUCED}}", team_member.get("outputs", "_(see brief)_"))
|
||||
out = out.replace("{{TOOLSETS}}", ", ".join(team_member["toolsets"]))
|
||||
out = out.replace(
|
||||
"{{SKILLS}}",
|
||||
", ".join(team_member["skills"]) if team_member["skills"] else "(none)"
|
||||
)
|
||||
out = out.replace(
|
||||
"{{EXTERNAL_TOOLS}}",
|
||||
team_member.get("external_tools", "ffmpeg, ffprobe (via terminal)")
|
||||
)
|
||||
out = out.replace(
|
||||
"{{ROLE_RULES}}",
|
||||
team_member.get("role_rules", "_(see TEAM.md and brief.md)_")
|
||||
)
|
||||
out = out.replace("{{COMMON_RULES}}", common_rules)
|
||||
out = out.replace("{{COMMON_COMMANDS}}", common_commands)
|
||||
return out
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("plan_json", help="Path to plan.json")
|
||||
ap.add_argument("--out", default="setup.sh",
|
||||
help="Output path for setup.sh (default: ./setup.sh)")
|
||||
ap.add_argument("--brief-out", default=None,
|
||||
help="Write brief.md alongside (default: skipped)")
|
||||
ap.add_argument("--team-out", default=None,
|
||||
help="Write TEAM.md alongside (default: skipped)")
|
||||
args = ap.parse_args()
|
||||
|
||||
plan = json.loads(Path(args.plan_json).read_text())
|
||||
errors = validate_plan(plan)
|
||||
if errors:
|
||||
print("Plan validation failed:", file=sys.stderr)
|
||||
for e in errors:
|
||||
print(f" - {e}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
brief = render_brief(plan)
|
||||
team = render_team_md(plan)
|
||||
setup = render_setup_sh(plan, brief, team)
|
||||
|
||||
Path(args.out).write_text(setup)
|
||||
os.chmod(args.out, 0o755)
|
||||
print(f"Wrote {args.out}")
|
||||
|
||||
if args.brief_out:
|
||||
Path(args.brief_out).write_text(brief)
|
||||
print(f"Wrote {args.brief_out}")
|
||||
if args.team_out:
|
||||
Path(args.team_out).write_text(team)
|
||||
print(f"Wrote {args.team_out}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,195 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Monitor a running video-production kanban. Polls `hermes kanban list` and
|
||||
`events` for a tenant and surfaces issues (stuck tasks, missing heartbeats,
|
||||
repeated retries, dependency deadlocks).
|
||||
|
||||
Usage:
|
||||
monitor.py --tenant <project-slug> [--interval 30]
|
||||
|
||||
Outputs a periodic snapshot to stdout. Sends alerts via stderr when issues
|
||||
are detected. Designed to run alongside the kanban — kill with Ctrl-C when
|
||||
you're satisfied (or scripted to stop on completion).
|
||||
|
||||
This is best-effort observability. It does not auto-restart tasks; intervention
|
||||
decisions should remain human/AI-overseen.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
from collections import defaultdict
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
|
||||
def hermes_available() -> bool:
|
||||
return shutil.which("hermes") is not None
|
||||
|
||||
|
||||
def kanban_list(tenant: str) -> list[dict]:
|
||||
"""Returns parsed task rows. Falls back to plain stdout parsing if JSON
|
||||
output isn't supported by the installed hermes CLI."""
|
||||
try:
|
||||
out = subprocess.run(
|
||||
["hermes", "kanban", "list", "--tenant", tenant, "--json"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if out.returncode == 0 and out.stdout.strip().startswith("["):
|
||||
return json.loads(out.stdout)
|
||||
except (FileNotFoundError, json.JSONDecodeError):
|
||||
pass
|
||||
# Fallback: textual parse of `hermes kanban list`
|
||||
out = subprocess.run(
|
||||
["hermes", "kanban", "list", "--tenant", tenant],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
rows = []
|
||||
for line in out.stdout.splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#") or "STATUS" in line.upper():
|
||||
continue
|
||||
parts = line.split()
|
||||
if len(parts) >= 4 and parts[0].startswith("t_"):
|
||||
rows.append({
|
||||
"id": parts[0],
|
||||
"status": parts[1] if len(parts) > 1 else "?",
|
||||
"assignee": parts[2] if len(parts) > 2 else "?",
|
||||
"title": " ".join(parts[3:]) if len(parts) > 3 else "",
|
||||
"started_at": None,
|
||||
"heartbeat_at": None,
|
||||
"max_runtime_s": None,
|
||||
})
|
||||
return rows
|
||||
|
||||
|
||||
def kanban_show(task_id: str) -> dict | None:
|
||||
out = subprocess.run(
|
||||
["hermes", "kanban", "show", task_id, "--json"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if out.returncode != 0:
|
||||
return None
|
||||
try:
|
||||
return json.loads(out.stdout)
|
||||
except json.JSONDecodeError:
|
||||
return None
|
||||
|
||||
|
||||
def detect_issues(tasks: list[dict]) -> list[str]:
|
||||
"""Return a list of issue strings, one per concern."""
|
||||
now = datetime.now()
|
||||
issues: list[str] = []
|
||||
by_status = defaultdict(list)
|
||||
for t in tasks:
|
||||
by_status[t.get("status", "?")].append(t)
|
||||
|
||||
# Stuck tasks: RUNNING with no heartbeat in 2 min
|
||||
for t in by_status.get("running", []) + by_status.get("RUNNING", []):
|
||||
hb = t.get("heartbeat_at")
|
||||
if not hb:
|
||||
continue
|
||||
try:
|
||||
hb_dt = datetime.fromisoformat(str(hb).rstrip("Z"))
|
||||
except ValueError:
|
||||
continue
|
||||
if now - hb_dt > timedelta(minutes=2):
|
||||
issues.append(
|
||||
f"STUCK: {t['id']} ({t.get('assignee', '?')}) — "
|
||||
f"no heartbeat in {(now - hb_dt).total_seconds():.0f}s"
|
||||
)
|
||||
|
||||
# Tasks exceeding max_runtime
|
||||
for t in by_status.get("running", []) + by_status.get("RUNNING", []):
|
||||
started = t.get("started_at")
|
||||
max_rt = t.get("max_runtime_s")
|
||||
if not started or not max_rt:
|
||||
continue
|
||||
try:
|
||||
started_dt = datetime.fromisoformat(str(started).rstrip("Z"))
|
||||
except ValueError:
|
||||
continue
|
||||
elapsed = (now - started_dt).total_seconds()
|
||||
if elapsed > max_rt:
|
||||
issues.append(
|
||||
f"OVERTIME: {t['id']} ({t.get('assignee', '?')}) — "
|
||||
f"running {elapsed:.0f}s, cap was {max_rt}s"
|
||||
)
|
||||
|
||||
# Repeated retries
|
||||
for t in tasks:
|
||||
retries = t.get("retries", 0)
|
||||
if retries and retries >= 2:
|
||||
issues.append(
|
||||
f"FLAPPING: {t['id']} ({t.get('assignee', '?')}) — "
|
||||
f"retried {retries}× — fix root cause before next run"
|
||||
)
|
||||
|
||||
return issues
|
||||
|
||||
|
||||
def snapshot(tenant: str) -> tuple[list[dict], list[str]]:
|
||||
tasks = kanban_list(tenant)
|
||||
issues = detect_issues(tasks)
|
||||
return tasks, issues
|
||||
|
||||
|
||||
def print_snapshot(tasks: list[dict], issues: list[str]):
|
||||
counts = defaultdict(int)
|
||||
for t in tasks:
|
||||
counts[str(t.get("status", "?")).lower()] += 1
|
||||
|
||||
print(f"\n[{datetime.now().strftime('%H:%M:%S')}] "
|
||||
f"Total: {len(tasks)} | "
|
||||
+ " | ".join(f"{k}: {v}" for k, v in sorted(counts.items())))
|
||||
|
||||
for t in tasks:
|
||||
bar = "✓" if str(t.get("status", "")).lower() == "done" else \
|
||||
"▶" if str(t.get("status", "")).lower() == "running" else \
|
||||
"·" if str(t.get("status", "")).lower() == "ready" else \
|
||||
"✗" if str(t.get("status", "")).lower() == "failed" else "?"
|
||||
print(f" {bar} {t.get('id', '?'):14} {t.get('assignee', '?'):20} "
|
||||
f"{t.get('title', '')[:60]}")
|
||||
|
||||
if issues:
|
||||
print("\n ⚠ ISSUES:", file=sys.stderr)
|
||||
for i in issues:
|
||||
print(f" {i}", file=sys.stderr)
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--tenant", required=True,
|
||||
help="Project tenant slug to monitor")
|
||||
ap.add_argument("--interval", type=int, default=30,
|
||||
help="Poll interval in seconds (default: 30)")
|
||||
ap.add_argument("--once", action="store_true",
|
||||
help="Print one snapshot and exit (no polling loop)")
|
||||
args = ap.parse_args()
|
||||
|
||||
if not hermes_available():
|
||||
print("ERROR: 'hermes' CLI not found in PATH", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if args.once:
|
||||
tasks, issues = snapshot(args.tenant)
|
||||
print_snapshot(tasks, issues)
|
||||
sys.exit(0 if not issues else 2)
|
||||
|
||||
print(f"Monitoring tenant '{args.tenant}' every {args.interval}s. "
|
||||
"Ctrl-C to exit.")
|
||||
try:
|
||||
while True:
|
||||
tasks, issues = snapshot(args.tenant)
|
||||
print_snapshot(tasks, issues)
|
||||
time.sleep(args.interval)
|
||||
except KeyboardInterrupt:
|
||||
print("\nStopped.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,46 @@
|
||||
# Meme Generation Examples
|
||||
|
||||
## Example 1: Debugging at 2 AM
|
||||
|
||||
**Topic:** debugging production at 2 AM
|
||||
**Template:** this-is-fine
|
||||
|
||||
```bash
|
||||
python generate_meme.py this-is-fine /tmp/meme.png "PRODUCTION IS DOWN" "This is fine"
|
||||
```
|
||||
|
||||
## Example 2: Developer Priorities
|
||||
|
||||
**Topic:** choosing between writing tests and shipping features
|
||||
**Template:** drake
|
||||
|
||||
```bash
|
||||
python generate_meme.py drake /tmp/meme.png "Writing unit tests" "Shipping straight to prod"
|
||||
```
|
||||
|
||||
## Example 3: Exam Stress
|
||||
|
||||
**Topic:** final exam preparation
|
||||
**Template:** two-buttons
|
||||
|
||||
```bash
|
||||
python generate_meme.py two-buttons /tmp/meme.png "Study everything" "Sleep" "Me at midnight"
|
||||
```
|
||||
|
||||
## Example 4: Escalating Solutions
|
||||
|
||||
**Topic:** fixing a CSS bug
|
||||
**Template:** expanding-brain
|
||||
|
||||
```bash
|
||||
python generate_meme.py expanding-brain /tmp/meme.png "Reading the docs" "Stack Overflow" "!important on everything" "Deleting the stylesheet"
|
||||
```
|
||||
|
||||
## Example 5: Hot Take
|
||||
|
||||
**Topic:** tabs vs spaces
|
||||
**Template:** change-my-mind
|
||||
|
||||
```bash
|
||||
python generate_meme.py change-my-mind /tmp/meme.png "Tabs are just thicc spaces"
|
||||
```
|
||||
@@ -0,0 +1,130 @@
|
||||
---
|
||||
name: meme-generation
|
||||
description: Generate real meme images by picking a template and overlaying text with Pillow. Produces actual .png meme files.
|
||||
version: 2.0.0
|
||||
author: adanaleycio
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [creative, memes, humor, images]
|
||||
related_skills: [ascii-art, generative-widgets]
|
||||
category: creative
|
||||
---
|
||||
|
||||
# Meme Generation
|
||||
|
||||
Generate actual meme images from a topic. Picks a template, writes captions, and renders a real .png file with text overlay.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks you to make or generate a meme
|
||||
- User wants a meme about a specific topic, situation, or frustration
|
||||
- User says "meme this" or similar
|
||||
|
||||
## Available Templates
|
||||
|
||||
The script supports **any of the ~100 popular imgflip templates** by name or ID, plus 10 curated templates with hand-tuned text positioning.
|
||||
|
||||
### Curated Templates (custom text placement)
|
||||
|
||||
| ID | Name | Fields | Best for |
|
||||
|----|------|--------|----------|
|
||||
| `this-is-fine` | This is Fine | top, bottom | chaos, denial |
|
||||
| `drake` | Drake Hotline Bling | reject, approve | rejecting/preferring |
|
||||
| `distracted-boyfriend` | Distracted Boyfriend | distraction, current, person | temptation, shifting priorities |
|
||||
| `two-buttons` | Two Buttons | left, right, person | impossible choice |
|
||||
| `expanding-brain` | Expanding Brain | 4 levels | escalating irony |
|
||||
| `change-my-mind` | Change My Mind | statement | hot takes |
|
||||
| `woman-yelling-at-cat` | Woman Yelling at Cat | woman, cat | arguments |
|
||||
| `one-does-not-simply` | One Does Not Simply | top, bottom | deceptively hard things |
|
||||
| `grus-plan` | Gru's Plan | step1-3, realization | plans that backfire |
|
||||
| `batman-slapping-robin` | Batman Slapping Robin | robin, batman | shutting down bad ideas |
|
||||
|
||||
### Dynamic Templates (from imgflip API)
|
||||
|
||||
Any template not in the curated list can be used by name or imgflip ID. These get smart default text positioning (top/bottom for 2-field, evenly spaced for 3+). Search with:
|
||||
```bash
|
||||
python "$SKILL_DIR/scripts/generate_meme.py" --search "disaster"
|
||||
```
|
||||
|
||||
## Procedure
|
||||
|
||||
### Mode 1: Classic Template (default)
|
||||
|
||||
1. Read the user's topic and identify the core dynamic (chaos, dilemma, preference, irony, etc.)
|
||||
2. Pick the template that best matches. Use the "Best for" column, or search with `--search`.
|
||||
3. Write short captions for each field (8-12 words max per field, shorter is better).
|
||||
4. Find the skill's script directory:
|
||||
```
|
||||
SKILL_DIR=$(dirname "$(find ~/.hermes/skills -path '*/meme-generation/SKILL.md' 2>/dev/null | head -1)")
|
||||
```
|
||||
5. Run the generator:
|
||||
```bash
|
||||
python "$SKILL_DIR/scripts/generate_meme.py" <template_id> /tmp/meme.png "caption 1" "caption 2" ...
|
||||
```
|
||||
6. Return the image with `MEDIA:/tmp/meme.png`
|
||||
|
||||
### Mode 2: Custom AI Image (when image_generate is available)
|
||||
|
||||
Use this when no classic template fits, or when the user wants something original.
|
||||
|
||||
1. Write the captions first.
|
||||
2. Use `image_generate` to create a scene that matches the meme concept. Do NOT include any text in the image prompt — text will be added by the script. Describe only the visual scene.
|
||||
3. Find the generated image path from the image_generate result URL. Download it to a local path if needed.
|
||||
4. Run the script with `--image` to overlay text, choosing a mode:
|
||||
- **Overlay** (text directly on image, white with black outline):
|
||||
```bash
|
||||
python "$SKILL_DIR/scripts/generate_meme.py" --image /path/to/scene.png /tmp/meme.png "top text" "bottom text"
|
||||
```
|
||||
- **Bars** (black bars above/below with white text — cleaner, always readable):
|
||||
```bash
|
||||
python "$SKILL_DIR/scripts/generate_meme.py" --image /path/to/scene.png --bars /tmp/meme.png "top text" "bottom text"
|
||||
```
|
||||
Use `--bars` when the image is busy/detailed and text would be hard to read on top of it.
|
||||
5. **Verify with vision** (if `vision_analyze` is available): Check the result looks good:
|
||||
```
|
||||
vision_analyze(image_url="/tmp/meme.png", question="Is the text legible and well-positioned? Does the meme work visually?")
|
||||
```
|
||||
If the vision model flags issues (text hard to read, bad placement, etc.), try the other mode (switch between overlay and bars) or regenerate the scene.
|
||||
6. Return the image with `MEDIA:/tmp/meme.png`
|
||||
|
||||
## Examples
|
||||
|
||||
**"debugging production at 2 AM":**
|
||||
```bash
|
||||
python generate_meme.py this-is-fine /tmp/meme.png "SERVERS ARE ON FIRE" "This is fine"
|
||||
```
|
||||
|
||||
**"choosing between sleep and one more episode":**
|
||||
```bash
|
||||
python generate_meme.py drake /tmp/meme.png "Getting 8 hours of sleep" "One more episode at 3 AM"
|
||||
```
|
||||
|
||||
**"the stages of a Monday morning":**
|
||||
```bash
|
||||
python generate_meme.py expanding-brain /tmp/meme.png "Setting an alarm" "Setting 5 alarms" "Sleeping through all alarms" "Working from bed"
|
||||
```
|
||||
|
||||
## Listing Templates
|
||||
|
||||
To see all available templates:
|
||||
```bash
|
||||
python generate_meme.py --list
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Keep captions SHORT. Memes with long text look terrible.
|
||||
- Match the number of text arguments to the template's field count.
|
||||
- Pick the template that fits the joke structure, not just the topic.
|
||||
- Do not generate hateful, abusive, or personally targeted content.
|
||||
- The script caches template images in `scripts/.cache/` after first download.
|
||||
|
||||
## Verification
|
||||
|
||||
The output is correct if:
|
||||
- A .png file was created at the output path
|
||||
- Text is legible (white with black outline) on the template
|
||||
- The joke lands — caption matches the template's intended structure
|
||||
- File can be delivered via MEDIA: path
|
||||
@@ -0,0 +1 @@
|
||||
.cache/
|
||||
@@ -0,0 +1,470 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate a meme image by overlaying text on a template.
|
||||
|
||||
Usage:
|
||||
python generate_meme.py <template_id_or_name> <output_path> <text1> [text2] [text3] [text4]
|
||||
|
||||
Example:
|
||||
python generate_meme.py drake /tmp/meme.png "Writing tests" "Shipping to prod and hoping"
|
||||
python generate_meme.py "Disaster Girl" /tmp/meme.png "Top text" "Bottom text"
|
||||
python generate_meme.py --list # show curated templates
|
||||
python generate_meme.py --search "distracted" # search all imgflip templates
|
||||
|
||||
Templates with custom text positioning are in templates.json (10 curated).
|
||||
Any of the ~100 popular imgflip templates can also be used by name or ID —
|
||||
unknown templates get smart default text positioning based on their box_count.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from io import BytesIO
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import requests as _requests
|
||||
except ImportError:
|
||||
_requests = None
|
||||
|
||||
from PIL import Image, ImageDraw, ImageFont
|
||||
|
||||
SCRIPT_DIR = Path(__file__).parent
|
||||
TEMPLATES_FILE = SCRIPT_DIR / "templates.json"
|
||||
CACHE_DIR = SCRIPT_DIR / ".cache"
|
||||
IMGFLIP_API = "https://api.imgflip.com/get_memes"
|
||||
IMGFLIP_CACHE_FILE = CACHE_DIR / "imgflip_memes.json"
|
||||
IMGFLIP_CACHE_MAX_AGE = 86400 # 24 hours
|
||||
|
||||
|
||||
def _fetch_url(url: str, timeout: int = 15) -> bytes:
|
||||
"""Fetch URL content, using requests if available, else urllib."""
|
||||
if _requests is not None:
|
||||
resp = _requests.get(url, timeout=timeout)
|
||||
resp.raise_for_status()
|
||||
return resp.content
|
||||
import urllib.request
|
||||
return urllib.request.urlopen(url, timeout=timeout).read()
|
||||
|
||||
|
||||
def load_curated_templates() -> dict:
|
||||
"""Load templates with hand-tuned text field positions."""
|
||||
with open(TEMPLATES_FILE) as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def _default_fields(box_count: int) -> list:
|
||||
"""Generate sensible default text field positions for unknown templates."""
|
||||
if box_count <= 0:
|
||||
box_count = 2
|
||||
if box_count == 1:
|
||||
return [{"name": "text", "x_pct": 0.5, "y_pct": 0.5, "w_pct": 0.90, "align": "center"}]
|
||||
if box_count == 2:
|
||||
return [
|
||||
{"name": "top", "x_pct": 0.5, "y_pct": 0.08, "w_pct": 0.95, "align": "center"},
|
||||
{"name": "bottom", "x_pct": 0.5, "y_pct": 0.92, "w_pct": 0.95, "align": "center"},
|
||||
]
|
||||
# 3+: evenly space vertically
|
||||
fields = []
|
||||
for i in range(box_count):
|
||||
y = 0.08 + (0.84 * i / (box_count - 1)) if box_count > 1 else 0.5
|
||||
fields.append({
|
||||
"name": f"text{i+1}",
|
||||
"x_pct": 0.5,
|
||||
"y_pct": round(y, 2),
|
||||
"w_pct": 0.90,
|
||||
"align": "center",
|
||||
})
|
||||
return fields
|
||||
|
||||
|
||||
def fetch_imgflip_templates() -> list:
|
||||
"""Fetch popular meme templates from imgflip API. Cached for 24h."""
|
||||
import time
|
||||
|
||||
CACHE_DIR.mkdir(exist_ok=True)
|
||||
# Check cache
|
||||
if IMGFLIP_CACHE_FILE.exists():
|
||||
age = time.time() - IMGFLIP_CACHE_FILE.stat().st_mtime
|
||||
if age < IMGFLIP_CACHE_MAX_AGE:
|
||||
with open(IMGFLIP_CACHE_FILE) as f:
|
||||
return json.load(f)
|
||||
|
||||
try:
|
||||
data = json.loads(_fetch_url(IMGFLIP_API))
|
||||
memes = data.get("data", {}).get("memes", [])
|
||||
with open(IMGFLIP_CACHE_FILE, "w") as f:
|
||||
json.dump(memes, f)
|
||||
return memes
|
||||
except Exception as e:
|
||||
# If fetch fails and we have stale cache, use it
|
||||
if IMGFLIP_CACHE_FILE.exists():
|
||||
with open(IMGFLIP_CACHE_FILE) as f:
|
||||
return json.load(f)
|
||||
print(f"Warning: could not fetch imgflip templates: {e}", file=sys.stderr)
|
||||
return []
|
||||
|
||||
|
||||
def _slugify(name: str) -> str:
|
||||
"""Convert a template name to a slug for matching."""
|
||||
return name.lower().replace(" ", "-").replace("'", "").replace("\"", "")
|
||||
|
||||
|
||||
def resolve_template(identifier: str) -> dict:
|
||||
"""Resolve a template by curated ID, imgflip name, or imgflip ID.
|
||||
|
||||
Returns dict with: name, url, fields, source.
|
||||
"""
|
||||
curated = load_curated_templates()
|
||||
|
||||
# 1. Exact curated ID match
|
||||
if identifier in curated:
|
||||
tmpl = curated[identifier]
|
||||
return {**tmpl, "source": "curated"}
|
||||
|
||||
# 2. Slugified curated match
|
||||
slug = _slugify(identifier)
|
||||
for tid, tmpl in curated.items():
|
||||
if _slugify(tmpl["name"]) == slug or tid == slug:
|
||||
return {**tmpl, "source": "curated"}
|
||||
|
||||
# 3. Search imgflip templates
|
||||
imgflip_memes = fetch_imgflip_templates()
|
||||
slug_lower = slug.lower()
|
||||
id_lower = identifier.strip()
|
||||
|
||||
for meme in imgflip_memes:
|
||||
meme_slug = _slugify(meme["name"])
|
||||
# Check curated first for this imgflip template (custom positioning)
|
||||
for tid, ctmpl in curated.items():
|
||||
if _slugify(ctmpl["name"]) == meme_slug:
|
||||
if meme_slug == slug_lower or meme["id"] == id_lower:
|
||||
return {**ctmpl, "source": "curated"}
|
||||
|
||||
if meme_slug == slug_lower or meme["id"] == id_lower or slug_lower in meme_slug:
|
||||
return {
|
||||
"name": meme["name"],
|
||||
"url": meme["url"],
|
||||
"fields": _default_fields(meme.get("box_count", 2)),
|
||||
"source": "imgflip",
|
||||
}
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def get_template_image(url: str) -> Image.Image:
|
||||
"""Download a template image, caching it locally."""
|
||||
CACHE_DIR.mkdir(exist_ok=True)
|
||||
# Use URL hash as cache key
|
||||
cache_name = url.split("/")[-1]
|
||||
cache_path = CACHE_DIR / cache_name
|
||||
|
||||
# Always cache as PNG to avoid JPEG/RGBA conflicts
|
||||
cache_path = cache_path.with_suffix(".png")
|
||||
|
||||
if cache_path.exists():
|
||||
return Image.open(cache_path).convert("RGBA")
|
||||
|
||||
data = _fetch_url(url)
|
||||
img = Image.open(BytesIO(data)).convert("RGBA")
|
||||
img.save(cache_path, "PNG")
|
||||
return img
|
||||
|
||||
|
||||
def find_font(size: int) -> ImageFont.FreeTypeFont:
|
||||
"""Find a bold font for meme text. Tries Impact, then falls back."""
|
||||
candidates = [
|
||||
"/usr/share/fonts/truetype/msttcorefonts/Impact.ttf",
|
||||
"/usr/share/fonts/truetype/liberation/LiberationSans-Bold.ttf",
|
||||
"/usr/share/fonts/liberation-sans/LiberationSans-Bold.ttf",
|
||||
"/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf",
|
||||
"/usr/share/fonts/dejavu-sans/DejaVuSans-Bold.ttf",
|
||||
"/System/Library/Fonts/Helvetica.ttc",
|
||||
"/System/Library/Fonts/SFCompact.ttf",
|
||||
]
|
||||
for path in candidates:
|
||||
if os.path.exists(path):
|
||||
try:
|
||||
return ImageFont.truetype(path, size)
|
||||
except (OSError, IOError):
|
||||
continue
|
||||
# Last resort: Pillow default
|
||||
try:
|
||||
return ImageFont.truetype("DejaVuSans-Bold", size)
|
||||
except (OSError, IOError):
|
||||
return ImageFont.load_default()
|
||||
|
||||
|
||||
def _wrap_text(text: str, font: ImageFont.FreeTypeFont, max_width: int) -> str:
|
||||
"""Word-wrap text to fit within max_width pixels. Never breaks mid-word."""
|
||||
words = text.split()
|
||||
if not words:
|
||||
return text
|
||||
lines = []
|
||||
current_line = words[0]
|
||||
for word in words[1:]:
|
||||
test_line = current_line + " " + word
|
||||
if font.getlength(test_line) <= max_width:
|
||||
current_line = test_line
|
||||
else:
|
||||
lines.append(current_line)
|
||||
current_line = word
|
||||
lines.append(current_line)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def draw_outlined_text(
|
||||
draw: ImageDraw.ImageDraw,
|
||||
text: str,
|
||||
x: int,
|
||||
y: int,
|
||||
font_size: int,
|
||||
max_width: int,
|
||||
align: str = "center",
|
||||
):
|
||||
"""Draw white text with black outline, auto-scaled to fit max_width."""
|
||||
# Auto-scale: reduce font size until text fits reasonably
|
||||
size = font_size
|
||||
while size > 12:
|
||||
font = find_font(size)
|
||||
wrapped = _wrap_text(text, font, max_width)
|
||||
bbox = draw.multiline_textbbox((0, 0), wrapped, font=font, align=align)
|
||||
text_w = bbox[2] - bbox[0]
|
||||
line_count = wrapped.count("\n") + 1
|
||||
# Accept if width fits and not too many lines
|
||||
if text_w <= max_width * 1.05 and line_count <= 4:
|
||||
break
|
||||
size -= 2
|
||||
else:
|
||||
font = find_font(size)
|
||||
wrapped = _wrap_text(text, font, max_width)
|
||||
|
||||
# Measure total text block
|
||||
bbox = draw.multiline_textbbox((0, 0), wrapped, font=font, align=align)
|
||||
text_w = bbox[2] - bbox[0]
|
||||
text_h = bbox[3] - bbox[1]
|
||||
|
||||
# Center horizontally at x, vertically at y
|
||||
tx = x - text_w // 2
|
||||
ty = y - text_h // 2
|
||||
|
||||
# Draw outline (black border)
|
||||
outline_range = max(2, font.size // 18)
|
||||
for dx in range(-outline_range, outline_range + 1):
|
||||
for dy in range(-outline_range, outline_range + 1):
|
||||
if dx == 0 and dy == 0:
|
||||
continue
|
||||
draw.multiline_text(
|
||||
(tx + dx, ty + dy), wrapped, font=font, fill="black", align=align
|
||||
)
|
||||
# Draw main text (white)
|
||||
draw.multiline_text((tx, ty), wrapped, font=font, fill="white", align=align)
|
||||
|
||||
|
||||
def _overlay_on_image(img: Image.Image, texts: list, fields: list) -> Image.Image:
|
||||
"""Overlay meme text directly on an image using field positions."""
|
||||
draw = ImageDraw.Draw(img)
|
||||
w, h = img.size
|
||||
base_font_size = max(16, min(w, h) // 12)
|
||||
|
||||
for i, field in enumerate(fields):
|
||||
if i >= len(texts):
|
||||
break
|
||||
text = texts[i].strip()
|
||||
if not text:
|
||||
continue
|
||||
fx = int(field["x_pct"] * w)
|
||||
fy = int(field["y_pct"] * h)
|
||||
fw = int(field["w_pct"] * w)
|
||||
draw_outlined_text(draw, text, fx, fy, base_font_size, fw, field.get("align", "center"))
|
||||
return img
|
||||
|
||||
|
||||
def _add_bars(img: Image.Image, texts: list) -> Image.Image:
|
||||
"""Add black bars with white text above/below the image.
|
||||
|
||||
Distributes texts across bars: first text on top bar, last text on
|
||||
bottom bar, any middle texts overlaid on the image center.
|
||||
"""
|
||||
w, h = img.size
|
||||
bar_font_size = max(20, w // 16)
|
||||
font = find_font(bar_font_size)
|
||||
padding = bar_font_size // 2
|
||||
|
||||
top_text = texts[0].strip() if texts else ""
|
||||
bottom_text = texts[-1].strip() if len(texts) > 1 else ""
|
||||
middle_texts = [t.strip() for t in texts[1:-1]] if len(texts) > 2 else []
|
||||
|
||||
def _measure_bar(text: str) -> int:
|
||||
if not text:
|
||||
return 0
|
||||
wrapped = _wrap_text(text, font, int(w * 0.92))
|
||||
bbox = ImageDraw.Draw(Image.new("RGB", (1, 1))).multiline_textbbox(
|
||||
(0, 0), wrapped, font=font, align="center"
|
||||
)
|
||||
return (bbox[3] - bbox[1]) + padding * 2
|
||||
|
||||
top_h = _measure_bar(top_text)
|
||||
bottom_h = _measure_bar(bottom_text)
|
||||
new_h = h + top_h + bottom_h
|
||||
|
||||
canvas = Image.new("RGB", (w, new_h), (0, 0, 0))
|
||||
canvas.paste(img.convert("RGB"), (0, top_h))
|
||||
draw = ImageDraw.Draw(canvas)
|
||||
|
||||
if top_text:
|
||||
wrapped = _wrap_text(top_text, font, int(w * 0.92))
|
||||
bbox = draw.multiline_textbbox((0, 0), wrapped, font=font, align="center")
|
||||
tw = bbox[2] - bbox[0]
|
||||
th = bbox[3] - bbox[1]
|
||||
tx = (w - tw) // 2
|
||||
ty = (top_h - th) // 2
|
||||
draw.multiline_text((tx, ty), wrapped, font=font, fill="white", align="center")
|
||||
|
||||
if bottom_text:
|
||||
wrapped = _wrap_text(bottom_text, font, int(w * 0.92))
|
||||
bbox = draw.multiline_textbbox((0, 0), wrapped, font=font, align="center")
|
||||
tw = bbox[2] - bbox[0]
|
||||
th = bbox[3] - bbox[1]
|
||||
tx = (w - tw) // 2
|
||||
ty = top_h + h + (bottom_h - th) // 2
|
||||
draw.multiline_text((tx, ty), wrapped, font=font, fill="white", align="center")
|
||||
|
||||
# Overlay any middle texts centered on the image
|
||||
if middle_texts:
|
||||
mid_fields = _default_fields(len(middle_texts))
|
||||
# Shift y positions to account for top bar offset
|
||||
for field in mid_fields:
|
||||
field["y_pct"] = (top_h + field["y_pct"] * h) / new_h
|
||||
field["w_pct"] = 0.90
|
||||
_overlay_on_image(canvas, middle_texts, mid_fields)
|
||||
|
||||
return canvas
|
||||
|
||||
|
||||
def generate_meme(template_id: str, texts: list[str], output_path: str) -> str:
|
||||
"""Generate a meme from a template and save it. Returns the path."""
|
||||
tmpl = resolve_template(template_id)
|
||||
|
||||
if tmpl is None:
|
||||
print(f"Unknown template: {template_id}", file=sys.stderr)
|
||||
print("Use --list to see curated templates or --search to find imgflip templates.", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
fields = tmpl["fields"]
|
||||
print(f"Using template: {tmpl['name']} ({tmpl['source']}, {len(fields)} fields)", file=sys.stderr)
|
||||
|
||||
img = get_template_image(tmpl["url"])
|
||||
img = _overlay_on_image(img, texts, fields)
|
||||
|
||||
output = Path(output_path)
|
||||
if output.suffix.lower() in {".jpg", ".jpeg"}:
|
||||
img = img.convert("RGB")
|
||||
img.save(str(output), quality=95)
|
||||
return str(output)
|
||||
|
||||
|
||||
def generate_from_image(
|
||||
image_path: str, texts: list[str], output_path: str, use_bars: bool = False
|
||||
) -> str:
|
||||
"""Generate a meme from a custom image (e.g. AI-generated). Returns the path."""
|
||||
img = Image.open(image_path).convert("RGBA")
|
||||
print(f"Custom image: {img.size[0]}x{img.size[1]}, {len(texts)} text(s), mode={'bars' if use_bars else 'overlay'}", file=sys.stderr)
|
||||
|
||||
if use_bars:
|
||||
result = _add_bars(img, texts)
|
||||
else:
|
||||
fields = _default_fields(len(texts))
|
||||
result = _overlay_on_image(img, texts, fields)
|
||||
|
||||
output = Path(output_path)
|
||||
if output.suffix.lower() in {".jpg", ".jpeg"}:
|
||||
result = result.convert("RGB")
|
||||
result.save(str(output), quality=95)
|
||||
return str(output)
|
||||
|
||||
|
||||
def list_templates():
|
||||
"""Print curated templates with custom positioning."""
|
||||
templates = load_curated_templates()
|
||||
print(f"{'ID':<25} {'Name':<30} {'Fields':<8} Best for")
|
||||
print("-" * 90)
|
||||
for tid, tmpl in sorted(templates.items()):
|
||||
fields = len(tmpl["fields"])
|
||||
print(f"{tid:<25} {tmpl['name']:<30} {fields:<8} {tmpl['best_for']}")
|
||||
print(f"\n{len(templates)} curated templates with custom text positioning.")
|
||||
print("Use --search to find any of the ~100 popular imgflip templates.")
|
||||
|
||||
|
||||
def search_templates(query: str):
|
||||
"""Search imgflip templates by name."""
|
||||
imgflip_memes = fetch_imgflip_templates()
|
||||
curated = load_curated_templates()
|
||||
curated_slugs = {_slugify(t["name"]) for t in curated.values()}
|
||||
query_lower = query.lower()
|
||||
|
||||
matches = []
|
||||
for meme in imgflip_memes:
|
||||
if query_lower in meme["name"].lower():
|
||||
slug = _slugify(meme["name"])
|
||||
has_custom = "curated" if slug in curated_slugs else "default"
|
||||
matches.append((meme["name"], meme["id"], meme.get("box_count", 2), has_custom))
|
||||
|
||||
if not matches:
|
||||
print(f"No templates found matching '{query}'")
|
||||
return
|
||||
|
||||
print(f"{'Name':<40} {'ID':<12} {'Fields':<8} Positioning")
|
||||
print("-" * 75)
|
||||
for name, mid, boxes, positioning in matches:
|
||||
print(f"{name:<40} {mid:<12} {boxes:<8} {positioning}")
|
||||
print(f"\n{len(matches)} template(s) found. Use the name or ID as the first argument.")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: generate_meme.py <template_id_or_name> <output_path> <text1> [text2] ...")
|
||||
print(" generate_meme.py --image <path> [--bars] <output_path> <text1> [text2] ...")
|
||||
print(" generate_meme.py --list # curated templates")
|
||||
print(" generate_meme.py --search <query> # search all imgflip templates")
|
||||
sys.exit(1)
|
||||
|
||||
if sys.argv[1] == "--list":
|
||||
list_templates()
|
||||
sys.exit(0)
|
||||
|
||||
if sys.argv[1] == "--search":
|
||||
if len(sys.argv) < 3:
|
||||
print("Usage: generate_meme.py --search <query>")
|
||||
sys.exit(1)
|
||||
search_templates(sys.argv[2])
|
||||
sys.exit(0)
|
||||
|
||||
if sys.argv[1] == "--image":
|
||||
# Custom image mode: --image <path> [--bars] <output> <text1> ...
|
||||
args = sys.argv[2:]
|
||||
if len(args) < 3:
|
||||
print("Usage: generate_meme.py --image <image_path> [--bars] <output_path> <text1> ...")
|
||||
sys.exit(1)
|
||||
image_path = args.pop(0)
|
||||
use_bars = False
|
||||
if args and args[0] == "--bars":
|
||||
use_bars = True
|
||||
args.pop(0)
|
||||
if len(args) < 2:
|
||||
print("Need at least: output_path and one text argument")
|
||||
sys.exit(1)
|
||||
output_path = args.pop(0)
|
||||
result = generate_from_image(image_path, args, output_path, use_bars=use_bars)
|
||||
print(f"Meme saved to: {result}")
|
||||
sys.exit(0)
|
||||
|
||||
if len(sys.argv) < 4:
|
||||
print("Need at least: template_id_or_name, output_path, and one text argument")
|
||||
sys.exit(1)
|
||||
|
||||
template_id = sys.argv[1]
|
||||
output_path = sys.argv[2]
|
||||
texts = sys.argv[3:]
|
||||
|
||||
result = generate_meme(template_id, texts, output_path)
|
||||
print(f"Meme saved to: {result}")
|
||||
@@ -0,0 +1,97 @@
|
||||
{
|
||||
"this-is-fine": {
|
||||
"name": "This is Fine",
|
||||
"url": "https://i.imgflip.com/wxica.jpg",
|
||||
"best_for": "chaos, denial, pretending things are okay",
|
||||
"fields": [
|
||||
{"name": "top", "x_pct": 0.5, "y_pct": 0.08, "w_pct": 0.95, "align": "center"},
|
||||
{"name": "bottom", "x_pct": 0.5, "y_pct": 0.92, "w_pct": 0.95, "align": "center"}
|
||||
]
|
||||
},
|
||||
"drake": {
|
||||
"name": "Drake Hotline Bling",
|
||||
"url": "https://i.imgflip.com/30b1gx.jpg",
|
||||
"best_for": "rejecting one thing, preferring another",
|
||||
"fields": [
|
||||
{"name": "reject", "x_pct": 0.73, "y_pct": 0.25, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "approve", "x_pct": 0.73, "y_pct": 0.75, "w_pct": 0.45, "align": "center"}
|
||||
]
|
||||
},
|
||||
"distracted-boyfriend": {
|
||||
"name": "Distracted Boyfriend",
|
||||
"url": "https://i.imgflip.com/1ur9b0.jpg",
|
||||
"best_for": "distraction, shifting priorities, temptation",
|
||||
"fields": [
|
||||
{"name": "distraction", "x_pct": 0.18, "y_pct": 0.90, "w_pct": 0.30, "align": "center"},
|
||||
{"name": "current", "x_pct": 0.55, "y_pct": 0.90, "w_pct": 0.30, "align": "center"},
|
||||
{"name": "person", "x_pct": 0.82, "y_pct": 0.90, "w_pct": 0.30, "align": "center"}
|
||||
]
|
||||
},
|
||||
"two-buttons": {
|
||||
"name": "Two Buttons",
|
||||
"url": "https://i.imgflip.com/1g8my4.jpg",
|
||||
"best_for": "impossible choice, dilemma between two options",
|
||||
"fields": [
|
||||
{"name": "left_button", "x_pct": 0.30, "y_pct": 0.20, "w_pct": 0.28, "align": "center"},
|
||||
{"name": "right_button", "x_pct": 0.62, "y_pct": 0.12, "w_pct": 0.28, "align": "center"},
|
||||
{"name": "person", "x_pct": 0.5, "y_pct": 0.85, "w_pct": 0.90, "align": "center"}
|
||||
]
|
||||
},
|
||||
"expanding-brain": {
|
||||
"name": "Expanding Brain",
|
||||
"url": "https://i.imgflip.com/1jwhww.jpg",
|
||||
"best_for": "escalating irony, increasingly absurd ideas",
|
||||
"fields": [
|
||||
{"name": "level1", "x_pct": 0.25, "y_pct": 0.12, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "level2", "x_pct": 0.25, "y_pct": 0.38, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "level3", "x_pct": 0.25, "y_pct": 0.63, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "level4", "x_pct": 0.25, "y_pct": 0.88, "w_pct": 0.45, "align": "center"}
|
||||
]
|
||||
},
|
||||
"change-my-mind": {
|
||||
"name": "Change My Mind",
|
||||
"url": "https://i.imgflip.com/24y43o.jpg",
|
||||
"best_for": "strong or ironic opinion, controversial take",
|
||||
"fields": [
|
||||
{"name": "statement", "x_pct": 0.58, "y_pct": 0.78, "w_pct": 0.35, "align": "center"}
|
||||
]
|
||||
},
|
||||
"woman-yelling-at-cat": {
|
||||
"name": "Woman Yelling at Cat",
|
||||
"url": "https://i.imgflip.com/345v97.jpg",
|
||||
"best_for": "argument, blame, misunderstanding",
|
||||
"fields": [
|
||||
{"name": "woman", "x_pct": 0.27, "y_pct": 0.10, "w_pct": 0.50, "align": "center"},
|
||||
{"name": "cat", "x_pct": 0.76, "y_pct": 0.10, "w_pct": 0.44, "align": "center"}
|
||||
]
|
||||
},
|
||||
"one-does-not-simply": {
|
||||
"name": "One Does Not Simply",
|
||||
"url": "https://i.imgflip.com/1bij.jpg",
|
||||
"best_for": "something that sounds easy but is actually hard",
|
||||
"fields": [
|
||||
{"name": "top", "x_pct": 0.5, "y_pct": 0.08, "w_pct": 0.95, "align": "center"},
|
||||
{"name": "bottom", "x_pct": 0.5, "y_pct": 0.92, "w_pct": 0.95, "align": "center"}
|
||||
]
|
||||
},
|
||||
"grus-plan": {
|
||||
"name": "Gru's Plan",
|
||||
"url": "https://i.imgflip.com/26jxvs.jpg",
|
||||
"best_for": "a plan that backfires, unexpected consequence",
|
||||
"fields": [
|
||||
{"name": "step1", "x_pct": 0.5, "y_pct": 0.05, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "step2", "x_pct": 0.5, "y_pct": 0.30, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "step3", "x_pct": 0.5, "y_pct": 0.55, "w_pct": 0.45, "align": "center"},
|
||||
{"name": "realization", "x_pct": 0.5, "y_pct": 0.80, "w_pct": 0.45, "align": "center"}
|
||||
]
|
||||
},
|
||||
"batman-slapping-robin": {
|
||||
"name": "Batman Slapping Robin",
|
||||
"url": "https://i.imgflip.com/9ehk.jpg",
|
||||
"best_for": "shutting down a bad idea, correcting someone",
|
||||
"fields": [
|
||||
{"name": "robin", "x_pct": 0.28, "y_pct": 0.08, "w_pct": 0.50, "align": "center"},
|
||||
{"name": "batman", "x_pct": 0.72, "y_pct": 0.08, "w_pct": 0.50, "align": "center"}
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
---
|
||||
name: inference-sh-cli
|
||||
description: "Run 150+ AI apps via inference.sh CLI (infsh) — image generation, video creation, LLMs, search, 3D, social automation. Uses the terminal tool. Triggers: inference.sh, infsh, ai apps, flux, veo, image generation, video generation, seedream, seedance, tavily"
|
||||
version: 1.0.0
|
||||
author: okaris
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [AI, image-generation, video, LLM, search, inference, FLUX, Veo, Claude]
|
||||
related_skills: []
|
||||
---
|
||||
|
||||
# inference.sh CLI
|
||||
|
||||
Run 150+ AI apps in the cloud with a simple CLI. No GPU required.
|
||||
|
||||
All commands use the **terminal tool** to run `infsh` commands.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks to generate images (FLUX, Reve, Seedream, Grok, Gemini image)
|
||||
- User asks to generate video (Veo, Wan, Seedance, OmniHuman)
|
||||
- User asks about inference.sh or infsh
|
||||
- User wants to run AI apps without managing individual provider APIs
|
||||
- User asks for AI-powered search (Tavily, Exa)
|
||||
- User needs avatar/lipsync generation
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The `infsh` CLI must be installed and authenticated. Check with:
|
||||
|
||||
```bash
|
||||
infsh me
|
||||
```
|
||||
|
||||
If not installed:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://cli.inference.sh | sh
|
||||
infsh login
|
||||
```
|
||||
|
||||
See `references/authentication.md` for full setup details.
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Always Search First
|
||||
|
||||
Never guess app names — always search to find the correct app ID:
|
||||
|
||||
```bash
|
||||
infsh app list --search flux
|
||||
infsh app list --search video
|
||||
infsh app list --search image
|
||||
```
|
||||
|
||||
### 2. Run an App
|
||||
|
||||
Use the exact app ID from the search results. Always use `--json` for machine-readable output:
|
||||
|
||||
```bash
|
||||
infsh app run <app-id> --input '{"prompt": "your prompt here"}' --json
|
||||
```
|
||||
|
||||
### 3. Parse the Output
|
||||
|
||||
The JSON output contains URLs to generated media. Present these to the user with `MEDIA:<url>` for inline display.
|
||||
|
||||
## Common Commands
|
||||
|
||||
### Image Generation
|
||||
|
||||
```bash
|
||||
# Search for image apps
|
||||
infsh app list --search image
|
||||
|
||||
# FLUX Dev with LoRA
|
||||
infsh app run falai/flux-dev-lora --input '{"prompt": "sunset over mountains", "num_images": 1}' --json
|
||||
|
||||
# Gemini image generation
|
||||
infsh app run google/gemini-2-5-flash-image --input '{"prompt": "futuristic city", "num_images": 1}' --json
|
||||
|
||||
# Seedream (ByteDance)
|
||||
infsh app run bytedance/seedream-5-lite --input '{"prompt": "nature scene"}' --json
|
||||
|
||||
# Grok Imagine (xAI)
|
||||
infsh app run xai/grok-imagine-image --input '{"prompt": "abstract art"}' --json
|
||||
```
|
||||
|
||||
### Video Generation
|
||||
|
||||
```bash
|
||||
# Search for video apps
|
||||
infsh app list --search video
|
||||
|
||||
# Veo 3.1 (Google)
|
||||
infsh app run google/veo-3-1-fast --input '{"prompt": "drone shot of coastline"}' --json
|
||||
|
||||
# Seedance (ByteDance)
|
||||
infsh app run bytedance/seedance-1-5-pro --input '{"prompt": "dancing figure", "resolution": "1080p"}' --json
|
||||
|
||||
# Wan 2.5
|
||||
infsh app run falai/wan-2-5 --input '{"prompt": "person walking through city"}' --json
|
||||
```
|
||||
|
||||
### Local File Uploads
|
||||
|
||||
The CLI automatically uploads local files when you provide a path:
|
||||
|
||||
```bash
|
||||
# Upscale a local image
|
||||
infsh app run falai/topaz-image-upscaler --input '{"image": "/path/to/photo.jpg", "upscale_factor": 2}' --json
|
||||
|
||||
# Image-to-video from local file
|
||||
infsh app run falai/wan-2-5-i2v --input '{"image": "/path/to/image.png", "prompt": "make it move"}' --json
|
||||
|
||||
# Avatar with audio
|
||||
infsh app run bytedance/omnihuman-1-5 --input '{"audio": "/path/to/audio.mp3", "image": "/path/to/face.jpg"}' --json
|
||||
```
|
||||
|
||||
### Search & Research
|
||||
|
||||
```bash
|
||||
infsh app list --search search
|
||||
infsh app run tavily/tavily-search --input '{"query": "latest AI news"}' --json
|
||||
infsh app run exa/exa-search --input '{"query": "machine learning papers"}' --json
|
||||
```
|
||||
|
||||
### Other Categories
|
||||
|
||||
```bash
|
||||
# 3D generation
|
||||
infsh app list --search 3d
|
||||
|
||||
# Audio / TTS
|
||||
infsh app list --search tts
|
||||
|
||||
# Twitter/X automation
|
||||
infsh app list --search twitter
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **Never guess app IDs** — always run `infsh app list --search <term>` first. App IDs change and new apps are added frequently.
|
||||
2. **Always use `--json`** — raw output is hard to parse. The `--json` flag gives structured output with URLs.
|
||||
3. **Check authentication** — if commands fail with auth errors, run `infsh login` or verify `INFSH_API_KEY` is set.
|
||||
4. **Long-running apps** — video generation can take 30-120 seconds. The terminal tool timeout should be sufficient, but warn the user it may take a moment.
|
||||
5. **Input format** — the `--input` flag takes a JSON string. Make sure to properly escape quotes.
|
||||
|
||||
## Reference Docs
|
||||
|
||||
- `references/authentication.md` — Setup, login, API keys
|
||||
- `references/app-discovery.md` — Searching and browsing the app catalog
|
||||
- `references/running-apps.md` — Running apps, input formats, output handling
|
||||
- `references/cli-reference.md` — Complete CLI command reference
|
||||
@@ -0,0 +1,112 @@
|
||||
# Discovering Apps
|
||||
|
||||
## List All Apps
|
||||
|
||||
```bash
|
||||
infsh app list
|
||||
```
|
||||
|
||||
## Pagination
|
||||
|
||||
```bash
|
||||
infsh app list --page 2
|
||||
```
|
||||
|
||||
## Filter by Category
|
||||
|
||||
```bash
|
||||
infsh app list --category image
|
||||
infsh app list --category video
|
||||
infsh app list --category audio
|
||||
infsh app list --category text
|
||||
infsh app list --category other
|
||||
```
|
||||
|
||||
## Search
|
||||
|
||||
```bash
|
||||
infsh app search "flux"
|
||||
infsh app search "video generation"
|
||||
infsh app search "tts" -l
|
||||
infsh app search "image" --category image
|
||||
```
|
||||
|
||||
Or use the flag form:
|
||||
|
||||
```bash
|
||||
infsh app list --search "flux"
|
||||
infsh app list --search "video generation"
|
||||
infsh app list --search "tts"
|
||||
```
|
||||
|
||||
## Featured Apps
|
||||
|
||||
```bash
|
||||
infsh app list --featured
|
||||
```
|
||||
|
||||
## Newest First
|
||||
|
||||
```bash
|
||||
infsh app list --new
|
||||
```
|
||||
|
||||
## Detailed View
|
||||
|
||||
```bash
|
||||
infsh app list -l
|
||||
```
|
||||
|
||||
Shows table with app name, category, description, and featured status.
|
||||
|
||||
## Save to File
|
||||
|
||||
```bash
|
||||
infsh app list --save apps.json
|
||||
```
|
||||
|
||||
## Your Apps
|
||||
|
||||
List apps you've deployed:
|
||||
|
||||
```bash
|
||||
infsh app my
|
||||
infsh app my -l # detailed
|
||||
```
|
||||
|
||||
## Get App Details
|
||||
|
||||
```bash
|
||||
infsh app get falai/flux-dev-lora
|
||||
infsh app get falai/flux-dev-lora --json
|
||||
```
|
||||
|
||||
Shows full app info including input/output schema.
|
||||
|
||||
## Popular Apps by Category
|
||||
|
||||
### Image Generation
|
||||
- `falai/flux-dev-lora` - FLUX.2 Dev (high quality)
|
||||
- `falai/flux-2-klein-lora` - FLUX.2 Klein (fastest)
|
||||
- `infsh/sdxl` - Stable Diffusion XL
|
||||
- `google/gemini-3-pro-image-preview` - Gemini 3 Pro
|
||||
- `xai/grok-imagine-image` - Grok image generation
|
||||
|
||||
### Video Generation
|
||||
- `google/veo-3-1-fast` - Veo 3.1 Fast
|
||||
- `google/veo-3` - Veo 3
|
||||
- `bytedance/seedance-1-5-pro` - Seedance 1.5 Pro
|
||||
- `infsh/ltx-video-2` - LTX Video 2 (with audio)
|
||||
- `bytedance/omnihuman-1-5` - OmniHuman avatar
|
||||
|
||||
### Audio
|
||||
- `infsh/dia-tts` - Conversational TTS
|
||||
- `infsh/kokoro-tts` - Kokoro TTS
|
||||
- `infsh/fast-whisper-large-v3` - Fast transcription
|
||||
- `infsh/diffrythm` - Music generation
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Browsing the Grid](https://inference.sh/docs/apps/browsing-grid) - Visual app browsing
|
||||
- [Apps Overview](https://inference.sh/docs/apps/overview) - Understanding apps
|
||||
- [Running Apps](https://inference.sh/docs/apps/running) - How to run apps
|
||||
@@ -0,0 +1,59 @@
|
||||
# Authentication & Setup
|
||||
|
||||
## Install the CLI
|
||||
|
||||
```bash
|
||||
curl -fsSL https://cli.inference.sh | sh
|
||||
```
|
||||
|
||||
## Login
|
||||
|
||||
```bash
|
||||
infsh login
|
||||
```
|
||||
|
||||
This opens a browser for authentication. After login, credentials are stored locally.
|
||||
|
||||
## Check Authentication
|
||||
|
||||
```bash
|
||||
infsh me
|
||||
```
|
||||
|
||||
Shows your user info if authenticated.
|
||||
|
||||
## Environment Variable
|
||||
|
||||
For CI/CD or scripts, set your API key:
|
||||
|
||||
```bash
|
||||
export INFSH_API_KEY=your-api-key
|
||||
```
|
||||
|
||||
The environment variable overrides the config file.
|
||||
|
||||
## Update CLI
|
||||
|
||||
```bash
|
||||
infsh update
|
||||
```
|
||||
|
||||
Or reinstall:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://cli.inference.sh | sh
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Error | Solution |
|
||||
|-------|----------|
|
||||
| "not authenticated" | Run `infsh login` |
|
||||
| "command not found" | Reinstall CLI or add to PATH |
|
||||
| "API key invalid" | Check `INFSH_API_KEY` or re-login |
|
||||
|
||||
## Documentation
|
||||
|
||||
- [CLI Setup](https://inference.sh/docs/extend/cli-setup) - Complete CLI installation guide
|
||||
- [API Authentication](https://inference.sh/docs/api/authentication) - API key management
|
||||
- [Secrets](https://inference.sh/docs/secrets/overview) - Managing credentials
|
||||
@@ -0,0 +1,104 @@
|
||||
# CLI Reference
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
curl -fsSL https://cli.inference.sh | sh
|
||||
```
|
||||
|
||||
## Global Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `infsh help` | Show help |
|
||||
| `infsh version` | Show CLI version |
|
||||
| `infsh update` | Update CLI to latest |
|
||||
| `infsh login` | Authenticate |
|
||||
| `infsh me` | Show current user |
|
||||
|
||||
## App Commands
|
||||
|
||||
### Discovery
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `infsh app list` | List available apps |
|
||||
| `infsh app list --category <cat>` | Filter by category (image, video, audio, text, other) |
|
||||
| `infsh app search <query>` | Search apps |
|
||||
| `infsh app list --search <query>` | Search apps (flag form) |
|
||||
| `infsh app list --featured` | Show featured apps |
|
||||
| `infsh app list --new` | Sort by newest |
|
||||
| `infsh app list --page <n>` | Pagination |
|
||||
| `infsh app list -l` | Detailed table view |
|
||||
| `infsh app list --save <file>` | Save to JSON file |
|
||||
| `infsh app my` | List your deployed apps |
|
||||
| `infsh app get <app>` | Get app details |
|
||||
| `infsh app get <app> --json` | Get app details as JSON |
|
||||
|
||||
### Execution
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `infsh app run <app> --input <file>` | Run app with input file |
|
||||
| `infsh app run <app> --input '<json>'` | Run with inline JSON |
|
||||
| `infsh app run <app> --input <file> --no-wait` | Run without waiting for completion |
|
||||
| `infsh app sample <app>` | Show sample input |
|
||||
| `infsh app sample <app> --save <file>` | Save sample to file |
|
||||
|
||||
## Task Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `infsh task get <task-id>` | Get task status and result |
|
||||
| `infsh task get <task-id> --json` | Get task as JSON |
|
||||
| `infsh task get <task-id> --save <file>` | Save task result to file |
|
||||
|
||||
### Development
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `infsh app init` | Create new app (interactive) |
|
||||
| `infsh app init <name>` | Create new app with name |
|
||||
| `infsh app test --input <file>` | Test app locally |
|
||||
| `infsh app deploy` | Deploy app |
|
||||
| `infsh app deploy --dry-run` | Validate without deploying |
|
||||
| `infsh app pull <id>` | Pull app source |
|
||||
| `infsh app pull --all` | Pull all your apps |
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `INFSH_API_KEY` | API key (overrides config) |
|
||||
|
||||
## Shell Completions
|
||||
|
||||
```bash
|
||||
# Bash
|
||||
infsh completion bash > /etc/bash_completion.d/infsh
|
||||
|
||||
# Zsh
|
||||
infsh completion zsh > "${fpath[1]}/_infsh"
|
||||
|
||||
# Fish
|
||||
infsh completion fish > ~/.config/fish/completions/infsh.fish
|
||||
```
|
||||
|
||||
## App Name Format
|
||||
|
||||
Apps use the format `namespace/app-name`:
|
||||
|
||||
- `falai/flux-dev-lora` - fal.ai's FLUX 2 Dev
|
||||
- `google/veo-3` - Google's Veo 3
|
||||
- `infsh/sdxl` - inference.sh's SDXL
|
||||
- `bytedance/seedance-1-5-pro` - ByteDance's Seedance
|
||||
- `xai/grok-imagine-image` - xAI's Grok
|
||||
|
||||
Version pinning: `namespace/app-name@version`
|
||||
|
||||
## Documentation
|
||||
|
||||
- [CLI Setup](https://inference.sh/docs/extend/cli-setup) - Complete CLI installation guide
|
||||
- [Running Apps](https://inference.sh/docs/apps/running) - How to run apps via CLI
|
||||
- [Creating an App](https://inference.sh/docs/extend/creating-app) - Build your own apps
|
||||
- [Deploying](https://inference.sh/docs/extend/deploying) - Deploy apps to the cloud
|
||||
@@ -0,0 +1,171 @@
|
||||
# Running Apps
|
||||
|
||||
## Basic Run
|
||||
|
||||
```bash
|
||||
infsh app run user/app-name --input input.json
|
||||
```
|
||||
|
||||
## Inline JSON
|
||||
|
||||
```bash
|
||||
infsh app run falai/flux-dev-lora --input '{"prompt": "a sunset over mountains"}'
|
||||
```
|
||||
|
||||
## Version Pinning
|
||||
|
||||
```bash
|
||||
infsh app run user/app-name@1.0.0 --input input.json
|
||||
```
|
||||
|
||||
## Local File Uploads
|
||||
|
||||
The CLI automatically uploads local files when you provide a file path instead of a URL. Any field that accepts a URL also accepts a local path:
|
||||
|
||||
```bash
|
||||
# Upscale a local image
|
||||
infsh app run falai/topaz-image-upscaler --input '{"image": "/path/to/photo.jpg", "upscale_factor": 2}'
|
||||
|
||||
# Image-to-video from local file
|
||||
infsh app run falai/wan-2-5-i2v --input '{"image": "./my-image.png", "prompt": "make it move"}'
|
||||
|
||||
# Avatar with local audio and image
|
||||
infsh app run bytedance/omnihuman-1-5 --input '{"audio": "/path/to/speech.mp3", "image": "/path/to/face.jpg"}'
|
||||
|
||||
# Post tweet with local media
|
||||
infsh app run x/post-create --input '{"text": "Check this out!", "media": "./screenshot.png"}'
|
||||
```
|
||||
|
||||
Supported paths:
|
||||
- Absolute paths: `/home/user/images/photo.jpg`
|
||||
- Relative paths: `./image.png`, `../data/video.mp4`
|
||||
- Home directory: `~/Pictures/photo.jpg`
|
||||
|
||||
## Generate Sample Input
|
||||
|
||||
Before running, generate a sample input file:
|
||||
|
||||
```bash
|
||||
infsh app sample falai/flux-dev-lora
|
||||
```
|
||||
|
||||
Save to file:
|
||||
|
||||
```bash
|
||||
infsh app sample falai/flux-dev-lora --save input.json
|
||||
```
|
||||
|
||||
Then edit `input.json` and run:
|
||||
|
||||
```bash
|
||||
infsh app run falai/flux-dev-lora --input input.json
|
||||
```
|
||||
|
||||
## Workflow Example
|
||||
|
||||
### Image Generation with FLUX
|
||||
|
||||
```bash
|
||||
# 1. Get app details
|
||||
infsh app get falai/flux-dev-lora
|
||||
|
||||
# 2. Generate sample input
|
||||
infsh app sample falai/flux-dev-lora --save input.json
|
||||
|
||||
# 3. Edit input.json
|
||||
# {
|
||||
# "prompt": "a cat astronaut floating in space",
|
||||
# "num_images": 1,
|
||||
# "image_size": "landscape_16_9"
|
||||
# }
|
||||
|
||||
# 4. Run
|
||||
infsh app run falai/flux-dev-lora --input input.json
|
||||
```
|
||||
|
||||
### Video Generation with Veo
|
||||
|
||||
```bash
|
||||
# 1. Generate sample
|
||||
infsh app sample google/veo-3-1-fast --save input.json
|
||||
|
||||
# 2. Edit prompt
|
||||
# {
|
||||
# "prompt": "A drone shot flying over a forest at sunset"
|
||||
# }
|
||||
|
||||
# 3. Run
|
||||
infsh app run google/veo-3-1-fast --input input.json
|
||||
```
|
||||
|
||||
### Text-to-Speech
|
||||
|
||||
```bash
|
||||
# Quick inline run
|
||||
infsh app run falai/kokoro-tts --input '{"text": "Hello, this is a test."}'
|
||||
```
|
||||
|
||||
## Task Tracking
|
||||
|
||||
When you run an app, the CLI shows the task ID:
|
||||
|
||||
```
|
||||
Running falai/flux-dev-lora
|
||||
Task ID: abc123def456
|
||||
```
|
||||
|
||||
For long-running tasks, you can check status anytime:
|
||||
|
||||
```bash
|
||||
# Check task status
|
||||
infsh task get abc123def456
|
||||
|
||||
# Get result as JSON
|
||||
infsh task get abc123def456 --json
|
||||
|
||||
# Save result to file
|
||||
infsh task get abc123def456 --save result.json
|
||||
```
|
||||
|
||||
### Run Without Waiting
|
||||
|
||||
For very long tasks, run in background:
|
||||
|
||||
```bash
|
||||
# Submit and return immediately
|
||||
infsh app run google/veo-3 --input input.json --no-wait
|
||||
|
||||
# Check later
|
||||
infsh task get <task-id>
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
The CLI returns the app output directly. For file outputs (images, videos, audio), you'll receive URLs to download.
|
||||
|
||||
Example output:
|
||||
|
||||
```json
|
||||
{
|
||||
"images": [
|
||||
{
|
||||
"url": "https://cloud.inference.sh/...",
|
||||
"content_type": "image/png"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Error | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| "invalid input" | Schema mismatch | Check `infsh app get` for required fields |
|
||||
| "app not found" | Wrong app name | Check `infsh app list --search` |
|
||||
| "quota exceeded" | Out of credits | Check account balance |
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Running Apps](https://inference.sh/docs/apps/running) - Complete running apps guide
|
||||
- [Streaming Results](https://inference.sh/docs/api/sdk/streaming) - Real-time progress updates
|
||||
- [Setup Parameters](https://inference.sh/docs/apps/setup-parameters) - Configuring app inputs
|
||||
+281
@@ -0,0 +1,281 @@
|
||||
---
|
||||
name: docker-management
|
||||
description: Manage Docker containers, images, volumes, networks, and Compose stacks — lifecycle ops, debugging, cleanup, and Dockerfile optimization.
|
||||
version: 1.0.0
|
||||
author: sprmn24
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [docker, containers, devops, infrastructure, compose, images, volumes, networks, debugging]
|
||||
category: devops
|
||||
requires_toolsets: [terminal]
|
||||
---
|
||||
|
||||
# Docker Management
|
||||
|
||||
Manage Docker containers, images, volumes, networks, and Compose stacks using standard Docker CLI commands. No additional dependencies beyond Docker itself.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Run, stop, restart, remove, or inspect containers
|
||||
- Build, pull, push, tag, or clean up Docker images
|
||||
- Work with Docker Compose (multi-service stacks)
|
||||
- Manage volumes or networks
|
||||
- Debug a crashing container or analyze logs
|
||||
- Check Docker disk usage or free up space
|
||||
- Review or optimize a Dockerfile
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker Engine installed and running
|
||||
- User added to the `docker` group (or use `sudo`)
|
||||
- Docker Compose v2 (included with modern Docker installations)
|
||||
|
||||
Quick check:
|
||||
|
||||
```bash
|
||||
docker --version && docker compose version
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Task | Command |
|
||||
|------|---------|
|
||||
| Run container (background) | `docker run -d --name NAME IMAGE` |
|
||||
| Stop + remove | `docker stop NAME && docker rm NAME` |
|
||||
| View logs (follow) | `docker logs --tail 50 -f NAME` |
|
||||
| Shell into container | `docker exec -it NAME /bin/sh` |
|
||||
| List all containers | `docker ps -a` |
|
||||
| Build image | `docker build -t TAG .` |
|
||||
| Compose up | `docker compose up -d` |
|
||||
| Compose down | `docker compose down` |
|
||||
| Disk usage | `docker system df` |
|
||||
| Cleanup dangling | `docker image prune && docker container prune` |
|
||||
|
||||
## Procedure
|
||||
|
||||
### 1. Identify the domain
|
||||
|
||||
Figure out which area the request falls into:
|
||||
|
||||
- **Container lifecycle** → run, stop, start, restart, rm, pause/unpause
|
||||
- **Container interaction** → exec, cp, logs, inspect, stats
|
||||
- **Image management** → build, pull, push, tag, rmi, save/load
|
||||
- **Docker Compose** → up, down, ps, logs, exec, build, config
|
||||
- **Volumes & networks** → create, inspect, rm, prune, connect
|
||||
- **Troubleshooting** → log analysis, exit codes, resource issues
|
||||
|
||||
### 2. Container operations
|
||||
|
||||
**Run a new container:**
|
||||
|
||||
```bash
|
||||
# Detached service with port mapping
|
||||
docker run -d --name web -p 8080:80 nginx
|
||||
|
||||
# With environment variables
|
||||
docker run -d -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=mydb --name db postgres:16
|
||||
|
||||
# With persistent data (named volume)
|
||||
docker run -d -v pgdata:/var/lib/postgresql/data --name db postgres:16
|
||||
|
||||
# For development (bind mount source code)
|
||||
docker run -d -v $(pwd)/src:/app/src -p 3000:3000 --name dev my-app
|
||||
|
||||
# Interactive debugging (auto-remove on exit)
|
||||
docker run -it --rm ubuntu:22.04 /bin/bash
|
||||
|
||||
# With resource limits and restart policy
|
||||
docker run -d --memory=512m --cpus=1.5 --restart=unless-stopped --name app my-app
|
||||
```
|
||||
|
||||
Key flags: `-d` detached, `-it` interactive+tty, `--rm` auto-remove, `-p` port (host:container), `-e` env var, `-v` volume, `--name` name, `--restart` restart policy.
|
||||
|
||||
**Manage running containers:**
|
||||
|
||||
```bash
|
||||
docker ps # running containers
|
||||
docker ps -a # all (including stopped)
|
||||
docker stop NAME # graceful stop
|
||||
docker start NAME # start stopped container
|
||||
docker restart NAME # stop + start
|
||||
docker rm NAME # remove stopped container
|
||||
docker rm -f NAME # force remove running container
|
||||
docker container prune # remove ALL stopped containers
|
||||
```
|
||||
|
||||
**Interact with containers:**
|
||||
|
||||
```bash
|
||||
docker exec -it NAME /bin/sh # shell access (use /bin/bash if available)
|
||||
docker exec NAME env # view environment variables
|
||||
docker exec -u root NAME apt update # run as specific user
|
||||
docker logs --tail 100 -f NAME # follow last 100 lines
|
||||
docker logs --since 2h NAME # logs from last 2 hours
|
||||
docker cp NAME:/path/file ./local # copy file from container
|
||||
docker cp ./file NAME:/path/ # copy file to container
|
||||
docker inspect NAME # full container details (JSON)
|
||||
docker stats --no-stream # resource usage snapshot
|
||||
docker top NAME # running processes
|
||||
```
|
||||
|
||||
### 3. Image management
|
||||
|
||||
```bash
|
||||
# Build
|
||||
docker build -t my-app:latest .
|
||||
docker build -t my-app:prod -f Dockerfile.prod .
|
||||
docker build --no-cache -t my-app . # clean rebuild
|
||||
DOCKER_BUILDKIT=1 docker build -t my-app . # faster with BuildKit
|
||||
|
||||
# Pull and push
|
||||
docker pull node:20-alpine
|
||||
docker login ghcr.io
|
||||
docker tag my-app:latest registry/my-app:v1.0
|
||||
docker push registry/my-app:v1.0
|
||||
|
||||
# Inspect
|
||||
docker images # list local images
|
||||
docker history IMAGE # see layers
|
||||
docker inspect IMAGE # full details
|
||||
|
||||
# Cleanup
|
||||
docker image prune # remove dangling (untagged) images
|
||||
docker image prune -a # remove ALL unused images (careful!)
|
||||
docker image prune -a --filter "until=168h" # unused images older than 7 days
|
||||
```
|
||||
|
||||
### 4. Docker Compose
|
||||
|
||||
```bash
|
||||
# Start/stop
|
||||
docker compose up -d # start all services detached
|
||||
docker compose up -d --build # rebuild images before starting
|
||||
docker compose down # stop and remove containers
|
||||
docker compose down -v # also remove volumes (DESTROYS DATA)
|
||||
|
||||
# Monitoring
|
||||
docker compose ps # list services
|
||||
docker compose logs -f api # follow logs for specific service
|
||||
docker compose logs --tail 50 # last 50 lines all services
|
||||
|
||||
# Interaction
|
||||
docker compose exec api /bin/sh # shell into running service
|
||||
docker compose run --rm api npm test # one-off command (new container)
|
||||
docker compose restart api # restart specific service
|
||||
|
||||
# Validation
|
||||
docker compose config # validate and view resolved config
|
||||
```
|
||||
|
||||
**Minimal compose.yml example:**
|
||||
|
||||
```yaml
|
||||
services:
|
||||
api:
|
||||
build: .
|
||||
ports:
|
||||
- "3000:3000"
|
||||
environment:
|
||||
- DATABASE_URL=postgres://user:pass@db:5432/mydb
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
|
||||
db:
|
||||
image: postgres:16-alpine
|
||||
environment:
|
||||
POSTGRES_USER: user
|
||||
POSTGRES_PASSWORD: pass
|
||||
POSTGRES_DB: mydb
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U user"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
```
|
||||
|
||||
### 5. Volumes and networks
|
||||
|
||||
```bash
|
||||
# Volumes
|
||||
docker volume ls # list volumes
|
||||
docker volume create mydata # create named volume
|
||||
docker volume inspect mydata # details (mount point, etc.)
|
||||
docker volume rm mydata # remove (fails if in use)
|
||||
docker volume prune # remove unused volumes
|
||||
|
||||
# Networks
|
||||
docker network ls # list networks
|
||||
docker network create mynet # create bridge network
|
||||
docker network inspect mynet # details (connected containers)
|
||||
docker network connect mynet NAME # attach container to network
|
||||
docker network disconnect mynet NAME # detach container
|
||||
docker network rm mynet # remove network
|
||||
docker network prune # remove unused networks
|
||||
```
|
||||
|
||||
### 6. Disk usage and cleanup
|
||||
|
||||
Always start with a diagnostic before cleaning:
|
||||
|
||||
```bash
|
||||
# Check what's using space
|
||||
docker system df # summary
|
||||
docker system df -v # detailed breakdown
|
||||
|
||||
# Targeted cleanup (safe)
|
||||
docker container prune # stopped containers
|
||||
docker image prune # dangling images
|
||||
docker volume prune # unused volumes
|
||||
docker network prune # unused networks
|
||||
|
||||
# Aggressive cleanup (confirm with user first!)
|
||||
docker system prune # containers + images + networks
|
||||
docker system prune -a # also unused images
|
||||
docker system prune -a --volumes # EVERYTHING — named volumes too
|
||||
```
|
||||
|
||||
**Warning:** Never run `docker system prune -a --volumes` without confirming with the user. This removes named volumes with potentially important data.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
| Problem | Cause | Fix |
|
||||
|---------|-------|-----|
|
||||
| Container exits immediately | Main process finished or crashed | Check `docker logs NAME`, try `docker run -it --entrypoint /bin/sh IMAGE` |
|
||||
| "port is already allocated" | Another process using that port | `docker ps` or `lsof -i :PORT` to find it |
|
||||
| "no space left on device" | Docker disk full | `docker system df` then targeted prune |
|
||||
| Can't connect to container | App binds to 127.0.0.1 inside container | App must bind to `0.0.0.0`, check `-p` mapping |
|
||||
| Permission denied on volume | UID/GID mismatch host vs container | Use `--user $(id -u):$(id -g)` or fix permissions |
|
||||
| Compose services can't reach each other | Wrong network or service name | Services use service name as hostname, check `docker compose config` |
|
||||
| Build cache not working | Layer order wrong in Dockerfile | Put rarely-changing layers first (deps before source code) |
|
||||
| Image too large | No multi-stage build, no .dockerignore | Use multi-stage builds, add `.dockerignore` |
|
||||
|
||||
## Verification
|
||||
|
||||
After any Docker operation, verify the result:
|
||||
|
||||
- **Container started?** → `docker ps` (check status is "Up")
|
||||
- **Logs clean?** → `docker logs --tail 20 NAME` (no errors)
|
||||
- **Port accessible?** → `curl -s http://localhost:PORT` or `docker port NAME`
|
||||
- **Image built?** → `docker images | grep TAG`
|
||||
- **Compose stack healthy?** → `docker compose ps` (all services "running" or "healthy")
|
||||
- **Disk freed?** → `docker system df` (compare before/after)
|
||||
|
||||
## Dockerfile Optimization Tips
|
||||
|
||||
When reviewing or creating a Dockerfile, suggest these improvements:
|
||||
|
||||
1. **Multi-stage builds** — separate build environment from runtime to reduce final image size
|
||||
2. **Layer ordering** — put dependencies before source code so changes don't invalidate cached layers
|
||||
3. **Combine RUN commands** — fewer layers, smaller image
|
||||
4. **Use .dockerignore** — exclude `node_modules`, `.git`, `__pycache__`, etc.
|
||||
5. **Pin base image versions** — `node:20-alpine` not `node:latest`
|
||||
6. **Run as non-root** — add `USER` instruction for security
|
||||
7. **Use slim/alpine bases** — `python:3.12-slim` not `python:3.12`
|
||||
@@ -0,0 +1,309 @@
|
||||
---
|
||||
name: pinggy-tunnel
|
||||
description: Zero-install localhost tunnels over SSH via Pinggy.
|
||||
version: 0.1.0
|
||||
author: Teknium (teknium1), Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Pinggy, Tunnel, Networking, SSH, Webhook, Localhost]
|
||||
related_skills: [cloudflared-quick-tunnel, webhook-subscriptions]
|
||||
---
|
||||
|
||||
# Pinggy Tunnel Skill
|
||||
|
||||
Expose a local service (dev server, webhook receiver, MCP endpoint, demo) to the public internet using a Pinggy SSH reverse tunnel. No daemon to install — the user's stock SSH client connects to `a.pinggy.io:443` and Pinggy hands back a public HTTP/HTTPS URL.
|
||||
|
||||
Free tier: 60-minute tunnels, random subdomain, no signup. Pro tier ($3/mo) is an opt-in with a token.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks to "expose this locally", "share my dev server", "make this URL public", "tunnel port N", "get a public URL for a webhook"
|
||||
- Need to receive a webhook callback during a local task (Stripe, GitHub, Discord, AgentMail)
|
||||
- Sharing a one-off HTTP demo (MCP server, Ollama/vLLM endpoint, dashboard) with a remote party
|
||||
- The host has SSH but no `cloudflared` / `ngrok` binary, and installing one would be overkill
|
||||
|
||||
If the host already has `cloudflared` configured, prefer the `cloudflared-quick-tunnel` skill — Cloudflare quick tunnels don't expire after 60 minutes.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `ssh` on PATH (`ssh -V`). Default on Linux, macOS, and Windows 10+. No other install.
|
||||
- A local service listening on `127.0.0.1:<port>` before the tunnel starts. Pinggy will return URLs but they'll 502 until the local origin is up.
|
||||
|
||||
Optional:
|
||||
|
||||
- `PINGGY_TOKEN` env var for paid Pro features (persistent subdomain, custom domain, multiple tunnels, no 60-minute cap). Free tier needs no credentials.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
```bash
|
||||
# Plain HTTP/HTTPS tunnel for port 8000 (free tier)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -o ServerAliveInterval=30 \
|
||||
-R0:localhost:8000 free@a.pinggy.io
|
||||
|
||||
# TCP tunnel (databases, raw SSH, etc.)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:5432 tcp@a.pinggy.io
|
||||
|
||||
# TLS tunnel (Pinggy can't decrypt — bring your own certs at origin)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:443 tls@a.pinggy.io
|
||||
|
||||
# Basic auth gate (b:user:pass)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:8000 \
|
||||
"b:admin:secret+free@a.pinggy.io"
|
||||
|
||||
# Bearer token gate (k:token)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:8000 \
|
||||
"k:mysecrettoken+free@a.pinggy.io"
|
||||
|
||||
# IP whitelist (w:CIDR)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:8000 \
|
||||
"w:203.0.113.0/24+free@a.pinggy.io"
|
||||
|
||||
# Enable CORS + force HTTPS redirect
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:8000 \
|
||||
"co+x:https+free@a.pinggy.io"
|
||||
|
||||
# Pro tier (persistent URL, no 60-min cap)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:8000 "$PINGGY_TOKEN+a.pinggy.io"
|
||||
```
|
||||
|
||||
## Procedure — Start a Tunnel and Get the URL
|
||||
|
||||
The model SHOULD use the `terminal` tool. The tunnel must stay alive for the duration of the share, so run it as a background process and parse the public URL from stdout.
|
||||
|
||||
### 1. Confirm a local origin is up
|
||||
|
||||
```bash
|
||||
curl -sI http://127.0.0.1:8000/ | head -1
|
||||
# expect HTTP/1.x 200 (or any non-connection-refused response)
|
||||
```
|
||||
|
||||
If nothing is listening yet, start it first (e.g. `python3 -m http.server 8000 --bind 127.0.0.1`). Pinggy will happily return a URL pointed at nothing — the user will see 502 until the origin comes up.
|
||||
|
||||
### 2. Launch the tunnel as a background process
|
||||
|
||||
Use `terminal(background=True)` and capture output to a logfile (Pinggy prints the URLs on stdout, then keeps the connection open):
|
||||
|
||||
```bash
|
||||
LOG=/tmp/pinggy-8000.log
|
||||
nohup ssh -p 443 \
|
||||
-o StrictHostKeyChecking=no \
|
||||
-o UserKnownHostsFile=/dev/null \
|
||||
-o ServerAliveInterval=30 \
|
||||
-o ServerAliveCountMax=3 \
|
||||
-R0:localhost:8000 free@a.pinggy.io \
|
||||
> "$LOG" 2>&1 &
|
||||
echo $! > /tmp/pinggy-8000.pid
|
||||
```
|
||||
|
||||
`StrictHostKeyChecking=no` + `UserKnownHostsFile=/dev/null` skips the first-run host-key prompt. `ServerAliveInterval=30` keeps the SSH session from getting torn down by an idle NAT.
|
||||
|
||||
### 3. Parse the URL out of the log
|
||||
|
||||
```bash
|
||||
sleep 4
|
||||
grep -oE 'https://[a-z0-9-]+\.[a-z]+\.pinggy\.link' /tmp/pinggy-8000.log | head -1
|
||||
```
|
||||
|
||||
Expected output looks like:
|
||||
|
||||
```
|
||||
You are not authenticated.
|
||||
Your tunnel will expire in 60 minutes.
|
||||
http://yqycl-98-162-69-48.a.free.pinggy.link
|
||||
https://yqycl-98-162-69-48.a.free.pinggy.link
|
||||
```
|
||||
|
||||
Hand the `https://...pinggy.link` URL to the user.
|
||||
|
||||
### 4. Verify
|
||||
|
||||
```bash
|
||||
curl -sI https://<the-url>/ | head -3
|
||||
# expect 200/302/whatever the local origin actually returns
|
||||
```
|
||||
|
||||
If you get `502 Bad Gateway`, the SSH session is up but the local origin isn't listening — fix step 1 first.
|
||||
|
||||
### 5. Teardown
|
||||
|
||||
```bash
|
||||
kill "$(cat /tmp/pinggy-8000.pid)"
|
||||
# or, if the pid file got lost:
|
||||
pkill -f 'ssh -p 443 .* free@a\.pinggy\.io'
|
||||
```
|
||||
|
||||
If you have a session_id from `terminal(background=True)`, prefer `process(action='kill', session_id=...)`.
|
||||
|
||||
## Access Control via Username Keywords
|
||||
|
||||
Pinggy stacks control flags into the SSH username separated by `+`. Always quote the whole `user@host` argument when it contains a `+`:
|
||||
|
||||
| Keyword | Effect |
|
||||
|---------|--------|
|
||||
| `b:user:pass` | HTTP Basic auth gate |
|
||||
| `k:token` | Bearer-token header gate (`Authorization: Bearer <token>`) |
|
||||
| `w:CIDR` | IP whitelist (single IP or CIDR, repeatable) |
|
||||
| `co` | Add `Access-Control-Allow-Origin: *` (CORS) |
|
||||
| `x:https` | Force HTTPS — auto-redirect HTTP to HTTPS |
|
||||
| `a:Name:Value` | Add request header |
|
||||
| `u:Name:Value` | Update request header |
|
||||
| `r:Name` | Remove request header |
|
||||
| `qr` | Print a QR code of the URL to stdout (handy for mobile sharing) |
|
||||
|
||||
Combine freely: `"b:admin:secret+co+x:https+free@a.pinggy.io"`.
|
||||
|
||||
## Web Debugger (optional)
|
||||
|
||||
Pinggy can mirror the inbound traffic to `localhost:4300` for inspection. Add a local forward to the SSH command:
|
||||
|
||||
```bash
|
||||
ssh -p 443 -L4300:localhost:4300 -R0:localhost:8000 free@a.pinggy.io
|
||||
```
|
||||
|
||||
Then open `http://localhost:4300` in a browser to see live request/response pairs.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **60-minute hard cap on the free tier.** The SSH session terminates at the 60-minute mark; the URL goes dead. For longer shares, either use `PINGGY_TOKEN` (Pro) or auto-restart with a shell loop (note that the URL changes on every restart for free-tier).
|
||||
- **Free-tier URL is random and changes on restart.** Don't bookmark it, don't paste it into a config file. Re-parse from the log each time.
|
||||
- **Concurrent free tunnels are limited to one per source IP.** Starting a second tunnel from the same machine usually kills the first. Pro tier lifts this.
|
||||
- **`+` in usernames must be quoted.** Bare `ssh ... b:admin:secret+free@a.pinggy.io` works in bash but breaks under shells that treat `+` specially or when assembled programmatically. Always wrap in double quotes.
|
||||
- **Don't tunnel anything sensitive without an access-control flag.** A bare HTTP tunnel is reachable by anyone with the URL. Use `b:`, `k:`, or `w:` for non-public services.
|
||||
- **`process(action='log')` may miss SSH banner output.** Pinggy prints the URLs and then the SSH session goes interactive. Always redirect to a logfile and `grep` the file directly — same pattern as `cloudflared-quick-tunnel`.
|
||||
- **Host-key prompt on first run.** Default OpenSSH config asks the user to accept Pinggy's host key. Always pass `-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null` for unattended runs.
|
||||
- **TCP and TLS tunnels return a `<subdomain>.a.pinggy.online:<port>` pair, not an https URL.** Parse with a different regex (`tcp://` and a port). Don't assume every Pinggy tunnel is HTTP.
|
||||
- **Pro mode requires the token as the username, not a flag.** Use `"$PINGGY_TOKEN+a.pinggy.io"` (no `free@`). With a token you can also add `:persistent` for a stable subdomain — see `pinggy.io/docs/`.
|
||||
|
||||
## Recipes
|
||||
|
||||
Composite patterns combining a local origin with a Pinggy tunnel. Each recipe is self-contained — start the origin, start the tunnel, parse the URL, hand it back to the user.
|
||||
|
||||
### Recipe 1 — Receive a webhook callback
|
||||
|
||||
Use this when an external service (Stripe, GitHub, Discord, AgentMail, etc.) needs to POST to a publicly reachable URL during a local task.
|
||||
|
||||
```bash
|
||||
# 1. Tiny capturing server: every request gets appended to /tmp/webhook-hits.log
|
||||
cat >/tmp/webhook-server.py <<'PY'
|
||||
import http.server, json, datetime, pathlib
|
||||
LOG = pathlib.Path("/tmp/webhook-hits.log")
|
||||
class H(http.server.BaseHTTPRequestHandler):
|
||||
def _capture(self):
|
||||
n = int(self.headers.get("content-length") or 0)
|
||||
body = self.rfile.read(n).decode("utf-8", "replace") if n else ""
|
||||
rec = {"t": datetime.datetime.utcnow().isoformat(), "path": self.path,
|
||||
"method": self.command, "headers": dict(self.headers), "body": body}
|
||||
with LOG.open("a") as f: f.write(json.dumps(rec) + "\n")
|
||||
self.send_response(200); self.send_header("content-type","application/json")
|
||||
self.end_headers(); self.wfile.write(b'{"ok":true}\n')
|
||||
def do_GET(self): self._capture()
|
||||
def do_POST(self): self._capture()
|
||||
def log_message(self,*a,**k): pass
|
||||
http.server.HTTPServer(("127.0.0.1", 18080), H).serve_forever()
|
||||
PY
|
||||
nohup python3 /tmp/webhook-server.py >/tmp/webhook-server.log 2>&1 &
|
||||
echo $! >/tmp/webhook-server.pid
|
||||
|
||||
# 2. Tunnel — bearer-token-gate so randos can't pollute the capture log
|
||||
nohup ssh -p 443 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
|
||||
-o ServerAliveInterval=30 \
|
||||
-R0:localhost:18080 "k:$(openssl rand -hex 12)+free@a.pinggy.io" \
|
||||
>/tmp/webhook-pinggy.log 2>&1 &
|
||||
echo $! >/tmp/webhook-pinggy.pid
|
||||
sleep 5
|
||||
URL=$(grep -oE 'https://[a-z0-9-]+\.[a-z]+\.pinggy\.link' /tmp/webhook-pinggy.log | head -1)
|
||||
echo "Webhook URL: $URL"
|
||||
|
||||
# 3. While the agent works, watch hits land
|
||||
tail -f /tmp/webhook-hits.log
|
||||
```
|
||||
|
||||
Hand `$URL` to the service that needs to call you. Teardown: `kill $(cat /tmp/webhook-server.pid) $(cat /tmp/webhook-pinggy.pid)`.
|
||||
|
||||
### Recipe 2 — Expose an MCP server over HTTP/SSE
|
||||
|
||||
Use when a remote MCP client (Claude Desktop on another machine, a teammate's editor, etc.) needs to reach an MCP server running on the local box. Only works for MCP servers that speak HTTP transport — stdio-mode servers can't be tunneled.
|
||||
|
||||
```bash
|
||||
# 1. Start the MCP server in HTTP mode (example: a FastMCP server on port 8765)
|
||||
nohup python3 my_mcp_server.py --transport http --port 8765 \
|
||||
>/tmp/mcp-server.log 2>&1 &
|
||||
echo $! >/tmp/mcp-server.pid
|
||||
|
||||
# 2. Tunnel with a bearer token — MCP traffic should not be open to the internet
|
||||
TOKEN=$(openssl rand -hex 16)
|
||||
nohup ssh -p 443 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
|
||||
-o ServerAliveInterval=30 \
|
||||
-R0:localhost:8765 "k:$TOKEN+free@a.pinggy.io" \
|
||||
>/tmp/mcp-pinggy.log 2>&1 &
|
||||
echo $! >/tmp/mcp-pinggy.pid
|
||||
sleep 5
|
||||
URL=$(grep -oE 'https://[a-z0-9-]+\.[a-z]+\.pinggy\.link' /tmp/mcp-pinggy.log | head -1)
|
||||
echo "MCP URL: $URL"
|
||||
echo "Bearer token: $TOKEN"
|
||||
```
|
||||
|
||||
The remote client connects to `$URL` with `Authorization: Bearer $TOKEN`. Hermes' own native MCP client config: `{"transport": "http", "url": "<URL>", "headers": {"Authorization": "Bearer <TOKEN>"}}`.
|
||||
|
||||
### Recipe 3 — Expose a local LLM endpoint (Ollama / vLLM / llama.cpp)
|
||||
|
||||
Share a local model with a remote caller (another agent, a phone, a teammate). Ollama listens on `:11434`, vLLM and llama.cpp typically on `:8000`.
|
||||
|
||||
```bash
|
||||
# Pre-req: the model server is already running on 127.0.0.1:11434 (Ollama default)
|
||||
TOKEN=$(openssl rand -hex 16)
|
||||
nohup ssh -p 443 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
|
||||
-o ServerAliveInterval=30 \
|
||||
-R0:localhost:11434 "k:$TOKEN+co+free@a.pinggy.io" \
|
||||
>/tmp/llm-pinggy.log 2>&1 &
|
||||
echo $! >/tmp/llm-pinggy.pid
|
||||
sleep 5
|
||||
URL=$(grep -oE 'https://[a-z0-9-]+\.[a-z]+\.pinggy\.link' /tmp/llm-pinggy.log | head -1)
|
||||
echo "Endpoint: $URL"
|
||||
echo "Token: $TOKEN"
|
||||
|
||||
# Verify
|
||||
curl -s "$URL/api/tags" -H "Authorization: Bearer $TOKEN" | head
|
||||
```
|
||||
|
||||
`co` enables CORS so a browser caller can hit the endpoint. Drop `co` for backend-only callers. For an OpenAI-compatible vLLM/llama.cpp endpoint, callers use base URL `$URL/v1` with `Authorization: Bearer $TOKEN` — but note Pinggy strips/replaces nothing in the body, so the model server itself sees Pinggy's token; the local server should be configured to ignore auth (it's already on `127.0.0.1`) and let Pinggy do the gating.
|
||||
|
||||
### Recipe 4 — Share a dev server with a one-shot password
|
||||
|
||||
The fastest "let a teammate poke at my running app" pattern. Random password, prints once, dies when you Ctrl-C.
|
||||
|
||||
```bash
|
||||
PASS=$(openssl rand -base64 12 | tr -d '+/=' | head -c 12)
|
||||
echo "Dev server password: $PASS"
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
|
||||
-o ServerAliveInterval=30 \
|
||||
-R0:localhost:3000 "b:dev:$PASS+co+x:https+free@a.pinggy.io"
|
||||
# URL prints to the terminal. Share URL + password. Ctrl-C to tear down.
|
||||
```
|
||||
|
||||
`b:dev:$PASS` gates the URL with HTTP Basic auth. `x:https` forces TLS. `co` adds CORS for SPA frontends.
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# End-to-end: spin up a trivial origin, tunnel it, hit it, tear down
|
||||
python3 -m http.server 18000 --bind 127.0.0.1 >/tmp/origin.log 2>&1 &
|
||||
ORIGIN_PID=$!
|
||||
|
||||
nohup ssh -p 443 \
|
||||
-o StrictHostKeyChecking=no \
|
||||
-o UserKnownHostsFile=/dev/null \
|
||||
-R0:localhost:18000 free@a.pinggy.io >/tmp/pinggy-verify.log 2>&1 &
|
||||
SSH_PID=$!
|
||||
|
||||
sleep 5
|
||||
URL=$(grep -oE 'https://[a-z0-9-]+\.[a-z]+\.pinggy\.link' /tmp/pinggy-verify.log | head -1)
|
||||
echo "URL: $URL"
|
||||
curl -sI "$URL/" | head -1
|
||||
|
||||
kill "$SSH_PID" "$ORIGIN_PID"
|
||||
```
|
||||
|
||||
Expected: a `pinggy.link` URL and `HTTP/2 200` on the curl head.
|
||||
@@ -0,0 +1,112 @@
|
||||
---
|
||||
name: watchers
|
||||
description: Poll RSS, JSON APIs, and GitHub with watermark dedup.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [cron, polling, rss, github, http, automation, monitoring]
|
||||
category: devops
|
||||
requires_toolsets: [terminal]
|
||||
related_skills: []
|
||||
---
|
||||
|
||||
# Watchers
|
||||
|
||||
Poll external sources on an interval and react only to new items. Three ready-made scripts plus a shared watermark helper; wire them into a cron job (or run them ad-hoc from the terminal).
|
||||
|
||||
## When to Use
|
||||
|
||||
- User wants to watch an RSS/Atom feed and be notified of new entries
|
||||
- User wants to watch a GitHub repo's issues / pulls / releases / commits
|
||||
- User wants to poll an arbitrary JSON endpoint and get notified on new items
|
||||
- User asks for "a watcher for X" or "notify me when X changes"
|
||||
|
||||
## Mental model
|
||||
|
||||
A watcher is just a script that:
|
||||
|
||||
1. Fetches data from the external source
|
||||
2. Compares against a watermark file of previously-seen IDs
|
||||
3. Writes the new watermark back
|
||||
4. Prints new items to stdout (or nothing on no-change)
|
||||
|
||||
The scripts below handle all three. The agent runs them via the terminal tool — from a cron job, a webhook, or an interactive chat — and reports what's new.
|
||||
|
||||
## Ready-made scripts
|
||||
|
||||
All three live in `$HERMES_HOME/skills/devops/watchers/scripts/` once the skill is installed. Each reads `WATCHER_STATE_DIR` (defaults to `$HERMES_HOME/watcher-state/`) for its state file, keyed by the `--name` argument.
|
||||
|
||||
| Script | What it watches | Dedup key |
|
||||
|---|---|---|
|
||||
| `watch_rss.py` | RSS 2.0 or Atom feed URL | `<guid>` / `<id>` |
|
||||
| `watch_http_json.py` | Any JSON endpoint returning a list of objects | Configurable id field |
|
||||
| `watch_github.py` | GitHub issues / pulls / releases / commits for a repo | `id` / `sha` |
|
||||
|
||||
All three:
|
||||
|
||||
- First run records a baseline — never replays existing feed
|
||||
- Watermark is a bounded ID set (max 500) to cap memory
|
||||
- Output format: `## <title>\n<url>\n\n<optional body>` per item
|
||||
- Empty stdout on no-new — the caller treats that as silent
|
||||
- Non-zero exit on fetch errors
|
||||
|
||||
## Usage
|
||||
|
||||
Run a watcher directly from the terminal tool:
|
||||
|
||||
```bash
|
||||
python $HERMES_HOME/skills/devops/watchers/scripts/watch_rss.py \
|
||||
--name hn --url https://news.ycombinator.com/rss --max 5
|
||||
```
|
||||
|
||||
Watch a GitHub repo (set `GITHUB_TOKEN` in `~/.hermes/.env` to avoid the 60 req/hr anonymous rate limit):
|
||||
|
||||
```bash
|
||||
python $HERMES_HOME/skills/devops/watchers/scripts/watch_github.py \
|
||||
--name hermes-issues --repo NousResearch/hermes-agent --scope issues
|
||||
```
|
||||
|
||||
Poll an arbitrary JSON API:
|
||||
|
||||
```bash
|
||||
python $HERMES_HOME/skills/devops/watchers/scripts/watch_http_json.py \
|
||||
--name api --url https://api.example.com/events \
|
||||
--id-field event_id --items-path data.events
|
||||
```
|
||||
|
||||
## Wiring into cron
|
||||
|
||||
Ask the agent to schedule a cron job with a prompt like:
|
||||
|
||||
> Every 15 minutes, run `watch_rss.py --name hn --url https://news.ycombinator.com/rss`. If it prints anything, summarize the headlines and deliver them. If it prints nothing, stay silent.
|
||||
|
||||
The agent invokes the script via the terminal tool inside the cron job's agent loop; no changes to cron's built-in `--script` flag are needed.
|
||||
|
||||
## State files
|
||||
|
||||
Every watcher writes `$HERMES_HOME/watcher-state/<name>.json`. Inspect:
|
||||
|
||||
```bash
|
||||
cat $HERMES_HOME/watcher-state/hn.json
|
||||
```
|
||||
|
||||
Force a replay (next run treated as first poll):
|
||||
|
||||
```bash
|
||||
rm $HERMES_HOME/watcher-state/hn.json
|
||||
```
|
||||
|
||||
## Writing your own
|
||||
|
||||
All three scripts use the same template: load watermark, fetch, diff, save, emit. `scripts/_watermark.py` is the shared helper; import it to get atomic writes + bounded ID set + first-run baseline for free. See any of the three reference scripts for how little boilerplate it takes.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **Printing a "no new items" header every tick.** Callers rely on empty stdout = silent. If you print anything on an empty delta, you spam the channel. The shipped scripts handle this; custom scripts must too.
|
||||
2. **Expecting the first run to emit items.** It won't — first run records a baseline. If you need an initial digest, delete the state file after the first run or add a `--prime-with-latest N` flag in your own script.
|
||||
3. **Unbounded watermark growth.** The shared helper caps at 500 IDs. Raise it for high-churn feeds; lower it on constrained filesystems.
|
||||
4. **Putting the state dir where the agent's sandbox can't write.** `$HERMES_HOME/watcher-state/` is always writable. Docker/Modal backends may not see arbitrary host paths.
|
||||
|
||||
+148
@@ -0,0 +1,148 @@
|
||||
"""Shared watermark helper used by the three watcher scripts.
|
||||
|
||||
A watermark is just a JSON file that records the IDs we've seen on previous
|
||||
runs, so the next run only emits items we haven't seen before.
|
||||
|
||||
Contract:
|
||||
- First run: record all IDs from the fetched batch, emit nothing.
|
||||
- Subsequent runs: emit items whose ID isn't in the stored set.
|
||||
- Bounded: keep at most `max_seen` IDs (default 500).
|
||||
- Atomic: write to a .tmp file and rename, so a crashed script can't
|
||||
leave a half-written state file that permanently breaks dedup.
|
||||
|
||||
Import and use from any custom watcher script:
|
||||
|
||||
from _watermark import Watermark
|
||||
|
||||
wm = Watermark.load("my-feed-name")
|
||||
new_items = wm.filter_new(fetched_items, id_key="id")
|
||||
wm.save()
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Iterable, List, Optional
|
||||
|
||||
|
||||
def _state_dir() -> Path:
|
||||
"""Where watermark files live — respects WATCHER_STATE_DIR override."""
|
||||
override = os.environ.get("WATCHER_STATE_DIR")
|
||||
if override:
|
||||
return Path(override)
|
||||
# Default: $HERMES_HOME/watcher-state/, falling back to ~/.hermes/watcher-state/.
|
||||
hermes_home = os.environ.get("HERMES_HOME") or str(Path.home() / ".hermes")
|
||||
return Path(hermes_home) / "watcher-state"
|
||||
|
||||
|
||||
class Watermark:
|
||||
"""Per-watcher state. Persisted to <state_dir>/<name>.json."""
|
||||
|
||||
def __init__(self, name: str, *, max_seen: int = 500) -> None:
|
||||
if not name or not name.replace("-", "").replace("_", "").isalnum():
|
||||
raise ValueError(
|
||||
f"watermark name must be alphanumeric + '-'/'_' (got {name!r})"
|
||||
)
|
||||
self.name = name
|
||||
self.max_seen = max_seen
|
||||
self._path = _state_dir() / f"{name}.json"
|
||||
self._data: Dict[str, Any] = {"seen_ids": [], "first_run": True}
|
||||
|
||||
@classmethod
|
||||
def load(cls, name: str, *, max_seen: int = 500) -> "Watermark":
|
||||
wm = cls(name, max_seen=max_seen)
|
||||
if wm._path.exists():
|
||||
try:
|
||||
wm._data = json.loads(wm._path.read_text(encoding="utf-8"))
|
||||
wm._data.setdefault("seen_ids", [])
|
||||
wm._data["first_run"] = False
|
||||
except (OSError, json.JSONDecodeError):
|
||||
# Corrupt state file — treat as a first run but don't crash.
|
||||
wm._data = {"seen_ids": [], "first_run": True}
|
||||
return wm
|
||||
|
||||
@property
|
||||
def is_first_run(self) -> bool:
|
||||
return bool(self._data.get("first_run", True))
|
||||
|
||||
@property
|
||||
def seen(self) -> List[str]:
|
||||
return list(self._data.get("seen_ids", []))
|
||||
|
||||
def filter_new(
|
||||
self, items: Iterable[Dict[str, Any]], *, id_key: str = "id"
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Return items whose id isn't in the stored set.
|
||||
|
||||
Side effect: updates the in-memory seen set with every id in the
|
||||
batch (so save() persists the full new watermark). On first run,
|
||||
records every id but returns an empty list (baseline, no replay).
|
||||
"""
|
||||
existing = set(str(x) for x in self._data.get("seen_ids", []))
|
||||
was_first_run = self.is_first_run
|
||||
|
||||
new_items: List[Dict[str, Any]] = []
|
||||
batch_ids: List[str] = []
|
||||
for item in items:
|
||||
ident = item.get(id_key)
|
||||
if ident is None:
|
||||
continue
|
||||
ident_str = str(ident)
|
||||
batch_ids.append(ident_str)
|
||||
if ident_str in existing:
|
||||
continue
|
||||
if was_first_run:
|
||||
continue # record but don't emit
|
||||
new_items.append(item)
|
||||
|
||||
combined = list(existing) + [i for i in batch_ids if i not in existing]
|
||||
if len(combined) > self.max_seen:
|
||||
combined = combined[-self.max_seen:]
|
||||
self._data["seen_ids"] = combined
|
||||
self._data["first_run"] = False
|
||||
return new_items
|
||||
|
||||
def save(self) -> None:
|
||||
self._path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = self._path.with_suffix(".tmp")
|
||||
tmp.write_text(
|
||||
json.dumps(self._data, indent=2, sort_keys=True),
|
||||
encoding="utf-8",
|
||||
)
|
||||
os.replace(tmp, self._path)
|
||||
|
||||
|
||||
def format_items_as_markdown(
|
||||
items: List[Dict[str, Any]],
|
||||
*,
|
||||
title_key: str = "title",
|
||||
url_key: str = "url",
|
||||
body_key: Optional[str] = None,
|
||||
max_body_chars: int = 500,
|
||||
) -> str:
|
||||
"""Render a list of items as Markdown for cron delivery.
|
||||
|
||||
One heading per item + its URL + optional snippet of body. Output is
|
||||
empty string when items is empty — cron will then treat stdout as
|
||||
silent and skip delivery (existing behavior).
|
||||
"""
|
||||
if not items:
|
||||
return ""
|
||||
lines: List[str] = []
|
||||
for item in items:
|
||||
title = (item.get(title_key) or "(no title)").strip()
|
||||
url = (item.get(url_key) or "").strip()
|
||||
lines.append(f"## {title}")
|
||||
if url:
|
||||
lines.append(url)
|
||||
if body_key:
|
||||
body = (item.get(body_key) or "").strip()
|
||||
if body:
|
||||
if len(body) > max_body_chars:
|
||||
body = body[:max_body_chars].rstrip() + "…"
|
||||
lines.append("")
|
||||
lines.append(body)
|
||||
lines.append("")
|
||||
return "\n".join(lines).rstrip() + "\n"
|
||||
+168
@@ -0,0 +1,168 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Watch GitHub activity — issues, pulls, releases, or commits — with dedup.
|
||||
|
||||
Usage (via cron with --no-agent):
|
||||
|
||||
hermes cron create hermes-issues \\
|
||||
--schedule "*/5 * * * *" --no-agent \\
|
||||
--script "$HERMES_HOME/skills/devops/watchers/scripts/watch_github.py" \\
|
||||
--script-args "--name hermes-issues --repo NousResearch/hermes-agent --scope issues"
|
||||
|
||||
Set GITHUB_TOKEN (or GH_TOKEN) in ~/.hermes/.env to avoid the 60 req/hr
|
||||
anonymous rate limit.
|
||||
|
||||
Scopes: issues | pulls | releases | commits. Or pass --search QUERY to
|
||||
use the /search/issues endpoint instead of /repos/:owner/:repo/:scope.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _watermark import Watermark, format_items_as_markdown # type: ignore
|
||||
|
||||
|
||||
VALID_SCOPES = ("issues", "pulls", "releases", "commits")
|
||||
|
||||
|
||||
def _flatten_commit(item):
|
||||
"""Commit objects nest title/author/date under 'commit' — flatten for rendering."""
|
||||
commit = item.get("commit") or {}
|
||||
msg = (commit.get("message") or "").strip().splitlines()
|
||||
title = msg[0] if msg else ""
|
||||
body = "\n".join(msg[1:]).strip() if len(msg) > 1 else ""
|
||||
author = (item.get("author") or {}).get("login") or (commit.get("author") or {}).get("name", "")
|
||||
date = (commit.get("author") or {}).get("date", "")
|
||||
return {
|
||||
"id": item.get("sha", ""),
|
||||
"title": f"{title} ({author})" if author else title,
|
||||
"url": item.get("html_url"),
|
||||
"body": body,
|
||||
"created_at": date,
|
||||
}
|
||||
|
||||
|
||||
def _flatten_issue_or_release(item):
|
||||
return {
|
||||
"id": str(item.get("id", "")),
|
||||
"title": item.get("title") or item.get("name") or "",
|
||||
"url": item.get("html_url") or item.get("url"),
|
||||
"body": (item.get("body") or "").strip(),
|
||||
"state": item.get("state"),
|
||||
"author": (item.get("user") or {}).get("login")
|
||||
or (item.get("author") or {}).get("login"),
|
||||
"created_at": item.get("created_at"),
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description="Watch GitHub issues / pulls / releases / commits.")
|
||||
p.add_argument("--name", required=True, help="Watcher name (used for state file)")
|
||||
p.add_argument("--repo", default="",
|
||||
help="owner/name of the repo (one of --repo or --search is required)")
|
||||
p.add_argument("--scope", default="issues", choices=VALID_SCOPES,
|
||||
help="What to poll (default: issues)")
|
||||
p.add_argument("--search", default="",
|
||||
help="GitHub issues search query (alternative to --repo/--scope)")
|
||||
p.add_argument("--per-page", type=int, default=30,
|
||||
help="Results per page (default: 30, max: 100)")
|
||||
p.add_argument("--max", type=int, default=20,
|
||||
help="Max new items to emit per tick (default: 20)")
|
||||
p.add_argument("--with-body", action="store_true",
|
||||
help="Include issue/commit body as a snippet under each item")
|
||||
p.add_argument("--timeout", type=float, default=30.0,
|
||||
help="HTTP timeout in seconds (default: 30)")
|
||||
args = p.parse_args()
|
||||
|
||||
if not args.repo and not args.search:
|
||||
print("watch_github: one of --repo or --search is required", file=sys.stderr)
|
||||
return 2
|
||||
if args.repo and not re.fullmatch(r"[A-Za-z0-9._-]+/[A-Za-z0-9._-]+", args.repo):
|
||||
print(f"watch_github: --repo must be owner/name (got {args.repo!r})", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
# URL + flattening strategy.
|
||||
if args.search:
|
||||
url = (
|
||||
"https://api.github.com/search/issues"
|
||||
f"?q={urllib.parse.quote(args.search)}&per_page={args.per_page}"
|
||||
)
|
||||
flatten = _flatten_issue_or_release
|
||||
items_path = "items"
|
||||
elif args.scope == "commits":
|
||||
url = f"https://api.github.com/repos/{args.repo}/commits?per_page={args.per_page}"
|
||||
flatten = _flatten_commit
|
||||
items_path = ""
|
||||
else:
|
||||
url = (
|
||||
f"https://api.github.com/repos/{args.repo}/{args.scope}"
|
||||
f"?per_page={args.per_page}&state=all"
|
||||
)
|
||||
flatten = _flatten_issue_or_release
|
||||
items_path = ""
|
||||
|
||||
headers = {
|
||||
"Accept": "application/vnd.github+json",
|
||||
"User-Agent": "Hermes-Watcher/1.0",
|
||||
}
|
||||
token = os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN")
|
||||
if token:
|
||||
headers["Authorization"] = f"Bearer {token}"
|
||||
|
||||
req = urllib.request.Request(url)
|
||||
for k, v in headers.items():
|
||||
req.add_header(k, v)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=args.timeout) as resp:
|
||||
raw = resp.read()
|
||||
except urllib.error.HTTPError as e:
|
||||
print(f"watch_github: HTTP {e.code} from {url}", file=sys.stderr)
|
||||
return 2
|
||||
except (urllib.error.URLError, TimeoutError, OSError) as e:
|
||||
print(f"watch_github: network error: {e}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
try:
|
||||
data = json.loads(raw.decode("utf-8"))
|
||||
except (UnicodeDecodeError, json.JSONDecodeError) as e:
|
||||
print(f"watch_github: response is not valid JSON: {e}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
# Drill into items_path if needed (search endpoint returns {"items":[...]}).
|
||||
if items_path:
|
||||
data = data.get(items_path) if isinstance(data, dict) else None
|
||||
if not isinstance(data, list):
|
||||
print(f"watch_github: expected a list of items; got {type(data).__name__}",
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
|
||||
items = [flatten(i) for i in data if isinstance(i, dict)]
|
||||
# Drop any items that flattened without an ID (defensive).
|
||||
items = [i for i in items if i.get("id")]
|
||||
|
||||
wm = Watermark.load(args.name)
|
||||
new_items = wm.filter_new(items, id_key="id")
|
||||
wm.save()
|
||||
|
||||
if args.max > 0:
|
||||
new_items = new_items[: args.max]
|
||||
|
||||
body_key = "body" if args.with_body else None
|
||||
output = format_items_as_markdown(new_items, body_key=body_key)
|
||||
if output:
|
||||
sys.stdout.write(output)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+131
@@ -0,0 +1,131 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Watch any JSON endpoint that returns a list of objects; dedup by ID field.
|
||||
|
||||
Usage (via cron with --no-agent):
|
||||
|
||||
hermes cron create api-events \\
|
||||
--schedule "*/1 * * * *" --no-agent \\
|
||||
--script "$HERMES_HOME/skills/devops/watchers/scripts/watch_http_json.py" \\
|
||||
--script-args "--name api --url https://api.example.com/events \\
|
||||
--id-field event_id --items-path data.events"
|
||||
|
||||
The response can be:
|
||||
- a top-level JSON list (default), or
|
||||
- a JSON object with a dotted ``--items-path`` pointing to the list.
|
||||
|
||||
Each item is deduped by ``--id-field`` (default "id").
|
||||
|
||||
Optional ``--header KEY:VALUE`` flags pass HTTP headers (repeatable).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _watermark import Watermark, format_items_as_markdown # type: ignore
|
||||
|
||||
|
||||
def _dig(obj, path: str):
|
||||
"""Dotted-path lookup: _dig({'a':{'b':[1,2]}}, 'a.b') → [1,2]."""
|
||||
if not path:
|
||||
return obj
|
||||
cur = obj
|
||||
for part in path.split("."):
|
||||
if isinstance(cur, dict) and part in cur:
|
||||
cur = cur[part]
|
||||
else:
|
||||
return None
|
||||
return cur
|
||||
|
||||
|
||||
def _parse_header(s: str):
|
||||
if ":" not in s:
|
||||
raise argparse.ArgumentTypeError(
|
||||
f"--header expects 'KEY: VALUE' (got {s!r})"
|
||||
)
|
||||
k, v = s.split(":", 1)
|
||||
return (k.strip(), v.strip())
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description="Poll a JSON endpoint.")
|
||||
p.add_argument("--name", required=True, help="Watcher name (used for state file)")
|
||||
p.add_argument("--url", required=True, help="JSON endpoint URL")
|
||||
p.add_argument("--id-field", default="id",
|
||||
help="Field used to dedup items (default: 'id')")
|
||||
p.add_argument("--items-path", default="",
|
||||
help="Dotted path to the list inside the JSON response (e.g. 'data.events')")
|
||||
p.add_argument("--title-field", default="title",
|
||||
help="Field used as the item title in the rendered output (default: 'title')")
|
||||
p.add_argument("--url-field", default="url",
|
||||
help="Field used as the item URL in the rendered output (default: 'url')")
|
||||
p.add_argument("--body-field", default="",
|
||||
help="Optional body field to include as a snippet under each item")
|
||||
p.add_argument("--max", type=int, default=20,
|
||||
help="Max new items to emit per tick (default: 20)")
|
||||
p.add_argument("--header", action="append", type=_parse_header, default=[],
|
||||
metavar="KEY: VALUE",
|
||||
help="HTTP header (repeatable)")
|
||||
p.add_argument("--timeout", type=float, default=20.0,
|
||||
help="HTTP timeout in seconds (default: 20)")
|
||||
args = p.parse_args()
|
||||
|
||||
req = urllib.request.Request(args.url, headers={"User-Agent": "Hermes-Watcher/1.0"})
|
||||
for k, v in args.header:
|
||||
req.add_header(k, v)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=args.timeout) as resp:
|
||||
raw = resp.read()
|
||||
except urllib.error.HTTPError as e:
|
||||
print(f"watch_http_json: HTTP {e.code} from {args.url}", file=sys.stderr)
|
||||
return 2
|
||||
except (urllib.error.URLError, TimeoutError, OSError) as e:
|
||||
print(f"watch_http_json: network error: {e}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
try:
|
||||
data = json.loads(raw.decode("utf-8"))
|
||||
except (UnicodeDecodeError, json.JSONDecodeError) as e:
|
||||
print(f"watch_http_json: response is not valid JSON: {e}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
items = _dig(data, args.items_path) if args.items_path else data
|
||||
if not isinstance(items, list):
|
||||
print(
|
||||
f"watch_http_json: items_path={args.items_path!r} did not resolve to a list "
|
||||
f"(got {type(items).__name__})",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
|
||||
# Keep only dicts — skip any bare strings / numbers so filter_new doesn't crash.
|
||||
items = [i for i in items if isinstance(i, dict)]
|
||||
|
||||
wm = Watermark.load(args.name)
|
||||
new_items = wm.filter_new(items, id_key=args.id_field)
|
||||
wm.save()
|
||||
|
||||
if args.max > 0:
|
||||
new_items = new_items[: args.max]
|
||||
|
||||
body_key = args.body_field or None
|
||||
output = format_items_as_markdown(
|
||||
new_items,
|
||||
title_key=args.title_field,
|
||||
url_key=args.url_field,
|
||||
body_key=body_key,
|
||||
)
|
||||
if output:
|
||||
sys.stdout.write(output)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+121
@@ -0,0 +1,121 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Watch an RSS 2.0 or Atom feed; print new items to stdout, silent on empty.
|
||||
|
||||
Usage (via cron with --no-agent):
|
||||
|
||||
hermes cron create my-feed \\
|
||||
--schedule "*/15 * * * *" --no-agent \\
|
||||
--script "$HERMES_HOME/skills/devops/watchers/scripts/watch_rss.py" \\
|
||||
--script-args "--name hn --url https://news.ycombinator.com/rss"
|
||||
|
||||
First run records a baseline (emits nothing). Subsequent runs emit only
|
||||
items whose <guid> / <id> isn't in the watermark.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from xml.etree import ElementTree as ET
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _watermark import Watermark, format_items_as_markdown # type: ignore
|
||||
|
||||
|
||||
def _strip_ns(tag: str) -> str:
|
||||
return tag.split("}", 1)[1] if "}" in tag else tag
|
||||
|
||||
|
||||
def _parse_feed(xml_bytes: bytes):
|
||||
"""Return a list of {id, title, url, summary} dicts.
|
||||
|
||||
Handles both RSS 2.0 ``<item>`` and Atom ``<entry>``.
|
||||
"""
|
||||
try:
|
||||
root = ET.fromstring(xml_bytes)
|
||||
except ET.ParseError as e:
|
||||
print(f"watch_rss: invalid XML: {e}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
entries = []
|
||||
for item in root.iter():
|
||||
tag = _strip_ns(item.tag)
|
||||
if tag not in {"item", "entry"}:
|
||||
continue
|
||||
# ElementTree Elements without children are *falsy* — use `is not None`.
|
||||
children = {_strip_ns(c.tag): c for c in item}
|
||||
|
||||
guid_el = children.get("guid")
|
||||
if guid_el is None:
|
||||
guid_el = children.get("id")
|
||||
link_el = children.get("link")
|
||||
if link_el is not None:
|
||||
href = link_el.attrib.get("href") or (link_el.text or "").strip()
|
||||
else:
|
||||
href = ""
|
||||
guid = (guid_el.text or "").strip() if guid_el is not None else ""
|
||||
guid = guid or href
|
||||
if not guid:
|
||||
continue
|
||||
|
||||
title_el = children.get("title")
|
||||
title = (title_el.text or "").strip() if title_el is not None else ""
|
||||
|
||||
summ_el = children.get("description")
|
||||
if summ_el is None:
|
||||
summ_el = children.get("summary")
|
||||
summary = (summ_el.text or "").strip() if summ_el is not None else ""
|
||||
|
||||
entries.append(
|
||||
{"id": guid, "title": title, "url": href, "summary": summary}
|
||||
)
|
||||
return entries
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description="Watch an RSS/Atom feed.")
|
||||
p.add_argument("--name", required=True, help="Watcher name (used for state file)")
|
||||
p.add_argument("--url", required=True, help="Feed URL")
|
||||
p.add_argument("--max", type=int, default=10,
|
||||
help="Max new items to emit per tick (default: 10)")
|
||||
p.add_argument("--with-summary", action="store_true",
|
||||
help="Include <description>/<summary> snippet under each item")
|
||||
p.add_argument("--timeout", type=float, default=20.0,
|
||||
help="HTTP timeout in seconds (default: 20)")
|
||||
args = p.parse_args()
|
||||
|
||||
try:
|
||||
req = urllib.request.Request(args.url, headers={"User-Agent": "Hermes-Watcher/1.0"})
|
||||
with urllib.request.urlopen(req, timeout=args.timeout) as resp:
|
||||
xml_bytes = resp.read()
|
||||
except urllib.error.HTTPError as e:
|
||||
print(f"watch_rss: HTTP {e.code} from {args.url}", file=sys.stderr)
|
||||
return 2
|
||||
except (urllib.error.URLError, TimeoutError, OSError) as e:
|
||||
print(f"watch_rss: network error: {e}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
entries = _parse_feed(xml_bytes)
|
||||
|
||||
wm = Watermark.load(args.name)
|
||||
new_items = wm.filter_new(entries, id_key="id")
|
||||
wm.save()
|
||||
|
||||
# Cap emitted items (watermark still records all seen IDs so we don't
|
||||
# re-emit them next tick).
|
||||
if args.max > 0:
|
||||
new_items = new_items[: args.max]
|
||||
|
||||
body_key = "summary" if args.with_summary else None
|
||||
output = format_items_as_markdown(new_items, body_key=body_key)
|
||||
if output:
|
||||
sys.stdout.write(output)
|
||||
# Empty stdout on no-new — cron treats that as silent.
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,3 @@
|
||||
# Dogfood — Advanced QA & Testing Skills
|
||||
|
||||
Specialized QA workflows that go beyond basic bug-finding. These skills use structured methodologies to surface UX friction, accessibility issues, and product-level problems that standard testing misses.
|
||||
@@ -0,0 +1,191 @@
|
||||
---
|
||||
name: adversarial-ux-test
|
||||
description: Roleplay the most difficult, tech-resistant user for your product. Browse the app as that persona, find every UX pain point, then filter complaints through a pragmatism layer to separate real problems from noise. Creates actionable tickets from genuine issues only.
|
||||
version: 1.0.0
|
||||
author: Omni @ Comelse
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [qa, ux, testing, adversarial, dogfood, personas, user-testing]
|
||||
related_skills: [dogfood]
|
||||
---
|
||||
|
||||
# Adversarial UX Test
|
||||
|
||||
Roleplay the worst-case user for your product — the person who hates technology, doesn't want your software, and will find every reason to complain. Then filter their feedback through a pragmatism layer to separate real UX problems from "I hate computers" noise.
|
||||
|
||||
Think of it as an automated "mom test" — but angry.
|
||||
|
||||
## Why This Works
|
||||
|
||||
Most QA finds bugs. This finds **friction**. A technically correct app can still be unusable for real humans. The adversarial persona catches:
|
||||
- Confusing terminology that makes sense to developers but not users
|
||||
- Too many steps to accomplish basic tasks
|
||||
- Missing onboarding or "aha moments"
|
||||
- Accessibility issues (font size, contrast, click targets)
|
||||
- Cold-start problems (empty states, no demo content)
|
||||
- Paywall/signup friction that kills conversion
|
||||
|
||||
The **pragmatism filter** (Phase 3) is what makes this useful instead of just entertaining. Without it, you'd add a "print this page" button to every screen because Grandpa can't figure out PDFs.
|
||||
|
||||
## How to Use
|
||||
|
||||
Tell the agent:
|
||||
```
|
||||
"Run an adversarial UX test on [URL]"
|
||||
"Be a grumpy [persona type] and test [app name]"
|
||||
"Do an asshole user test on my staging site"
|
||||
```
|
||||
|
||||
You can provide a persona or let the agent generate one based on your product's target audience.
|
||||
|
||||
## Step 1: Define the Persona
|
||||
|
||||
If no persona is provided, generate one by answering:
|
||||
|
||||
1. **Who is the HARDEST user for this product?** (age 50+, non-technical role, decades of experience doing it "the old way")
|
||||
2. **What is their tech comfort level?** (the lower the better — WhatsApp-only, paper notebooks, wife set up their email)
|
||||
3. **What is the ONE thing they need to accomplish?** (their core job, not your feature list)
|
||||
4. **What would make them give up?** (too many clicks, jargon, slow, confusing)
|
||||
5. **How do they talk when frustrated?** (blunt, sweary, dismissive, sighing)
|
||||
|
||||
### Good Persona Example
|
||||
> **"Big Mick" McAllister** — 58-year-old S&C coach. Uses WhatsApp and that's it. His "spreadsheet" is a paper notebook. "If I can't figure it out in 10 seconds I'm going back to my notebook." Needs to log session results for 25 players. Hates small text, jargon, and passwords.
|
||||
|
||||
### Bad Persona Example
|
||||
> "A user who doesn't like the app" — too vague, no constraints, no voice.
|
||||
|
||||
The persona must be **specific enough to stay in character** for 20 minutes of testing.
|
||||
|
||||
## Step 2: Become the Asshole (Browse as the Persona)
|
||||
|
||||
1. Read any available project docs for app context and URLs
|
||||
2. **Fully inhabit the persona** — their frustrations, limitations, goals
|
||||
3. Navigate to the app using browser tools
|
||||
4. **Attempt the persona's ACTUAL TASKS** (not a feature tour):
|
||||
- Can they do what they came to do?
|
||||
- How many clicks/screens to accomplish it?
|
||||
- What confuses them?
|
||||
- What makes them angry?
|
||||
- Where do they get lost?
|
||||
- What would make them give up and go back to their old way?
|
||||
|
||||
5. Test these friction categories:
|
||||
- **First impression** — would they even bother past the landing page?
|
||||
- **Core workflow** — the ONE thing they need to do most often
|
||||
- **Error recovery** — what happens when they do something wrong?
|
||||
- **Readability** — text size, contrast, information density
|
||||
- **Speed** — does it feel faster than their current method?
|
||||
- **Terminology** — any jargon they wouldn't understand?
|
||||
- **Navigation** — can they find their way back? do they know where they are?
|
||||
|
||||
6. Take screenshots of every pain point
|
||||
7. Check browser console for JS errors on every page
|
||||
|
||||
## Step 3: The Rant (Write Feedback in Character)
|
||||
|
||||
Write the feedback AS THE PERSONA — in their voice, with their frustrations. This is not a bug report. This is a real human venting.
|
||||
|
||||
```
|
||||
[PERSONA NAME]'s Review of [PRODUCT]
|
||||
|
||||
Overall: [Would they keep using it? Yes/No/Maybe with conditions]
|
||||
|
||||
THE GOOD (grudging admission):
|
||||
- [things even they have to admit work]
|
||||
|
||||
THE BAD (legitimate UX issues):
|
||||
- [real problems that would stop them from using the product]
|
||||
|
||||
THE UGLY (showstoppers):
|
||||
- [things that would make them uninstall/cancel immediately]
|
||||
|
||||
SPECIFIC COMPLAINTS:
|
||||
1. [Page/feature]: "[quote in persona voice]" — [what happened, expected]
|
||||
2. ...
|
||||
|
||||
VERDICT: "[one-line persona quote summarizing their experience]"
|
||||
```
|
||||
|
||||
## Step 4: The Pragmatism Filter (Critical — Do Not Skip)
|
||||
|
||||
Step OUT of the persona. Evaluate each complaint as a product person:
|
||||
|
||||
- **RED: REAL UX BUG** — Any user would have this problem, not just grumpy ones. Fix it.
|
||||
- **YELLOW: VALID BUT LOW PRIORITY** — Real issue but only for extreme users. Note it.
|
||||
- **WHITE: PERSONA NOISE** — "I hate computers" talking, not a product problem. Skip it.
|
||||
- **GREEN: FEATURE REQUEST** — Good idea hidden in the complaint. Consider it.
|
||||
|
||||
### Filter Criteria
|
||||
1. Would a 35-year-old competent-but-busy user have the same complaint? → RED
|
||||
2. Is this a genuine accessibility issue (font size, contrast, click targets)? → RED
|
||||
3. Is this "I want it to work like paper" resistance to digital? → WHITE
|
||||
4. Is this a real workflow inefficiency the persona stumbled on? → YELLOW or RED
|
||||
5. Would fixing this add complexity for the 80% who are fine? → WHITE
|
||||
6. Does the complaint reveal a missing onboarding moment? → GREEN
|
||||
|
||||
**This filter is MANDATORY.** Never ship raw persona complaints as tickets.
|
||||
|
||||
## Step 5: Create Tickets
|
||||
|
||||
For **RED** and **GREEN** items only:
|
||||
- Clear, actionable title
|
||||
- Include the persona's verbatim quote (entertaining + memorable)
|
||||
- The real UX issue underneath (objective)
|
||||
- A suggested fix (actionable)
|
||||
- Tag/label: "ux-review"
|
||||
|
||||
For **YELLOW** items: one catch-all ticket with all notes.
|
||||
|
||||
**WHITE** items appear in the report only. No tickets.
|
||||
|
||||
**Max 10 tickets per session** — focus on the worst issues.
|
||||
|
||||
## Step 6: Report
|
||||
|
||||
Deliver:
|
||||
1. The persona rant (Step 3) — entertaining and visceral
|
||||
2. The filtered assessment (Step 4) — pragmatic and actionable
|
||||
3. Tickets created (Step 5) — with links
|
||||
4. Screenshots of key issues
|
||||
|
||||
## Tips
|
||||
|
||||
- **One persona per session.** Don't mix perspectives.
|
||||
- **Stay in character during Steps 2-3.** Break character only at Step 4.
|
||||
- **Test the CORE WORKFLOW first.** Don't get distracted by settings pages.
|
||||
- **Empty states are gold.** New user experience reveals the most friction.
|
||||
- **The best findings are RED items the persona found accidentally** while trying to do something else.
|
||||
- **If the persona has zero complaints, your persona is too tech-savvy.** Make them older, less patient, more set in their ways.
|
||||
- **Run this before demos, launches, or after shipping a batch of features.**
|
||||
- **Register as a NEW user when possible.** Don't use pre-seeded admin accounts — the cold start experience is where most friction lives.
|
||||
- **Zero WHITE items is a signal, not a failure.** If the pragmatism filter finds no noise, your product has real UX problems, not just a grumpy persona.
|
||||
- **Check known issues in project docs AFTER the test.** If the persona found a bug that's already in the known issues list, that's actually the most damning finding — it means the team knew about it but never felt the user's pain.
|
||||
- **Subscription/paywall testing is critical.** Test with expired accounts, not just active ones. The "what happens when you can't pay" experience reveals whether the product respects users or holds their data hostage.
|
||||
- **Count the clicks to accomplish the persona's ONE task.** If it's more than 5, that's almost always a RED finding regardless of persona tech level.
|
||||
|
||||
## Example Personas by Industry
|
||||
|
||||
These are starting points — customize for your specific product:
|
||||
|
||||
| Product Type | Persona | Age | Key Trait |
|
||||
|-------------|---------|-----|-----------|
|
||||
| CRM | Retirement home director | 68 | Filing cabinet is the current CRM |
|
||||
| Photography SaaS | Rural wedding photographer | 62 | Books clients by phone, invoices on paper |
|
||||
| AI/ML Tool | Department store buyer | 55 | Burned by 3 failed tech startups |
|
||||
| Fitness App | Old-school gym coach | 58 | Paper notebook, thick fingers, bad eyes |
|
||||
| Accounting | Family bakery owner | 64 | Shoebox of receipts, hates subscriptions |
|
||||
| E-commerce | Market stall vendor | 60 | Cash only, smartphone is for calls |
|
||||
| Healthcare | Senior GP | 63 | Dictates notes, nurse handles the computer |
|
||||
| Education | Veteran teacher | 57 | Chalk and talk, worksheets in ring binders |
|
||||
|
||||
## Rules
|
||||
|
||||
- Stay in character during Steps 2-3
|
||||
- Be genuinely mean but fair — find real problems, not manufactured ones
|
||||
- The pragmatism filter (Step 4) is **MANDATORY**
|
||||
- Screenshots required for every complaint
|
||||
- Max 10 tickets per session
|
||||
- Test on staging/deployed app, not local dev
|
||||
- One persona, one session, one report
|
||||
@@ -0,0 +1,126 @@
|
||||
---
|
||||
name: agentmail
|
||||
description: Give the agent its own dedicated email inbox via AgentMail. Send, receive, and manage email autonomously using agent-owned email addresses (e.g. hermes-agent@agentmail.to).
|
||||
version: 1.0.0
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [email, communication, agentmail, mcp]
|
||||
category: email
|
||||
---
|
||||
|
||||
# AgentMail — Agent-Owned Email Inboxes
|
||||
|
||||
## Requirements
|
||||
|
||||
- **AgentMail API key** (required) — sign up at https://console.agentmail.to (free tier: 3 inboxes, 3,000 emails/month; paid plans from $20/mo)
|
||||
- Node.js 18+ (for the MCP server)
|
||||
|
||||
## When to Use
|
||||
Use this skill when you need to:
|
||||
- Give the agent its own dedicated email address
|
||||
- Send emails autonomously on behalf of the agent
|
||||
- Receive and read incoming emails
|
||||
- Manage email threads and conversations
|
||||
- Sign up for services or authenticate via email
|
||||
- Communicate with other agents or humans via email
|
||||
|
||||
This is NOT for reading the user's personal email (use himalaya or Gmail for that).
|
||||
AgentMail gives the agent its own identity and inbox.
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Get an API Key
|
||||
- Go to https://console.agentmail.to
|
||||
- Create an account and generate an API key (starts with `am_`)
|
||||
|
||||
### 2. Configure MCP Server
|
||||
Add to `~/.hermes/config.yaml` (paste your actual key — MCP env vars are not expanded from .env):
|
||||
```yaml
|
||||
mcp_servers:
|
||||
agentmail:
|
||||
command: "npx"
|
||||
args: ["-y", "agentmail-mcp"]
|
||||
env:
|
||||
AGENTMAIL_API_KEY: "am_your_key_here"
|
||||
```
|
||||
|
||||
### 3. Restart Hermes
|
||||
```bash
|
||||
hermes
|
||||
```
|
||||
All 11 AgentMail tools are now available automatically.
|
||||
|
||||
## Available Tools (via MCP)
|
||||
|
||||
| Tool | Description |
|
||||
|------|-------------|
|
||||
| `list_inboxes` | List all agent inboxes |
|
||||
| `get_inbox` | Get details of a specific inbox |
|
||||
| `create_inbox` | Create a new inbox (gets a real email address) |
|
||||
| `delete_inbox` | Delete an inbox |
|
||||
| `list_threads` | List email threads in an inbox |
|
||||
| `get_thread` | Get a specific email thread |
|
||||
| `send_message` | Send a new email |
|
||||
| `reply_to_message` | Reply to an existing email |
|
||||
| `forward_message` | Forward an email |
|
||||
| `update_message` | Update message labels/status |
|
||||
| `get_attachment` | Download an email attachment |
|
||||
|
||||
## Procedure
|
||||
|
||||
### Create an inbox and send an email
|
||||
1. Create a dedicated inbox:
|
||||
- Use `create_inbox` with a username (e.g. `hermes-agent`)
|
||||
- The agent gets address: `hermes-agent@agentmail.to`
|
||||
2. Send an email:
|
||||
- Use `send_message` with `inbox_id`, `to`, `subject`, `text`
|
||||
3. Check for replies:
|
||||
- Use `list_threads` to see incoming conversations
|
||||
- Use `get_thread` to read a specific thread
|
||||
|
||||
### Check incoming email
|
||||
1. Use `list_inboxes` to find your inbox ID
|
||||
2. Use `list_threads` with the inbox ID to see conversations
|
||||
3. Use `get_thread` to read a thread and its messages
|
||||
|
||||
### Reply to an email
|
||||
1. Get the thread with `get_thread`
|
||||
2. Use `reply_to_message` with the message ID and your reply text
|
||||
|
||||
## Example Workflows
|
||||
|
||||
**Sign up for a service:**
|
||||
```
|
||||
1. create_inbox (username: "signup-bot")
|
||||
2. Use the inbox address to register on the service
|
||||
3. list_threads to check for verification email
|
||||
4. get_thread to read the verification code
|
||||
```
|
||||
|
||||
**Agent-to-human outreach:**
|
||||
```
|
||||
1. create_inbox (username: "hermes-outreach")
|
||||
2. send_message (to: user@example.com, subject: "Hello", text: "...")
|
||||
3. list_threads to check for replies
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
- Free tier limited to 3 inboxes and 3,000 emails/month
|
||||
- Emails come from `@agentmail.to` domain on free tier (custom domains on paid plans)
|
||||
- Node.js (18+) is required for the MCP server (`npx -y agentmail-mcp`)
|
||||
- The `mcp` Python package must be installed: `pip install mcp`
|
||||
- Real-time inbound email (webhooks) requires a public server — use `list_threads` polling via cronjob instead for personal use
|
||||
|
||||
## Verification
|
||||
After setup, test with:
|
||||
```
|
||||
hermes --toolsets mcp -q "Create an AgentMail inbox called test-agent and tell me its email address"
|
||||
```
|
||||
You should see the new inbox address returned.
|
||||
|
||||
## References
|
||||
- AgentMail docs: https://docs.agentmail.to/
|
||||
- AgentMail console: https://console.agentmail.to
|
||||
- AgentMail MCP repo: https://github.com/agentmail-to/agentmail-mcp
|
||||
- Pricing: https://www.agentmail.to/pricing
|
||||
@@ -0,0 +1,433 @@
|
||||
---
|
||||
name: 3-statement-model
|
||||
description: Build fully-integrated 3-statement models (IS, BS, CF) in Excel with working capital schedules, D&A roll-forwards, debt schedule, and the plugs that make cash and retained earnings tie. Pairs with excel-author.
|
||||
version: 1.0.0
|
||||
author: Anthropic (adapted by Nous Research)
|
||||
license: Apache-2.0
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [finance, three-statement, income-statement, balance-sheet, cash-flow, excel, openpyxl, modeling]
|
||||
related_skills: [excel-author, pptx-author, dcf-model, lbo-model]
|
||||
---
|
||||
|
||||
## Environment
|
||||
|
||||
This skill assumes **headless openpyxl** — you are producing an .xlsx file on disk.
|
||||
Follow the `excel-author` skill's conventions for cell coloring, formulas, named ranges, and sensitivity tables.
|
||||
Recalculate before delivery: `python /path/to/excel-author/scripts/recalc.py ./out/model.xlsx`.
|
||||
|
||||
# 3-Statement Financial Model Template Completion
|
||||
|
||||
Complete and populate integrated financial model templates with proper linkages between Income Statement, Balance Sheet, and Cash Flow Statement.
|
||||
|
||||
## ⚠️ CRITICAL PRINCIPLES — Read Before Populating Any Template
|
||||
|
||||
**Formulas over hardcodes (non-negotiable):**
|
||||
- Every projection cell, roll-forward, linkage, and subtotal MUST be an Excel formula — never a pre-computed value
|
||||
- When using Python/openpyxl: write formula strings (`ws["D15"] = "=D14*(1+Assumptions!$B$5)"`), NOT computed results (`ws["D15"] = 12500`)
|
||||
- The ONLY cells that should contain hardcoded numbers are: (1) historical actuals, (2) assumption drivers in the Assumptions tab
|
||||
- If you find yourself computing a value in Python and writing the result to a cell — STOP. Write the formula instead.
|
||||
- Why: the model must flex when scenarios toggle or assumptions change. Hardcodes break every downstream integrity check silently.
|
||||
|
||||
**Verify step-by-step with the user:**
|
||||
1. **After mapping the template** → show the user which tabs/sections you've identified and confirm before touching any cells
|
||||
2. **After populating historicals** → show the user the historical block and confirm values/periods match source data
|
||||
3. **After building IS projections** → run the subtotal checks, show the user the projected IS, confirm before moving to BS
|
||||
4. **After building BS** → show the user the balance check (Assets = L+E) for every period, confirm before moving to CF
|
||||
5. **After building CF** → show the user the cash tie-out (CF ending cash = BS cash), confirm before finalizing
|
||||
6. **Do NOT populate the entire model end-to-end and present it complete** — break at each statement, show the work, catch errors early
|
||||
|
||||
## Formatting — Professional Blue/Grey Palette (Default unless template/user specifies otherwise)
|
||||
|
||||
**Keep colors minimal.** Use only blues and greys for cell fills. Do NOT introduce greens, yellows, oranges, or multiple accent colors — a clean model uses restraint.
|
||||
|
||||
| Element | Fill | Font |
|
||||
|---|---|---|
|
||||
| Section headers (IS / BS / CF titles) | Dark blue `#1F4E79` | White bold |
|
||||
| Column headers (FY2024A, FY2025E, etc.) | Light blue `#D9E1F2` | Black bold |
|
||||
| Input cells (historicals, assumption drivers) | Light grey `#F2F2F2` or white | Blue `#0000FF` |
|
||||
| Formula cells | White | Black |
|
||||
| Cross-tab links | White | Green `#008000` |
|
||||
| Check rows / key totals | Medium blue `#BDD7EE` | Black bold |
|
||||
|
||||
**That's 3 blues + 1 grey + white.** If the template has its own color scheme, follow the template instead.
|
||||
|
||||
Font color signals *what* a cell is (input/formula/link). Fill color signals *where* you are (header/data/check).
|
||||
|
||||
## Model Structure
|
||||
|
||||
### Identifying Template Tab Organization
|
||||
|
||||
Templates vary in their tab naming conventions and organization. Before populating, review all tabs to understand the template's structure. Below are common tab names and their typical contents:
|
||||
|
||||
| Common Tab Names | Contents to Look For |
|
||||
|------------------|----------------------|
|
||||
| IS, P&L, Income Statement | Income Statement |
|
||||
| BS, Balance Sheet | Balance Sheet |
|
||||
| CF, CFS, Cash Flow | Cash Flow Statement |
|
||||
| WC, Working Capital | Working Capital Schedule |
|
||||
| DA, D&A, Depreciation, PP&E | Depreciation & Amortization Schedule |
|
||||
| Debt, Debt Schedule | Debt Schedule |
|
||||
| NOL, Tax, DTA | Net Operating Loss Schedule |
|
||||
| Assumptions, Inputs, Drivers | Driver assumptions and inputs |
|
||||
| Checks, Audit, Validation | Error-checking dashboard |
|
||||
|
||||
**Template Review Checklist**
|
||||
- Identify which tabs exist in the template (not all templates include every schedule)
|
||||
- Note any template-specific tabs not listed above
|
||||
- Understand tab dependencies (e.g., which schedules feed into the main statements)
|
||||
- Locate input cells vs. formula cells on each tab
|
||||
|
||||
### Understanding Template Structure
|
||||
|
||||
Before populating a template, familiarize yourself with its existing layout to ensure data is entered in the correct locations and formulas remain intact.
|
||||
|
||||
**Identifying Row Structure**
|
||||
- Locate the model title at top of each tab
|
||||
- Identify section headers and their visual separation
|
||||
- Find the units row indicating $ millions, %, x, etc.
|
||||
- Note column headers distinguishing Actuals vs. Estimates periods
|
||||
- Confirm period labels (e.g., FY2024A, FY2025E)
|
||||
- Identify input cells vs. formula cells (typically distinguished by font color)
|
||||
|
||||
**Identifying Column Structure**
|
||||
- Confirm line item labels in leftmost column
|
||||
- Verify historical years precede projection years
|
||||
- Note the visual border separating historical from projected periods
|
||||
- Check for consistent column order across all tabs
|
||||
|
||||
**Working with Named Ranges**
|
||||
Templates often use named ranges for key inputs and outputs. Before entering data:
|
||||
- Review existing named ranges in the template (Formulas → Name Manager in Excel)
|
||||
- Common named ranges include: Revenue growth rates, cost percentages, key outputs (Net Income, EBITDA, Total Debt, Cash), scenario selector cell
|
||||
- Ensure inputs are entered in cells that feed into these named ranges
|
||||
|
||||
### Projection Period
|
||||
- Templates typically project 5 years forward from last historical year
|
||||
- Verify historical (A) vs. projected (E) columns are clearly separated
|
||||
- Confirm columns use fiscal year notation (e.g., FY2024A, FY2025E)
|
||||
|
||||
## Margin Analysis
|
||||
|
||||
**Note: The following margin analysis should only be performed if prompted by the user or if the template explicitly requires it. If no prompt is given, skip this section.**
|
||||
|
||||
Calculate and display profitability margins on the Income Statement (IS) tab to track operational efficiency and enable peer comparison.
|
||||
|
||||
### Core Margins to Include
|
||||
|
||||
| Margin | Formula | What It Measures |
|
||||
|--------|---------|------------------|
|
||||
| Gross Margin | Gross Profit / Revenue | Pricing power, production efficiency |
|
||||
| EBITDA Margin | EBITDA / Revenue | Core operating profitability |
|
||||
| EBIT Margin | EBIT / Revenue | Operating profitability after D&A |
|
||||
| Net Income Margin | Net Income / Revenue | Bottom-line profitability |
|
||||
|
||||
### Income Statement Layout with Margins
|
||||
|
||||
Display margin percentages directly below each profit line item:
|
||||
- Gross Margin % below Gross Profit
|
||||
- EBIT Margin % below EBIT
|
||||
- EBITDA Margin % below EBITDA
|
||||
- Net Income Margin % below Net Income
|
||||
|
||||
## Credit Metrics
|
||||
|
||||
**Note: The following Credit analysis should only be performed if prompted by the user or if the template explicitly requires it. If no prompt is given, skip this section.**
|
||||
|
||||
Calculate and display credit/leverage metrics on the Balance Sheet (BS) tab to assess financial health, debt capacity, and covenant compliance.
|
||||
|
||||
### Core Credit Metrics to Include
|
||||
|
||||
| Metric | Formula | What It Measures |
|
||||
|--------|---------|------------------|
|
||||
| Total Debt / EBITDA | Total Debt / LTM EBITDA | Leverage multiple |
|
||||
| Net Debt / EBITDA | (Total Debt - Cash) / LTM EBITDA | Leverage net of cash |
|
||||
| Interest Coverage | EBITDA / Interest Expense | Ability to service debt |
|
||||
| Debt / Total Cap | Total Debt / (Total Debt + Equity) | Capital structure |
|
||||
| Debt / Equity | Total Debt / Total Equity | Financial leverage |
|
||||
| Current Ratio | Current Assets / Current Liabilities | Short-term liquidity |
|
||||
| Quick Ratio | (Current Assets - Inventory) / Current Liabilities | Immediate liquidity |
|
||||
|
||||
### Credit Metric Hierarchy Checks
|
||||
|
||||
Validate that Upside shows strongest credit profile:
|
||||
- Leverage: Upside < Base < Downside (lower is better)
|
||||
- Coverage: Upside > Base > Downside (higher is better)
|
||||
- Liquidity: Upside > Base > Downside (higher is better)
|
||||
|
||||
### Covenant Compliance Tracking
|
||||
|
||||
If debt covenants are known, add explicit compliance checks comparing actual metrics to covenant thresholds.
|
||||
|
||||
## Scenario Analysis (Base / Upside / Downside)
|
||||
|
||||
Use a scenario toggle (dropdown) in the Assumptions tab with CHOOSE or INDEX/MATCH formulas.
|
||||
|
||||
| Scenario | Description |
|
||||
|----------|-------------|
|
||||
| Base Case | Management guidance or consensus estimates |
|
||||
| Upside Case | Above-guidance growth, margin expansion |
|
||||
| Downside Case | Below-trend growth, margin compression |
|
||||
|
||||
**Key Drivers to Sensitize**: Revenue growth, Gross margin, SG&A %, DSO/DIO/DPO, CapEx %, Interest rate, Tax rate.
|
||||
|
||||
**Scenario Audit Checks**: Toggle switches all statements, BS balances in all scenarios, Cash ties out, Hierarchy holds (Upside > Base > Downside for NI, EBITDA, FCF, margins).
|
||||
|
||||
## SEC Filings Data Extraction
|
||||
|
||||
If the template specifically requires pulling data from SEC filings (10-K, 10-Q), see [references/sec-filings.md](references/sec-filings.md) for detailed extraction guidance. This reference is only needed when populating templates with public company data from regulatory filings.
|
||||
|
||||
## Completing Model Templates
|
||||
|
||||
This section provides general guidance for completing any 3-statement financial model template while preserving existing formulas and ensuring data integrity.
|
||||
|
||||
### Step 1: Analyze the Template Structure
|
||||
|
||||
Before entering any data, thoroughly review the template to understand its architecture:
|
||||
|
||||
**Identify Input vs. Formula Cells**
|
||||
- Look for visual cues (font color, cell shading) that distinguish input cells from formula cells
|
||||
- Common conventions: Blue font = inputs, Black font = formulas, Green font = links to other sheets
|
||||
- Use Excel's Trace Precedents/Dependents (Formulas → Trace Precedents) to understand cell relationships
|
||||
- Check for named ranges that may control key inputs (Formulas → Name Manager)
|
||||
|
||||
**Map the Template's Flow**
|
||||
- Identify which tabs feed into others (e.g., Assumptions → IS → BS → CF)
|
||||
- Note any supporting schedules and their linkages to main statements
|
||||
- Document the template's specific line items and structure before populating
|
||||
|
||||
### Step 2: Filling in Data Without Breaking Formulas
|
||||
|
||||
**Golden Rules for Data Entry**
|
||||
|
||||
| Rule | Description |
|
||||
|------|-------------|
|
||||
| Only edit input cells | Never overwrite cells containing formulas unless intentionally replacing the formula |
|
||||
| Preserve cell references | When copying data, use Paste Values (Ctrl+Shift+V) to avoid overwriting formulas with source formatting |
|
||||
| Match the template's units | Verify if template uses thousands, millions, or actual values before entering data |
|
||||
| Respect sign conventions | Follow the template's existing sign convention (e.g., expenses as positive or negative) |
|
||||
| Check for circular references | If the template uses iterative calculations, ensure Enable Iterative Calculation is turned on |
|
||||
|
||||
**Safe Data Entry Process**
|
||||
1. Identify the exact cells designated for input (usually highlighted or labeled)
|
||||
2. Enter historical data first, then verify formulas are calculating correctly for those periods
|
||||
3. Enter assumption drivers that feed forecast calculations
|
||||
4. Review calculated outputs to confirm formulas are working as intended
|
||||
5. If a formula cell must be modified, document the original formula before making changes
|
||||
|
||||
**Handling Pre-Built Formulas**
|
||||
- If formulas reference cells you haven't populated yet, expect temporary errors (#REF!, #DIV/0!) until all inputs are complete
|
||||
- When formulas produce unexpected results, trace precedents to identify missing or incorrect inputs
|
||||
- Never delete rows/columns without checking for formula dependencies across all tabs
|
||||
|
||||
### Step 3: Validating Formulas
|
||||
|
||||
**Formula Integrity Checks**
|
||||
|
||||
Before relying on template outputs, validate that formulas are functioning correctly:
|
||||
|
||||
| Check Type | Method |
|
||||
|------------|--------|
|
||||
| Trace precedents | Select a formula cell → Formulas → Trace Precedents to verify it references correct inputs |
|
||||
| Trace dependents | Verify key inputs flow to expected output cells |
|
||||
| Evaluate formula | Use Formulas → Evaluate Formula to step through complex calculations |
|
||||
| Check for hardcodes | Projection formulas should reference assumptions, not contain hardcoded values |
|
||||
| Test with known values | Input simple test values to verify formulas produce expected results |
|
||||
| Cross-tab consistency | Ensure the same formula logic applies across all projection periods |
|
||||
|
||||
**Common Formula Issues to Watch For**
|
||||
- Mixed absolute/relative references causing incorrect results when copied across periods
|
||||
- Broken links to external files or deleted ranges (#REF! errors)
|
||||
- Division by zero in early periods before revenue ramps (#DIV/0! errors)
|
||||
- Circular reference warnings (may be intentional for interest calculations)
|
||||
- Inconsistent formulas across projection columns (use Ctrl+\ to find differences)
|
||||
|
||||
**Validating Cross-Tab Linkages**
|
||||
- Confirm values that appear on multiple tabs are linked (not duplicated)
|
||||
- Verify schedule totals tie to corresponding line items on main statements
|
||||
- Check that period labels align across all tabs
|
||||
|
||||
### Step 4: Quality Checks by Sheet
|
||||
|
||||
Perform these validation checks on each sheet after populating the template:
|
||||
|
||||
**Income Statement (IS) Quality Checks**
|
||||
- Revenue figures match source data for historical periods
|
||||
- All expense line items sum to reported totals
|
||||
- Subtotals (Gross Profit, EBIT, EBT, Net Income) calculate correctly
|
||||
- Tax calculation logic is appropriate (handles losses correctly)
|
||||
- Forecast drivers reference assumptions tab (no hardcodes)
|
||||
- Period-over-period changes are directionally reasonable
|
||||
|
||||
**Balance Sheet (BS) Quality Checks**
|
||||
- Assets = Liabilities + Equity for every period (primary check)
|
||||
- Cash balance matches Cash Flow Statement ending cash
|
||||
- Working capital accounts tie to supporting schedules (if applicable)
|
||||
- Retained Earnings rolls forward correctly: Prior RE + Net Income - Dividends +/- Adjustments = Ending RE
|
||||
- Debt balances tie to debt schedule (if applicable)
|
||||
- All balance sheet items have appropriate signs (assets positive, most liabilities positive)
|
||||
|
||||
**Cash Flow Statement (CF) Quality Checks**
|
||||
- Net Income at top of CFO matches Income Statement Net Income
|
||||
- Non-cash add-backs (D&A, SBC, etc.) tie to their source schedules/statements
|
||||
- Working capital changes have correct signs (increase in asset = use of cash = negative)
|
||||
- CapEx ties to PP&E schedule or fixed asset roll-forward
|
||||
- Financing activities tie to changes in debt and equity accounts on BS
|
||||
- Ending Cash matches Balance Sheet Cash
|
||||
- Beginning Cash equals prior period Ending Cash
|
||||
|
||||
**Supporting Schedule Quality Checks**
|
||||
- Opening balances equal prior period closing balances
|
||||
- Roll-forward logic is complete (Beginning + Additions - Deductions = Ending)
|
||||
- Schedule totals tie to main statement line items
|
||||
- Assumptions used in calculations match Assumptions tab
|
||||
|
||||
### Step 5: Cross-Statement Integrity Checks
|
||||
|
||||
After validating individual sheets, confirm the three statements are properly integrated:
|
||||
|
||||
| Check | Formula | Expected Result |
|
||||
|-------|---------|-----------------|
|
||||
| Balance Sheet Balance | Assets - Liabilities - Equity | = 0 |
|
||||
| Cash Tie-Out | CF Ending Cash - BS Cash | = 0 |
|
||||
| Net Income Link | IS Net Income - CF Starting Net Income | = 0 |
|
||||
| Retained Earnings | Prior RE + NI - Dividends - BS Ending RE | = 0 (adjust for SBC/other items as needed) |
|
||||
|
||||
### Step 6: Final Review
|
||||
|
||||
Before considering the model complete:
|
||||
- Toggle through all scenarios (if applicable) to verify checks pass in each case
|
||||
- Review all #REF!, #DIV/0!, #VALUE!, and #NAME? errors and resolve or document
|
||||
- Confirm all input cells have been populated (search for placeholder values)
|
||||
- Verify units are consistent across all tabs
|
||||
- Save a clean version before making any additional modifications
|
||||
|
||||
## Model Validation and Audit
|
||||
|
||||
This section consolidates all validation checks and audit procedures for completed templates.
|
||||
|
||||
### Core Linkages (Must Always Hold)
|
||||
|
||||
See [references/formulas.md](references/formulas.md) for all formula details.
|
||||
|
||||
| Check | Formula | Expected Result |
|
||||
|-------|---------|-----------------|
|
||||
| Balance Sheet Balance | Assets - Liabilities - Equity | = 0 |
|
||||
| Cash Tie-Out | CF Ending Cash - BS Cash | = 0 |
|
||||
| Cash Monthly vs Annual | Closing Cash (Monthly) - Closing Cash (Annual) | = 0 |
|
||||
| Net Income Link | IS Net Income - CF Starting Net Income | = 0 |
|
||||
| Retained Earnings | Prior RE + NI + SBC - Dividends - BS Ending RE | = 0 |
|
||||
| Equity Financing | ΔCommon Stock/APIC (BS) - Equity Issuance (CFF) | = 0 |
|
||||
| Year 0 Equity | Equity Raised (Year 0) - Beginning Equity Capital (Year 1) | = 0 |
|
||||
|
||||
### Sign Convention Reference
|
||||
|
||||
| Statement | Item | Sign Convention |
|
||||
|-----------|------|-----------------|
|
||||
| CFO | D&A, SBC | Positive (add-back) |
|
||||
| CFO | ΔAR (increase) | Negative (use of cash) |
|
||||
| CFO | ΔAP (increase) | Positive (source of cash) |
|
||||
| CFI | CapEx | Negative |
|
||||
| CFF | Debt issuance | Positive |
|
||||
| CFF | Debt repayments | Negative |
|
||||
| CFF | Dividends | Negative |
|
||||
|
||||
### Circular Reference Handling
|
||||
|
||||
Interest expense creates circularity: Interest → Net Income → Cash → Debt Balance → Interest
|
||||
|
||||
Enable iterative calculation in Excel: File → Options → Formulas → Enable iterative calculation. Set maximum iterations to 100, maximum change to 0.001. Add a circuit breaker toggle in Assumptions tab.
|
||||
|
||||
### Check Categories
|
||||
|
||||
**Section 1: Currency Consistency**
|
||||
- Currency identified and documented in Assumptions
|
||||
- All tabs use consistent currency symbol and scale
|
||||
- Units row matches model currency
|
||||
|
||||
**Section 2: Balance Sheet Integrity**
|
||||
- Assets = Liabilities + Equity (for each period)
|
||||
- Formula: Assets - Liabilities - Equity (must = 0)
|
||||
|
||||
**Section 3: Cash Flow Integrity**
|
||||
- Cash ties to BS (CF Ending Cash = BS Cash)
|
||||
- Cash Monthly vs Annual: Closing Cash (Monthly) = Closing Cash (Annual)
|
||||
- NI ties to IS (CF Net Income = IS Net Income)
|
||||
- D&A ties to schedule
|
||||
- SBC ties to IS
|
||||
- ΔAR, ΔInventory, ΔAP tie to WC schedule
|
||||
- CapEx ties to DA schedule
|
||||
|
||||
**Section 4: Retained Earnings**
|
||||
- RE roll-forward check: Prior RE + NI + SBC - Dividends = Ending RE
|
||||
- Show component breakdown for debugging
|
||||
|
||||
**Section 5: Working Capital**
|
||||
- AR, Inventory, AP tie to BS
|
||||
- DSO, DIO, DPO reasonability checks (flag if outside normal ranges)
|
||||
|
||||
**Section 6: Debt Schedule**
|
||||
- Total Debt ties to BS (Current + LT Debt)
|
||||
- Interest calculation ties to IS
|
||||
|
||||
**Section 6b: Equity Financing**
|
||||
- Equity issuance proceeds tie to BS Common Stock/APIC increase
|
||||
- Cash increase from equity = Equity account increase (must balance)
|
||||
- Equity Raise Tie-Out: ΔCommon Stock/APIC (BS) = Equity Issuance (CFF) (must = 0)
|
||||
- Year 0 Equity Tie-Out: Equity Raised (Year 0) = Beginning Equity Capital (Year 1)
|
||||
|
||||
**Section 6c: NOL Schedule**
|
||||
- Beginning NOL (Year 1 / Formation) = 0 (new business starts with zero NOL)
|
||||
- NOL increases only when EBT < 0 (losses must be realized to generate NOL)
|
||||
- DTA ties to BS (NOL Schedule DTA = BS Deferred Tax Asset)
|
||||
- NOL utilization ≤ 80% of EBT (post-2017 federal limitation)
|
||||
- NOL balance is non-negative (cannot utilize more than available)
|
||||
- NOL generated only when EBT < 0
|
||||
- Tax expense = 0 when taxable income ≤ 0
|
||||
|
||||
**Section 7: Scenario Hierarchy**
|
||||
- Absolute metrics: Upside > Base > Downside (NI, EBITDA, FCF)
|
||||
- Margins: Upside > Base > Downside (GM%, EBITDA%, NI%)
|
||||
- Credit metrics: Upside < Base < Downside for leverage (inverted)
|
||||
|
||||
**Section 8: Formula Integrity**
|
||||
- COGS, S&M, G&A, R&D, SBC driven by % of Revenue (no hardcodes)
|
||||
- Consistent formulas across projection years
|
||||
- No #REF!, #DIV/0!, #VALUE! errors
|
||||
|
||||
**Section 9: Credit Metric Thresholds**
|
||||
- Flag metrics as Green/Yellow/Red based on covenant thresholds
|
||||
- Summary of any red flags
|
||||
|
||||
### Master Check Formula
|
||||
|
||||
Aggregate all section statuses into a single master check:
|
||||
- If all sections pass → "✓ ALL CHECKS PASS"
|
||||
- If any section fails → "✗ ERRORS DETECTED - REVIEW BELOW"
|
||||
|
||||
### Quick Debug Workflow
|
||||
|
||||
When Master Status shows errors:
|
||||
1. Scroll to find red-highlighted sections
|
||||
2. Identify which check category has failures
|
||||
3. Navigate to source tab to investigate
|
||||
4. Fix the underlying issue
|
||||
5. Return to Checks tab to verify resolution
|
||||
|
||||
|
||||
## Data sources — MCP first, web fallback
|
||||
|
||||
Many passages below say "use the S&P Kensho MCP / Daloopa MCP / FactSet MCP". Those are commercial financial-data MCPs from the original Cowork plugin context. In Hermes:
|
||||
|
||||
- **If you have any structured financial-data MCP configured** (Hermes supports MCP — see `native-mcp` skill), prefer it for point-in-time comps, precedent transactions, and filings.
|
||||
- **Otherwise**, fall back to:
|
||||
- `web_search` / `web_extract` against SEC EDGAR (`https://www.sec.gov/cgi-bin/browse-edgar`) for US filings
|
||||
- Company IR pages for press releases, earnings decks
|
||||
- `browser_navigate` for interactive data portals
|
||||
- User-provided data (explicitly ask when the context doesn't have it)
|
||||
- **Never fabricate**. If a multiple, precedent, or filing number can't be sourced, flag the cell as `[UNSOURCED]` and surface it to the user.
|
||||
|
||||
## Attribution
|
||||
|
||||
This skill is adapted from Anthropic's Claude for Financial Services plugin suite (Apache-2.0). The Office-JS / Cowork live-Excel paths have been removed; this version targets headless openpyxl via the `excel-author` skill's conventions. Original: https://github.com/anthropics/financial-services
|
||||
@@ -0,0 +1,118 @@
|
||||
# Formatting Standards Reference
|
||||
|
||||
| Element | Format |
|
||||
|---------|--------|
|
||||
| Hard-coded inputs | Blue font |
|
||||
| Formulas | Black font |
|
||||
| Links to other sheets | Green font |
|
||||
| Check cells | Red if error, green if balanced |
|
||||
| Negative values | Parentheses, not minus signs |
|
||||
| Currency | No decimals for large figures, 2 decimals for per-share |
|
||||
| Percentages | 1 decimal place |
|
||||
| Headers | Bold, bottom border |
|
||||
| Units row | Include units row below headers ($ millions, %, etc.) |
|
||||
|
||||
## Visual Separation Guidelines
|
||||
|
||||
- Thin vertical border between historical and projected columns
|
||||
- Thick bottom border after section totals (e.g., Total Assets)
|
||||
- Single bottom border for subtotals
|
||||
- Double bottom border for grand totals
|
||||
|
||||
## Total and Subtotal Row Formatting
|
||||
|
||||
All total and subtotal rows must use **bold font formatting** for their numerical values to clearly distinguish aggregated figures from individual line items.
|
||||
|
||||
### Income Statement (P&L) Tab
|
||||
| Row | Formatting |
|
||||
|-----|------------|
|
||||
| Gross Revenue | Bold |
|
||||
| Total Cost of Revenue | Bold |
|
||||
| Gross Profit | Bold |
|
||||
| Total SG&A | Bold |
|
||||
| EBITDA | Bold |
|
||||
| EBIT | Bold |
|
||||
| EBT | Bold |
|
||||
| Net Profit After Tax | Bold |
|
||||
|
||||
### Balance Sheet Tab
|
||||
| Row | Formatting |
|
||||
|-----|------------|
|
||||
| Total Current Assets | Bold |
|
||||
| Total Non-Current Assets | Bold |
|
||||
| Total Other Assets | Bold |
|
||||
| Total Assets | Bold |
|
||||
| Total Current Liabilities | Bold |
|
||||
| Total Non-Current Liabilities | Bold |
|
||||
| Total Equity | Bold |
|
||||
| Total Liabilities and Equity | Bold |
|
||||
|
||||
### Cash Flow Statement Tab
|
||||
| Row | Formatting |
|
||||
|-----|------------|
|
||||
| Cash Generated from Operations Before Working Capital Changes | Bold |
|
||||
| Total Working Capital Changes | Bold |
|
||||
| Net Cash Generated from Operations | Bold |
|
||||
| Net Cash Flow from Investing Activities | Bold |
|
||||
| Net Cash Flow from Financing Activities | Bold |
|
||||
| Closing Cash Balance | Bold |
|
||||
|
||||
**Note:** This list is non-exhaustive. Apply bold formatting to any row that represents a total, subtotal, or summary calculation across the model.
|
||||
|
||||
## Balance Sheet Check Row Formatting
|
||||
|
||||
The Balance Sheet check row (below Total Liabilities and Equity) uses conditional number formatting that displays non-zero values in red. When the balance sheet balances correctly (check = 0), the values display in black or standard formatting.
|
||||
|
||||
| Check Value | Font Color |
|
||||
|-------------|------------|
|
||||
| = 0 (balanced) | Black (standard) |
|
||||
| ≠ 0 (error) | Red |
|
||||
|
||||
**Implementation:** Apply custom number format `[Red][<>0]0.00;[Red][<>0](0.00);0.00` or use Excel conditional formatting with the rule "Cell Value ≠ 0" → Red font.
|
||||
|
||||
## Margin Row Formatting
|
||||
|
||||
| Element | Format |
|
||||
|---------|--------|
|
||||
| Margin % rows | Indent, italics, 1 decimal place |
|
||||
| Positive trend | No special formatting (or subtle green) |
|
||||
| Negative trend | Flag for review (subtle yellow) |
|
||||
| Below peer average | Consider highlighting for discussion |
|
||||
|
||||
## Credit Metric Formatting
|
||||
|
||||
| Element | Format |
|
||||
|---------|--------|
|
||||
| Leverage multiples | 1 decimal with "x" suffix (e.g., 2.5x) |
|
||||
| Percentages | 1 decimal with "%" suffix |
|
||||
| Net Debt negative | Parentheses, indicates net cash position |
|
||||
| Section header | Bold, "CREDIT METRICS" |
|
||||
| Separator line | Thin border above credit metrics section |
|
||||
|
||||
## Credit Metric Threshold Colors
|
||||
|
||||
| Metric | Green | Yellow | Red |
|
||||
|--------|-------|--------|-----|
|
||||
| Total Debt / EBITDA | < 2.5x | 2.5x-4.0x | > 4.0x |
|
||||
| Net Debt / EBITDA | < 2.0x | 2.0x-3.5x | > 3.5x |
|
||||
| Interest Coverage | > 4.0x | 2.5x-4.0x | < 2.5x |
|
||||
| Debt / Total Cap | < 40% | 40%-60% | > 60% |
|
||||
| Current Ratio | > 1.5x | 1.0x-1.5x | < 1.0x |
|
||||
| Quick Ratio | > 1.0x | 0.75x-1.0x | < 0.75x |
|
||||
|
||||
## Conditional Formatting for Checks Tab
|
||||
|
||||
- Cell contains pass indicator → Green fill
|
||||
- Cell contains fail indicator → Red fill
|
||||
- Cell contains warning → Yellow fill
|
||||
- Difference cells = 0 → Light green fill
|
||||
- Difference cells ≠ 0 → Light red fill
|
||||
|
||||
## Margin Reasonability Flags
|
||||
|
||||
- Gross Margin < 0% → ERROR: Review COGS
|
||||
- Gross Margin > 80% → WARNING: Verify revenue/COGS
|
||||
- EBITDA Margin < 0% → FLAG: Operating losses
|
||||
- EBITDA Margin > 50% → WARNING: Unusually high
|
||||
- Net Margin < 0% → FLAG: Net losses (may be acceptable in growth phase)
|
||||
- Net Margin > Gross Margin → ERROR: Formula issue
|
||||
@@ -0,0 +1,292 @@
|
||||
# Formula Reference
|
||||
|
||||
**IMPORTANT:** Use the formulas outlined in this reference document unless otherwise specified by the user.
|
||||
|
||||
---
|
||||
|
||||
## Core Linkages
|
||||
|
||||
```
|
||||
Balance Sheet: Assets = Liabilities + Equity
|
||||
Net Income: IS Net Income → CF Operations (starting point)
|
||||
Cash Flow: ΔCash = CFO + CFI + CFF
|
||||
Cash Tie-Out: Ending Cash (CF) = Cash (BS Asset)
|
||||
Cash Monthly/Annual: Closing Cash (Monthly) = Closing Cash (Annual)
|
||||
Retained Earnings: Prior RE + Net Income - Dividends = Ending RE
|
||||
Equity Raise: ΔCommon Stock/APIC (BS) = Equity Issuance (CFF)
|
||||
Year 0 Equity: Equity Raised (Year 0) = Beginning Equity (Year 1)
|
||||
```
|
||||
|
||||
## Gross Profit Calculation
|
||||
|
||||
**IMPORTANT:** Gross Profit must be calculated from Net Revenue, not Gross Revenue.
|
||||
|
||||
```
|
||||
Net Revenue - Cost of Revenue = Gross Profit
|
||||
```
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| Gross Revenue | Total revenue before any deductions |
|
||||
| Net Revenue | Gross Revenue - Returns - Allowances - Discounts |
|
||||
| Cost of Revenue | Direct costs attributable to production of goods/services sold |
|
||||
| Gross Profit | Net Revenue - Cost of Revenue |
|
||||
|
||||
**Note:** Always use Net Revenue (also called "Net Sales" or simply "Revenue" on most financial statements) as the starting point for profitability calculations. Gross Revenue overstates the true top-line performance.
|
||||
|
||||
## Margin Formulas
|
||||
|
||||
```
|
||||
Gross Margin % = Gross Profit / Net Revenue
|
||||
EBITDA = EBIT + D&A (or = Gross Profit - OpEx)
|
||||
EBITDA Margin % = EBITDA / Net Revenue
|
||||
EBIT Margin % = EBIT / Net Revenue
|
||||
Net Income Margin % = Net Income / Net Revenue
|
||||
```
|
||||
|
||||
## Credit Metric Formulas
|
||||
|
||||
```
|
||||
Total Debt = Current Portion of Debt + Long-Term Debt
|
||||
Net Debt = Total Debt - Cash
|
||||
Total Debt / EBITDA = Total Debt / EBITDA (from IS)
|
||||
Net Debt / EBITDA = Net Debt / EBITDA (from IS)
|
||||
Interest Coverage = EBITDA / Interest Expense (from IS)
|
||||
Net Int Exp % Debt = Net Interest Expense / Long-Term Debt
|
||||
Debt / Total Cap = Total Debt / (Total Debt + Total Equity)
|
||||
Debt / Equity = Total Debt / Total Equity
|
||||
Current Ratio = Total Current Assets / Total Current Liabilities
|
||||
Quick Ratio = (Total Current Assets - Inventory) / Total Current Liabilities
|
||||
```
|
||||
|
||||
## Forecast Formulas (% of Net Revenue Method)
|
||||
|
||||
```
|
||||
Cost of Revenue (Forecast) = Net Revenue × Cost of Revenue % Assumption
|
||||
S&M (Forecast) = Net Revenue × S&M % Assumption
|
||||
G&A (Forecast) = Net Revenue × G&A % Assumption
|
||||
R&D (Forecast) = Net Revenue × R&D % Assumption
|
||||
SBC (Forecast) = Net Revenue × SBC % Assumption
|
||||
```
|
||||
|
||||
## Working Capital Formulas
|
||||
|
||||
```
|
||||
Accounts Receivable
|
||||
Prior AR
|
||||
+ Revenue (from IS)
|
||||
- Cash Collections (plug)
|
||||
= Ending AR
|
||||
DSO = (AR / Revenue) × 365
|
||||
|
||||
Inventory
|
||||
Prior Inventory
|
||||
+ Purchases (plug)
|
||||
- COGS (from IS)
|
||||
= Ending Inventory
|
||||
DIO = (Inventory / COGS) × 365
|
||||
|
||||
Accounts Payable
|
||||
Prior AP
|
||||
+ Purchases (from Inventory calc)
|
||||
- Cash Payments (plug)
|
||||
= Ending AP
|
||||
DPO = (AP / COGS) × 365
|
||||
|
||||
Net Working Capital = AR + Inventory - AP
|
||||
ΔWC = Current NWC - Prior NWC
|
||||
```
|
||||
|
||||
## D&A Schedule Formulas
|
||||
|
||||
```
|
||||
Beginning PP&E (Gross)
|
||||
+ CapEx
|
||||
= Ending PP&E (Gross)
|
||||
|
||||
Beginning Accumulated Depreciation
|
||||
+ Depreciation Expense
|
||||
= Ending Accumulated Depreciation
|
||||
|
||||
PP&E (Net) = Gross PP&E - Accumulated Depreciation
|
||||
```
|
||||
|
||||
## Debt Schedule Formulas
|
||||
|
||||
```
|
||||
Beginning Debt Balance
|
||||
+ New Borrowings
|
||||
- Repayments
|
||||
= Ending Debt Balance
|
||||
|
||||
Interest Expense = Avg Debt Balance × Interest Rate
|
||||
(Use beginning balance to avoid circularity, or iterate if circular refs enabled)
|
||||
```
|
||||
|
||||
## Retained Earnings Formula
|
||||
|
||||
```
|
||||
Beginning Retained Earnings
|
||||
+ Net Income (from IS)
|
||||
+ Stock-Based Compensation (SBC) (from IS)
|
||||
- Dividends
|
||||
= Ending Retained Earnings
|
||||
```
|
||||
|
||||
## NOL (Net Operating Loss) Schedule Formulas
|
||||
|
||||
```
|
||||
NOL CARRYFORWARD SCHEDULE
|
||||
|
||||
Beginning NOL Balance (Year 1 / Formation = 0)
|
||||
+ NOL Generated (if EBT < 0, then ABS(EBT), else 0)
|
||||
- NOL Utilized (limited by taxable income and utilization cap)
|
||||
= Ending NOL Balance
|
||||
|
||||
STARTING BALANCE RULE
|
||||
|
||||
For a new business or first modeled period:
|
||||
Beginning NOL Balance = 0
|
||||
NOL can only increase through realized losses (EBT < 0)
|
||||
NOL cannot be created from thin air or assumed
|
||||
|
||||
NOL UTILIZATION CALCULATION
|
||||
|
||||
Pre-Tax Income (EBT)
|
||||
If EBT > 0:
|
||||
NOL Available = Beginning NOL Balance
|
||||
Utilization Limit = EBT × 80% (post-2017 federal limit)
|
||||
NOL Utilized = MIN(NOL Available, Utilization Limit)
|
||||
Taxable Income = EBT - NOL Utilized
|
||||
If EBT ≤ 0:
|
||||
NOL Utilized = 0
|
||||
Taxable Income = 0
|
||||
NOL Generated = ABS(EBT)
|
||||
|
||||
TAX CALCULATION WITH NOL
|
||||
|
||||
Taxes Payable = MAX(0, Taxable Income × Tax Rate)
|
||||
(Taxes cannot be negative; losses create NOL asset instead)
|
||||
|
||||
DEFERRED TAX ASSET (DTA) FOR NOL
|
||||
|
||||
DTA - NOL Carryforward = Ending NOL Balance × Tax Rate
|
||||
ΔDTA = Current DTA - Prior DTA
|
||||
(Increase in DTA = non-cash benefit on CF)
|
||||
(Decrease in DTA = non-cash expense on CF)
|
||||
```
|
||||
|
||||
## Balance Sheet Structure
|
||||
|
||||
```
|
||||
ASSETS
|
||||
Cash (from CF ending cash)
|
||||
Accounts Receivable (from WC)
|
||||
Inventory (from WC)
|
||||
Total Current Assets
|
||||
|
||||
PP&E, Net (from DA)
|
||||
Deferred Tax Asset - NOL (from NOL schedule)
|
||||
Total Non-Current Assets
|
||||
Total Assets
|
||||
|
||||
LIABILITIES
|
||||
Accounts Payable (from WC)
|
||||
Current Portion of Debt (from Debt)
|
||||
Total Current Liabilities
|
||||
|
||||
Long-Term Debt (from Debt)
|
||||
Total Liabilities
|
||||
|
||||
EQUITY
|
||||
Common Stock
|
||||
Retained Earnings (from RE schedule)
|
||||
Total Equity
|
||||
|
||||
CHECK: Assets - Liabilities - Equity = 0
|
||||
```
|
||||
|
||||
## Cash Flow Statement Structure
|
||||
|
||||
```
|
||||
CASH FROM OPERATIONS (CFO)
|
||||
Net Income (LINK: IS)
|
||||
+ D&A (LINK: DA schedule)
|
||||
+ Stock-Based Compensation (SBC) (LINK: IS or Assumptions)
|
||||
- ΔDTA (Deferred Tax Asset) (LINK: NOL schedule; increase in DTA = use of cash)
|
||||
- ΔAR (LINK: WC)
|
||||
- ΔInventory (LINK: WC)
|
||||
+ ΔAP (LINK: WC)
|
||||
= CFO
|
||||
|
||||
CASH FROM INVESTING (CFI)
|
||||
- CapEx (LINK: DA schedule)
|
||||
= CFI
|
||||
|
||||
CASH FROM FINANCING (CFF)
|
||||
+ Debt Issuance (LINK: Debt)
|
||||
- Debt Repayment (LINK: Debt)
|
||||
+ Equity Issuance (LINK: BS Common Stock/APIC)
|
||||
- Dividends (LINK: RE schedule)
|
||||
= CFF
|
||||
|
||||
Net Change in Cash = CFO + CFI + CFF
|
||||
Beginning Cash
|
||||
+ Net Change in Cash
|
||||
= Ending Cash (LINK TO: BS Cash)
|
||||
```
|
||||
|
||||
## Income Statement Structure
|
||||
|
||||
```
|
||||
Net Revenue
|
||||
Growth %
|
||||
(-) Cost of Revenue
|
||||
% of Net Revenue
|
||||
────────────────
|
||||
Gross Profit (= Net Revenue - Cost of Revenue)
|
||||
Gross Margin %
|
||||
|
||||
(-) S&M
|
||||
% of Net Revenue
|
||||
(-) G&A
|
||||
% of Net Revenue
|
||||
(-) R&D
|
||||
% of Net Revenue
|
||||
(-) D&A
|
||||
(-) SBC
|
||||
% of Net Revenue
|
||||
────────────────
|
||||
EBIT
|
||||
EBIT Margin %
|
||||
|
||||
EBITDA
|
||||
EBITDA Margin %
|
||||
|
||||
(-) Interest Expense
|
||||
────────────────
|
||||
EBT (Pre-Tax Income)
|
||||
(-) NOL Utilization (from NOL schedule, reduces taxable income)
|
||||
────────────────
|
||||
Taxable Income
|
||||
(-) Taxes (Taxable Income × Tax Rate)
|
||||
────────────────
|
||||
Net Income
|
||||
Net Income Margin %
|
||||
```
|
||||
|
||||
## Check Formulas
|
||||
|
||||
```
|
||||
BS Balance Check: = Assets - Liabilities - Equity (must = 0)
|
||||
Cash Tie-Out: = BS Cash - CF Ending Cash (must = 0)
|
||||
RE Roll-Forward: = Prior RE + NI + SBC - Div - BS RE (must = 0)
|
||||
DTA Tie-Out: = NOL Schedule DTA - BS DTA (must = 0)
|
||||
Equity Raise Tie-Out: = ΔCommon Stock/APIC (BS) - Equity Issuance (CFF) (must = 0)
|
||||
Year 0 Equity Tie-Out: = Equity Raised (Year 0) - Beginning Equity (Year 1) (must = 0)
|
||||
Cash Monthly vs Annual: = Closing Cash (Monthly) - Closing Cash (Annual) (must = 0)
|
||||
NOL Utilization Cap: = NOL Utilized ≤ EBT × 80% (must be TRUE for post-2017)
|
||||
NOL Non-Negative: = Ending NOL Balance ≥ 0 (must be TRUE)
|
||||
NOL Starting Balance: = Beginning NOL (Year 1) = 0 (must be TRUE for new business)
|
||||
NOL Accumulation: = NOL increases only when EBT < 0 (losses generate NOL)
|
||||
```
|
||||
@@ -0,0 +1,125 @@
|
||||
# SEC Filings Data Extraction Reference
|
||||
|
||||
**When to Use:** Only reference this file when a model template specifically requires pulling data from SEC filings (10-K, 10-Q). For templates that provide data directly or use other data sources, this reference is not needed.
|
||||
|
||||
---
|
||||
|
||||
## Extracting Data from SEC Filings (10-K / 10-Q)
|
||||
|
||||
When populating a model template with public company data, extract financials directly from SEC filings.
|
||||
|
||||
### Step 1: Locate the Filing
|
||||
|
||||
1. Use SEC EDGAR: `https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=[TICKER]&type=10-K`
|
||||
2. For quarterly data, use `type=10-Q`
|
||||
|
||||
### Step 2: Identify Filing Currency
|
||||
|
||||
Before extracting data, identify the reporting currency:
|
||||
- Check the cover page or header for reporting currency
|
||||
- Look at statement headers (e.g., "in thousands of U.S. dollars")
|
||||
- Review Note 1 (Summary of Significant Accounting Policies)
|
||||
|
||||
**Common Currency Indicators**
|
||||
|
||||
| Indicator | Currency |
|
||||
|-----------|----------|
|
||||
| $, USD | US Dollar |
|
||||
| €, EUR | Euro |
|
||||
| £, GBP | British Pound |
|
||||
| ¥, JPY | Japanese Yen |
|
||||
| ¥, CNY, RMB | Chinese Yuan |
|
||||
| CHF | Swiss Franc |
|
||||
| CAD, C$ | Canadian Dollar |
|
||||
|
||||
Set model currency to match filing; document in Assumptions tab.
|
||||
|
||||
### Step 3: Navigate to Financial Statements
|
||||
|
||||
Within the 10-K or 10-Q, locate:
|
||||
- **Item 8** (10-K) or **Item 1** (10-Q): Financial Statements
|
||||
- Key sections to extract:
|
||||
- Consolidated Statements of Operations (Income Statement)
|
||||
- Consolidated Balance Sheets
|
||||
- Consolidated Statements of Cash Flows
|
||||
- Notes to Financial Statements (for schedule details)
|
||||
|
||||
### Step 4: Data Extraction Mapping
|
||||
|
||||
**Income Statement (from Consolidated Statements of Operations)**
|
||||
|
||||
| Filing Line Item | Model Line Item |
|
||||
|------------------|-----------------|
|
||||
| Net revenues / Net sales | Revenue |
|
||||
| Cost of goods sold | COGS |
|
||||
| Selling, general and administrative | SG&A |
|
||||
| Depreciation and amortization | D&A |
|
||||
| Interest expense, net | Interest Expense |
|
||||
| Income tax expense | Taxes |
|
||||
| Net income | Net Income |
|
||||
|
||||
**Balance Sheet (from Consolidated Balance Sheets)**
|
||||
|
||||
| Filing Line Item | Model Line Item |
|
||||
|------------------|-----------------|
|
||||
| Cash and cash equivalents | Cash |
|
||||
| Accounts receivable, net | AR |
|
||||
| Inventories | Inventory |
|
||||
| Property, plant and equipment, net | PP&E (Net) |
|
||||
| Total assets | Total Assets |
|
||||
| Accounts payable | AP |
|
||||
| Short-term debt / Current portion of LT debt | Current Debt |
|
||||
| Long-term debt | LT Debt |
|
||||
| Retained earnings | Retained Earnings |
|
||||
| Total stockholders' equity | Total Equity |
|
||||
|
||||
**Cash Flow Statement (from Consolidated Statements of Cash Flows)**
|
||||
|
||||
| Filing Line Item | Model Line Item |
|
||||
|------------------|-----------------|
|
||||
| Net income | Net Income |
|
||||
| Depreciation and amortization | D&A |
|
||||
| Changes in accounts receivable | ΔAR |
|
||||
| Changes in inventories | ΔInventory |
|
||||
| Changes in accounts payable | ΔAP |
|
||||
| Capital expenditures | CapEx |
|
||||
| Proceeds from issuance of common stock | Equity Issuance |
|
||||
| Proceeds from / Repayments of debt | Debt activity |
|
||||
| Dividends paid | Dividends |
|
||||
|
||||
### Step 5: Extract Supporting Detail from Notes
|
||||
|
||||
For schedules, pull from Notes to Financial Statements:
|
||||
- **Note: Debt** → Maturity schedule, interest rates, covenants
|
||||
- **Note: Property, Plant & Equipment** → Gross PP&E, accumulated depreciation, useful lives
|
||||
- **Note: Revenue** → Segment breakdowns, geographic splits
|
||||
- **Note: Leases** → Operating vs. finance lease obligations
|
||||
|
||||
### Step 6: Historical Data Requirements
|
||||
|
||||
Extract 3 years of historical data minimum:
|
||||
- 10-K provides 3 years of IS/CF, 2 years of BS
|
||||
- For 3rd year BS, pull from prior year's 10-K
|
||||
- Use 10-Qs to fill in quarterly granularity if needed
|
||||
|
||||
### Data Extraction Checklist
|
||||
|
||||
- Identify reporting currency and scale (thousands, millions)
|
||||
- 3 years historical Income Statement
|
||||
- 3 years historical Cash Flow Statement
|
||||
- 3 years historical Balance Sheet
|
||||
- Verify IS Net Income = CF starting Net Income (each year)
|
||||
- Verify BS Cash = CF Ending Cash (each year)
|
||||
- Extract debt maturity schedule from notes
|
||||
- Extract D&A detail or useful life assumptions
|
||||
- Note any non-recurring / one-time items to normalize
|
||||
|
||||
### Handling Common Filing Variations
|
||||
|
||||
| Variation | How to Handle |
|
||||
|-----------|---------------|
|
||||
| D&A embedded in COGS/SG&A | Pull D&A from Cash Flow Statement |
|
||||
| "Other" line items are material | Check notes for breakdown |
|
||||
| Restatements | Use restated figures, note in assumptions |
|
||||
| Fiscal year ≠ calendar year | Label with fiscal year end (e.g., FYE Jan 2025) |
|
||||
| Non-USD reporting currency | Adapt model currency to match filing |
|
||||
@@ -0,0 +1,662 @@
|
||||
---
|
||||
name: comps-analysis
|
||||
description: Build comparable company analysis in Excel — operating metrics, valuation multiples, statistical benchmarking vs peer sets. Pairs with excel-author. Use for public-company valuation, IPO pricing, sector benchmarking, or outlier detection.
|
||||
version: 1.0.0
|
||||
author: Anthropic (adapted by Nous Research)
|
||||
license: Apache-2.0
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [finance, valuation, comps, excel, openpyxl, modeling, investment-banking]
|
||||
related_skills: [excel-author, pptx-author, dcf-model, lbo-model]
|
||||
---
|
||||
|
||||
## Environment
|
||||
|
||||
This skill assumes **headless openpyxl** — you are producing an .xlsx file on disk.
|
||||
Follow the `excel-author` skill's conventions for cell coloring, formulas, named ranges, and sensitivity tables.
|
||||
Recalculate before delivery: `python /path/to/excel-author/scripts/recalc.py ./out/model.xlsx`.
|
||||
|
||||
# Comparable Company Analysis
|
||||
|
||||
## ⚠️ CRITICAL: Data Source Priority (READ FIRST)
|
||||
|
||||
**ALWAYS follow this data source hierarchy:**
|
||||
|
||||
1. **FIRST: Check for MCP data sources** - If S&P Kensho MCP, FactSet MCP, or Daloopa MCP are available, use them exclusively for financial and trading information
|
||||
2. **DO NOT use web search** if the above MCP data sources are available
|
||||
3. **ONLY if MCPs are unavailable:** Then use Bloomberg Terminal, SEC EDGAR filings, or other institutional sources
|
||||
4. **NEVER use web search as a primary data source** - it lacks the accuracy, audit trails, and reliability required for institutional-grade analysis
|
||||
|
||||
**Why this matters:** MCP sources provide verified, institutional-grade data with proper citations. Web search results can be outdated, inaccurate, or unreliable for financial analysis.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
This skill teaches the agent to build institutional-grade comparable company analyses that combine operating metrics, valuation multiples, and statistical benchmarking. The output is a structured Excel/spreadsheet that enables informed investment decisions through peer comparison.
|
||||
|
||||
**Reference Material & Contextualization:**
|
||||
|
||||
An example comparable company analysis is provided in `examples/comps_example.xlsx`. When using this or other example files in this skill directory, use them intelligently:
|
||||
|
||||
**DO use examples for:**
|
||||
- Understanding structural hierarchy (how sections flow)
|
||||
- Grasping the level of rigor expected (statistical depth, documentation standards)
|
||||
- Learning principles (clear headers, transparent formulas, audit trails)
|
||||
|
||||
**DO NOT use examples for:**
|
||||
- Exact reproduction of format or metrics
|
||||
- Copying layout without considering context
|
||||
- Applying the same visual style regardless of audience
|
||||
|
||||
**ALWAYS ask yourself first:**
|
||||
1. **"Do you have a preferred format or should I adapt the template style?"**
|
||||
2. **"Who is the audience?"** (Investment committee, board presentation, quick reference, detailed memo)
|
||||
3. **"What's the key question?"** (Valuation, growth analysis, competitive positioning, efficiency)
|
||||
4. **"What's the context?"** (M&A evaluation, investment decision, sector benchmarking, performance review)
|
||||
|
||||
**Adapt based on specifics:**
|
||||
- **Industry context**: Big tech mega-caps need different metrics than emerging SaaS startups
|
||||
- **Sector-specific needs**: Add relevant metrics early (e.g., cloud ARR, enterprise customers, developer ecosystem for tech)
|
||||
- **Company familiarity**: Well-known companies may need less background, more focus on delta analysis
|
||||
- **Decision type**: M&A requires different emphasis than ongoing portfolio monitoring
|
||||
|
||||
**Core principle:** Use template principles (clear structure, statistical rigor, transparent formulas) but vary execution based on context. The goal is institutional-quality analysis, not institutional-looking templates.
|
||||
|
||||
User-provided examples and explicit preferences always take precedence over defaults.
|
||||
|
||||
## Core Philosophy
|
||||
**"Build the right structure first, then let the data tell the story."**
|
||||
|
||||
Start with headers that force strategic thinking about what matters, input clean data, build transparent formulas, and let statistics emerge automatically. A good comp should be immediately readable by someone who didn't build it.
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ CRITICAL: Formulas Over Hardcodes + Step-by-Step Verification
|
||||
|
||||
**Formulas, not hardcodes:**
|
||||
- Every derived value (margin, multiple, statistic) MUST be an Excel formula referencing input cells — never a pre-computed number pasted in
|
||||
- When using Python/openpyxl to build the sheet: write `cell.value = "=E7/C7"` (formula string), NOT `cell.value = 0.687` (computed result)
|
||||
- The only hardcoded values should be raw input data (revenue, EBITDA, share price, etc.) — and every one of those gets a cell comment with its source
|
||||
- Why: the model must update automatically when an input changes. A hardcoded margin is a silent bug waiting to happen.
|
||||
|
||||
**Verify step-by-step with the user:**
|
||||
- After setting up the structure → show the user the header layout before filling data
|
||||
- After entering raw inputs → show the user the input block and confirm sources/periods before building formulas
|
||||
- After building operating metrics formulas → show the calculated margins and sanity-check with the user before moving to valuation
|
||||
- After building valuation multiples → show the multiples and confirm they look reasonable before adding statistics
|
||||
- Do NOT build the entire sheet end-to-end and then present it — catch errors early by confirming each section
|
||||
|
||||
---
|
||||
|
||||
## Section 1: Document Structure & Setup
|
||||
|
||||
### Header Block (Rows 1-3)
|
||||
```
|
||||
Row 1: [ANALYSIS TITLE] - COMPARABLE COMPANY ANALYSIS
|
||||
Row 2: [List of Companies with Tickers] • [Company 1 (TICK1)] • [Company 2 (TICK2)] • [Company 3 (TICK3)]
|
||||
Row 3: As of [Period] | All figures in [USD Millions/Billions] except per-share amounts and ratios
|
||||
```
|
||||
|
||||
**Why this matters:** Establishes context immediately. Anyone opening this file knows what they're looking at, when it was created, and how to interpret the numbers.
|
||||
|
||||
### Visual Convention Standards (OPTIONAL - User preferences and uploaded templates always override)
|
||||
|
||||
**IMPORTANT: These are suggested defaults only. Always prioritize:**
|
||||
1. User's explicit formatting preferences
|
||||
2. Formatting from any uploaded template files
|
||||
3. Company/team style guides
|
||||
4. These defaults (only if no other guidance provided)
|
||||
|
||||
**Suggested Font & Typography:**
|
||||
- **Font family**: Times New Roman (professional, readable, industry standard)
|
||||
- **Font size**: 11pt for data cells, 12pt for headers
|
||||
- **Bold text**: Section headers, company names, statistic labels
|
||||
|
||||
**Default Color & Shading — Professional Blue/Grey Palette (minimal is better):**
|
||||
- **Keep it restrained** — only blues and greys. Do NOT introduce greens, oranges, reds, or multiple accent colors. A clean comps sheet uses 3-4 colors total.
|
||||
- **Section headers** (e.g., "OPERATING STATISTICS & FINANCIAL METRICS"):
|
||||
- Dark blue background (`#1F4E79` or `#17365D` navy)
|
||||
- White bold text
|
||||
- Full row shading across all columns
|
||||
- **Column headers** (e.g., "Company", "Revenue", "Margin"):
|
||||
- Light blue background (`#D9E1F2` or similar pale blue)
|
||||
- Black bold text
|
||||
- Centered alignment
|
||||
- **Data rows**:
|
||||
- White background for company data
|
||||
- Black text for formulas; blue text for hardcoded inputs
|
||||
- **Statistics rows** (Maximum, 75th Percentile, etc.):
|
||||
- Light grey background (`#F2F2F2`)
|
||||
- Black text, left-aligned labels
|
||||
- **That's the whole palette**: dark blue + light blue + light grey + white. Nothing else unless the user's template says otherwise.
|
||||
|
||||
**Suggested Formatting Conventions:**
|
||||
- **Decimal precision**:
|
||||
- Percentages: 1 decimal (12.3%)
|
||||
- Multiples: 1 decimal (13.5x)
|
||||
- Dollar amounts: No decimals, thousands separator (69,632)
|
||||
- Margins shown as percentages: 1 decimal (68.7%)
|
||||
- **Borders**: No borders (clean, minimal appearance)
|
||||
- **Alignment**: All metrics center-aligned for clean, uniform appearance
|
||||
- **Cell dimensions**: All column widths should be uniform/even, all row heights should be consistent (creates clean, professional grid)
|
||||
|
||||
**Note:** If the user provides a template file or specifies different formatting, use that instead.
|
||||
|
||||
---
|
||||
|
||||
## Section 2: Operating Statistics & Financial Metrics
|
||||
|
||||
### Core Columns (Start with these)
|
||||
1. **Company** - Names with consistent formatting
|
||||
2. **Revenue** - Size metric (can be LTM, quarterly, or annual depending on context)
|
||||
3. **Revenue Growth** - Year-over-year percentage change
|
||||
4. **Gross Profit** - Revenue minus cost of goods sold
|
||||
5. **Gross Margin** - GP/Revenue (fundamental profitability)
|
||||
6. **EBITDA** - Earnings before interest, tax, depreciation, amortization
|
||||
7. **EBITDA Margin** - EBITDA/Revenue (operating efficiency)
|
||||
|
||||
### Optional Additions (Choose based on industry/purpose)
|
||||
- **Quarterly vs LTM** - Include both if seasonality matters
|
||||
- **Free Cash Flow** - For capital-intensive or SaaS businesses
|
||||
- **FCF Margin** - FCF/Revenue (cash generation efficiency)
|
||||
- **Net Income** - For mature, profitable companies
|
||||
- **Operating Income** - For businesses with varying D&A
|
||||
- **CapEx metrics** - For asset-heavy industries
|
||||
- **Rule of 40** - Specifically for SaaS (Growth % + Margin %)
|
||||
- **FCF Conversion** - For quality of earnings analysis (advanced)
|
||||
|
||||
### Formula Examples (Using Row 7 as example)
|
||||
```excel
|
||||
// Core ratios - these are always calculated
|
||||
Gross Margin (F7): =E7/C7
|
||||
EBITDA Margin (H7): =G7/C7
|
||||
|
||||
// Optional ratios - include if relevant
|
||||
FCF Margin: =[FCF]/[Revenue]
|
||||
Net Margin: =[Net Income]/[Revenue]
|
||||
Rule of 40: =[Growth %]+[FCF Margin %]
|
||||
```
|
||||
|
||||
**Golden Rule:** Every ratio should be [Something] / [Revenue] or [Something] / [Something from this sheet]. Keep it simple.
|
||||
|
||||
### Statistics Block (After company data)
|
||||
|
||||
**CRITICAL: Add statistics formulas for all comparable metrics (ratios, margins, growth rates, multiples).**
|
||||
|
||||
```
|
||||
[Leave one blank row for visual separation]
|
||||
- Maximum: =MAX(B7:B9)
|
||||
- 75th Percentile: =QUARTILE(B7:B9,3)
|
||||
- Median: =MEDIAN(B7:B9)
|
||||
- 25th Percentile: =QUARTILE(B7:B9,1)
|
||||
- Minimum: =MIN(B7:B9)
|
||||
```
|
||||
|
||||
**Columns that NEED statistics (comparable metrics):**
|
||||
- Revenue Growth %, Gross Margin %, EBITDA Margin %, EPS
|
||||
- EV/Revenue, EV/EBITDA, P/E, Dividend Yield %, Beta
|
||||
|
||||
**Columns that DON'T need statistics (size metrics):**
|
||||
- Revenue, EBITDA, Net Income (absolute size varies by company scale)
|
||||
- Market Cap, Enterprise Value (not comparable across different-sized companies)
|
||||
|
||||
**Note:** Add one blank row between company data and statistics rows for visual separation. Do NOT add a "SECTOR STATISTICS" or "VALUATION STATISTICS" header row.
|
||||
|
||||
**Why quartiles matter:** They show distribution, not just average. A 75th percentile multiple tells you what "premium" companies trade at.
|
||||
|
||||
---
|
||||
|
||||
## Section 3: Valuation Multiples & Investment Metrics
|
||||
|
||||
### Core Valuation Columns (Start with these)
|
||||
1. **Company** - Same order as operating section
|
||||
2. **Market Cap** - Current market valuation
|
||||
3. **Enterprise Value** - Market Cap ± Net Debt/Cash
|
||||
4. **EV/Revenue** - How much market pays per dollar of sales
|
||||
5. **EV/EBITDA** - How much market pays per dollar of earnings
|
||||
6. **P/E Ratio** - Price relative to net earnings
|
||||
|
||||
### Optional Valuation Metrics (Choose based on context)
|
||||
- **FCF Yield** - FCF/Market Cap (for cash-focused analysis)
|
||||
- **PEG Ratio** - P/E/Growth Rate (for growth companies)
|
||||
- **Price/Book** - Market value vs. book value (for asset-heavy businesses)
|
||||
- **ROE/ROA** - Return metrics (for profitability comparison)
|
||||
- **Revenue/EBITDA CAGR** - Historical growth rates (for trend analysis)
|
||||
- **Asset Turnover** - Revenue/Assets (for operational efficiency)
|
||||
- **Debt/Equity** - Leverage (for capital structure analysis)
|
||||
|
||||
**Key Principle:** Include 3-5 core multiples that matter for your industry. Don't include every possible metric just because you can.
|
||||
|
||||
### Formula Examples
|
||||
```excel
|
||||
// Core multiples - always include these
|
||||
EV/Revenue: =[Enterprise Value]/[LTM Revenue]
|
||||
EV/EBITDA: =[Enterprise Value]/[LTM EBITDA]
|
||||
P/E Ratio: =[Market Cap]/[Net Income]
|
||||
|
||||
// Optional multiples - include if data available
|
||||
FCF Yield: =[LTM FCF]/[Market Cap]
|
||||
PEG Ratio: =[P/E]/[Growth Rate %]
|
||||
```
|
||||
|
||||
### Cross-Reference Rule
|
||||
**CRITICAL:** Valuation multiples MUST reference the operating metrics section. Never input the same raw data twice. If revenue is in C7, then EV/Revenue formula should reference C7.
|
||||
|
||||
### Statistics Block
|
||||
Same structure as operating section: Max, 75th, Median, 25th, Min for every metric. Add one blank row for visual separation between company data and statistics. Do NOT add a "VALUATION STATISTICS" header row.
|
||||
|
||||
---
|
||||
|
||||
## Section 4: Notes & Methodology Documentation
|
||||
|
||||
### Required Components
|
||||
|
||||
**Data Sources & Quality:**
|
||||
- Where did the data come from? (S&P Kensho MCP, FactSet MCP, Daloopa MCP, Bloomberg, SEC filings)
|
||||
- What period does it cover? (Q4 2024, audited figures)
|
||||
- How was it verified? (Cross-checked against 10-K/10-Q)
|
||||
- Note: Prioritize MCP data sources (S&P Kensho, FactSet, Daloopa) if available for better accuracy and traceability
|
||||
|
||||
**Key Definitions:**
|
||||
- EBITDA calculation method (Gross Profit + D&A, or Operating Income + D&A)
|
||||
- Free Cash Flow formula (Operating CF - CapEx)
|
||||
- Special metrics explained (Rule of 40, FCF Conversion)
|
||||
- Time period definitions (LTM, CAGR calculation periods)
|
||||
|
||||
**Valuation Methodology:**
|
||||
- How was Enterprise Value calculated? (Market Cap + Net Debt)
|
||||
- What growth rates were used? (Historical CAGR, forward estimates)
|
||||
- Any adjustments made? (One-time items excluded, normalized margins)
|
||||
|
||||
**Analysis Framework:**
|
||||
- What's the investment thesis? (Cloud/SaaS efficiency)
|
||||
- What metrics matter most? (Cash generation, capital efficiency)
|
||||
- How should readers interpret the statistics? (Quartiles provide context)
|
||||
|
||||
---
|
||||
|
||||
## Section 5: Choosing the Right Metrics (Decision Framework)
|
||||
|
||||
### Start with "What question am I answering?"
|
||||
|
||||
**"Which company is undervalued?"**
|
||||
→ Focus on: EV/Revenue, EV/EBITDA, P/E, Market Cap
|
||||
→ Skip: Operational details, growth metrics
|
||||
|
||||
**"Which company is most efficient?"**
|
||||
→ Focus on: Gross Margin, EBITDA Margin, FCF Margin, Asset Turnover
|
||||
→ Skip: Size metrics, absolute dollar amounts
|
||||
|
||||
**"Which company is growing fastest?"**
|
||||
→ Focus on: Revenue Growth %, EBITDA CAGR, User/Customer Growth
|
||||
→ Skip: Margin metrics, leverage ratios
|
||||
|
||||
**"Which is the best cash generator?"**
|
||||
→ Focus on: FCF, FCF Margin, FCF Conversion, CapEx intensity
|
||||
→ Skip: EBITDA, P/E ratios
|
||||
|
||||
### Industry-Specific Metric Selection
|
||||
|
||||
**Software/SaaS:**
|
||||
Must have: Revenue Growth, Gross Margin, Rule of 40
|
||||
Optional: ARR, Net Dollar Retention, CAC Payback
|
||||
Skip: Asset Turnover, Inventory metrics
|
||||
|
||||
**Manufacturing/Industrials:**
|
||||
Must have: EBITDA Margin, Asset Turnover, CapEx/Revenue
|
||||
Optional: ROA, Inventory Turns, Backlog
|
||||
Skip: Rule of 40, SaaS metrics
|
||||
|
||||
**Financial Services:**
|
||||
Must have: ROE, ROA, Efficiency Ratio, P/E
|
||||
Optional: Net Interest Margin, Loan Loss Reserves
|
||||
Skip: Gross Margin, EBITDA (not meaningful for banks)
|
||||
|
||||
**Retail/E-commerce:**
|
||||
Must have: Revenue Growth, Gross Margin, Inventory Turnover
|
||||
Optional: Same-Store Sales, Customer Acquisition Cost
|
||||
Skip: Heavy R&D or CapEx metrics
|
||||
|
||||
### The "5-10 Rule"
|
||||
|
||||
**5 operating metrics** - Revenue, Growth, 2-3 margins/efficiency metrics
|
||||
**5 valuation metrics** - Market Cap, EV, 3 multiples
|
||||
**= 10 total columns** - Enough to tell the story, not so many you lose the thread
|
||||
|
||||
If you have more than 15 metrics, you're probably including noise. Edit ruthlessly.
|
||||
|
||||
---
|
||||
|
||||
## Section 6: Best Practices & Quality Checks
|
||||
|
||||
### Before You Start
|
||||
1. **Define the peer group** - Companies must be truly comparable (similar business model, scale, geography)
|
||||
2. **Choose the right period** - LTM smooths seasonality; quarterly shows trends
|
||||
3. **Standardize units upfront** - Millions vs. billions decision affects everything
|
||||
4. **Map data sources** - Know where each number comes from
|
||||
|
||||
### As You Build
|
||||
1. **Input all raw data first** - Complete the blue text before writing formulas
|
||||
2. **Add cell comments to ALL hard-coded inputs** - Right-click cell → Insert Comment → Document source OR assumption
|
||||
|
||||
**For sourced data, cite exactly where it came from:**
|
||||
- Example: "Bloomberg Terminal - MSFT Equity DES, accessed 2024-10-02"
|
||||
- Example: "Q4 2024 10-K filing, page 42, line item 'Total Revenue'"
|
||||
- Example: "FactSet consensus estimate as of 2024-10-02"
|
||||
- **Include hyperlinks when possible**: Right-click cell → Link → paste URL to SEC filing, data source, or report
|
||||
|
||||
**For assumptions, explain the reasoning:**
|
||||
- Example: "Assumed 15% EBITDA margin based on peer median, company does not disclose"
|
||||
- Example: "Estimated Enterprise Value as Market Cap + $50M net debt (from Q3 balance sheet, Q4 not yet available)"
|
||||
- Example: "Forward P/E based on street consensus EPS of $3.45 (average of 12 analyst estimates)"
|
||||
|
||||
**Why this matters**: Enables audit trails, data verification, assumption transparency, and future updates
|
||||
3. **Build formulas row by row** - Test each calculation before moving on
|
||||
4. **Use absolute references for headers** - $C$6 locks the header row
|
||||
5. **Format consistently** - Percentages as percentages, not decimals
|
||||
6. **Add conditional formatting** - Highlight outliers automatically
|
||||
|
||||
### Sanity Checks
|
||||
- **Margin test**: Gross margin > EBITDA margin > Net margin (always true by definition)
|
||||
- **Multiple reasonableness**:
|
||||
- EV/Revenue: typically 0.5-20x (varies widely by industry)
|
||||
- EV/EBITDA: typically 8-25x (fairly consistent across industries)
|
||||
- P/E: typically 10-50x (depends on growth rate)
|
||||
- **Growth-multiple correlation**: Higher growth usually means higher multiples
|
||||
- **Size-efficiency trade-off**: Larger companies often have better margins (scale benefits)
|
||||
|
||||
### Common Mistakes to Avoid
|
||||
❌ Mixing market cap and enterprise value in formulas
|
||||
❌ Using different time periods for numerator and denominator (LTM vs quarterly)
|
||||
❌ Hardcoding numbers into formulas instead of cell references
|
||||
❌ **Hard-coded inputs without cell comments citing the source OR explaining the assumption**
|
||||
❌ Missing hyperlinks to SEC filings or data sources when available
|
||||
❌ Including too many metrics without clear purpose
|
||||
❌ Including non-comparable companies (different business models)
|
||||
❌ Using outdated data without disclosure
|
||||
❌ Calculating averages of percentages incorrectly (should be median)
|
||||
|
||||
---
|
||||
|
||||
## Section 6: Advanced Features
|
||||
|
||||
### Dynamic Headers
|
||||
For columns showing calculations, use clear unit labels:
|
||||
```
|
||||
Revenue Growth (YoY) % | EBITDA Margin | FCF Margin | Rule of 40
|
||||
```
|
||||
|
||||
### Quartile Analysis Benefits
|
||||
Instead of just mean/median, quartiles show:
|
||||
- **75th percentile** = "Premium" companies trade here
|
||||
- **Median** = Typical market valuation
|
||||
- **25th percentile** = "Discount" territory
|
||||
|
||||
This helps answer: "Is our target company trading rich or cheap vs. peers?"
|
||||
|
||||
### Industry-Specific Modifications
|
||||
|
||||
**Software/SaaS:**
|
||||
- Add: ARR, Net Dollar Retention, CAC Payback Period
|
||||
- Emphasize: Rule of 40, FCF margins, gross margins >70%
|
||||
|
||||
**Healthcare:**
|
||||
- Add: R&D/Revenue, Pipeline value, Regulatory status
|
||||
- Emphasize: EBITDA margins, growth rates, reimbursement risk
|
||||
|
||||
**Industrials:**
|
||||
- Add: Backlog, Order book trends, Geographic mix
|
||||
- Emphasize: ROIC, asset turnover, cyclical adjustments
|
||||
|
||||
**Consumer:**
|
||||
- Add: Same-store sales, Customer acquisition cost, Brand value
|
||||
- Emphasize: Revenue growth, gross margins, inventory turns
|
||||
|
||||
---
|
||||
|
||||
## Section 7: Workflow & Practical Tips
|
||||
|
||||
### Step-by-Step Process
|
||||
1. **Set up structure** (30 minutes)
|
||||
- Create all headers
|
||||
- Format cells (blue for inputs, black for formulas)
|
||||
- Lock in units and date references
|
||||
|
||||
2. **Gather data** (60-90 minutes)
|
||||
- Pull from primary sources (S&P Kensho MCP, FactSet MCP, Daloopa MCP if available; otherwise Bloomberg, SEC)
|
||||
- Input all raw numbers in blue
|
||||
- Document sources in notes section
|
||||
|
||||
3. **Build formulas** (30 minutes)
|
||||
- Start with simple ratios (margins)
|
||||
- Progress to multiples (EV/Revenue)
|
||||
- Add cross-checks (do margins make sense?)
|
||||
|
||||
4. **Add statistics** (15 minutes)
|
||||
- Copy formula structure for all columns
|
||||
- Verify ranges are correct (B7:B9, not B7:B10)
|
||||
- Check quartile logic
|
||||
|
||||
5. **Quality control** (30 minutes)
|
||||
- Run sanity checks
|
||||
- Verify formula references
|
||||
- Check for #DIV/0! or #REF! errors
|
||||
- Compare against known benchmarks
|
||||
|
||||
6. **Documentation** (15 minutes)
|
||||
- Complete notes section
|
||||
- Add data sources
|
||||
- Define methodologies
|
||||
- Date-stamp the analysis
|
||||
|
||||
### Pro Tips
|
||||
- **Save templates**: Build once, reuse forever
|
||||
- **Color-code outliers**: Conditional formatting for values >2 standard deviations
|
||||
- **Link to source files**: Hyperlink to Bloomberg screenshots or SEC filings
|
||||
- **Version control**: Save as "Comps_v1_2024-12-15" with clear dating
|
||||
- **Collaborative reviews**: Have someone else check your formulas
|
||||
|
||||
### Excel Formatting Checklist (Optional - adapt to user preferences)
|
||||
- [ ] Font set to user's preferred style (default: Times New Roman, 11pt data, 12pt headers)
|
||||
- [ ] Section headers formatted per user's template (default: dark blue #17365D with white bold text)
|
||||
- [ ] Column headers formatted per user's template (default: light blue/gray #D9E2F3 with black bold text)
|
||||
- [ ] Statistics rows formatted per user's template (default: light gray #F2F2F2)
|
||||
- [ ] No borders applied (clean, minimal appearance)
|
||||
- [ ] **Column widths set to uniform/even width** (creates clean, professional appearance)
|
||||
- [ ] **Row heights set to consistent height** (typically 20-25pt for data rows)
|
||||
- [ ] Numbers formatted with proper decimal precision and thousands separators
|
||||
- [ ] **All metrics center-aligned** for clean, uniform appearance
|
||||
- [ ] **One blank row for separation between company data and statistics rows**
|
||||
- [ ] **No separate "SECTOR STATISTICS" or "VALUATION STATISTICS" header rows**
|
||||
- [ ] **Every hard-coded input cell has a comment with either: (1) exact data source, OR (2) assumption explanation**
|
||||
- [ ] **Hyperlinks added to cells where applicable** (SEC filings, data provider pages, reports)
|
||||
|
||||
---
|
||||
|
||||
## Section 8: Example Template Layout
|
||||
|
||||
**Simple Version (Start here):**
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ TECHNOLOGY - COMPARABLE COMPANY ANALYSIS │
|
||||
│ Microsoft • Alphabet • Amazon │
|
||||
│ As of Q4 2024 | All figures in USD Millions │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ OPERATING METRICS │
|
||||
├──────────┬─────────┬─────────┬──────────┬──────────────────┤
|
||||
│ Company │ Revenue │ Growth │ Gross │ EBITDA │ EBITDA │
|
||||
│ │ (LTM) │ (YoY) │ Margin │ (LTM) │ Margin │
|
||||
├──────────┼─────────┼─────────┼──────────┼─────────┼────────┤
|
||||
│ MSFT │ 261,400 │ 12.3% │ 68.7% │ 205,100 │ 78.4% │
|
||||
│ GOOGL │ 349,800 │ 11.8% │ 57.9% │ 239,300 │ 68.4% │
|
||||
│ AMZN │ 638,100 │ 10.5% │ 47.3% │ 152,600 │ 23.9% │
|
||||
│ │ │ │ │ │ │ [blank row]
|
||||
│ Median │ =MEDIAN │ =MEDIAN │ =MEDIAN │ =MEDIAN │=MEDIAN │
|
||||
│ 75th % │ =QUART │ =QUART │ =QUART │ =QUART │=QUART │
|
||||
│ 25th % │ =QUART │ =QUART │ =QUART │ =QUART │=QUART │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ VALUATION MULTIPLES │
|
||||
├──────────┬──────────┬──────────┬──────────┬────────────────┤
|
||||
│ Company │ Mkt Cap │ EV │ EV/Rev │ EV/EBITDA │ P/E│
|
||||
├──────────┼──────────┼──────────┼──────────┼───────────┼────┤
|
||||
│ MSFT │3,550,000 │3,530,000 │ 13.5x │ 17.2x │36.0│
|
||||
│ GOOGL │2,030,000 │1,960,000 │ 5.6x │ 8.2x │24.5│
|
||||
│ AMZN │2,226,000 │2,320,000 │ 3.6x │ 15.2x │58.3│
|
||||
│ │ │ │ │ │ │ [blank row]
|
||||
│ Median │ =MEDIAN │ =MEDIAN │ =MEDIAN │ =MEDIAN │=MED│
|
||||
│ 75th % │ =QUART │ =QUART │ =QUART │ =QUART │=QRT│
|
||||
│ 25th % │ =QUART │ =QUART │ =QUART │ =QUART │=QRT│
|
||||
└──────────┴──────────┴──────────┴──────────┴───────────┴────┘
|
||||
```
|
||||
|
||||
**Add complexity only when needed:**
|
||||
- Include quarterly AND LTM if seasonality matters
|
||||
- Add FCF metrics if cash generation is key story
|
||||
- Include industry-specific metrics (Rule of 40 for SaaS, etc.)
|
||||
- Add more statistics rows if you have >5 companies
|
||||
|
||||
---
|
||||
|
||||
## Section 9: Industry-Specific Additions (Optional)
|
||||
|
||||
Only add these if they're critical to your analysis. Most comps work fine with just core metrics.
|
||||
|
||||
**Software/SaaS:**
|
||||
Add if relevant: ARR, Net Dollar Retention, Rule of 40
|
||||
|
||||
**Financial Services:**
|
||||
Add if relevant: ROE, Net Interest Margin, Efficiency Ratio
|
||||
|
||||
**E-commerce:**
|
||||
Add if relevant: GMV, Take Rate, Active Buyers
|
||||
|
||||
**Healthcare:**
|
||||
Add if relevant: R&D/Revenue, Pipeline Value, Patent Timeline
|
||||
|
||||
**Manufacturing:**
|
||||
Add if relevant: Asset Turnover, Inventory Turns, Backlog
|
||||
|
||||
---
|
||||
|
||||
## Section 10: Red Flags & Warning Signs
|
||||
|
||||
### Data Quality Issues
|
||||
🚩 Inconsistent time periods (mixing quarterly and annual)
|
||||
🚩 Missing data without explanation
|
||||
🚩 Significant differences between data sources (>10% variance)
|
||||
|
||||
### Valuation Red Flags
|
||||
🚩 Negative EBITDA companies being valued on EBITDA multiples (use revenue multiples instead)
|
||||
🚩 P/E ratios >100x without hypergrowth story
|
||||
🚩 Margins that don't make sense for the industry
|
||||
|
||||
### Comparability Issues
|
||||
🚩 Different fiscal year ends (causes timing problems)
|
||||
🚩ixing pure-play and conglomerates
|
||||
🚩 Materially different business models labeled as "comps"
|
||||
|
||||
**When in doubt, exclude the company.** Better to have 3 perfect comps than 6 questionable ones.
|
||||
|
||||
---
|
||||
|
||||
## Section 11: Formulas Reference Guide
|
||||
|
||||
### Essential Excel Formulas
|
||||
```excel
|
||||
// Statistical Functions
|
||||
=AVERAGE(range) // Simple mean
|
||||
=MEDIAN(range) // Middle value
|
||||
=QUARTILE(range, 1) // 25th percentile
|
||||
=QUARTILE(range, 3) // 75th percentile
|
||||
=MAX(range) // Maximum value
|
||||
=MIN(range) // Minimum value
|
||||
=STDEV.P(range) // Standard deviation
|
||||
|
||||
// Financial Calculations
|
||||
=B7/C7 // Simple ratio (Margin)
|
||||
=SUM(B7:B9)/3 // Average of multiple companies
|
||||
=IF(B7>0, C7/B7, "N/A") // Conditional calculation
|
||||
=IFERROR(C7/D7, 0) // Handle divide by zero
|
||||
|
||||
// Cross-Sheet References
|
||||
='Sheet1'!B7 // Reference another sheet
|
||||
=VLOOKUP(A7, Table1, 2) // Lookup from data table
|
||||
=INDEX(MATCH()) // Advanced lookup
|
||||
|
||||
// Formatting
|
||||
=TEXT(B7, "0.0%") // Format as percentage
|
||||
=TEXT(C7, "#,##0") // Thousands separator
|
||||
```
|
||||
|
||||
### Common Ratio Formulas
|
||||
```excel
|
||||
Gross Margin = Gross Profit / Revenue
|
||||
EBITDA Margin = EBITDA / Revenue
|
||||
FCF Margin = Free Cash Flow / Revenue
|
||||
FCF Conversion = FCF / Operating Cash Flow
|
||||
ROE = Net Income / Shareholders' Equity
|
||||
ROA = Net Income / Total Assets
|
||||
Asset Turnover = Revenue / Total Assets
|
||||
Debt/Equity = Total Debt / Shareholders' Equity
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Principles Summary
|
||||
|
||||
1. **Structure drives insight** - Right headers force right thinking
|
||||
2. **Less is more** - 5-10 metrics that matter beat 20 that don't
|
||||
3. **Choose metrics for your question** - Valuation analysis ≠ efficiency analysis
|
||||
4. **Statistics show patterns** - Median/quartiles reveal more than average
|
||||
5. **Transparency beats complexity** - Simple formulas everyone understands
|
||||
6. **Comparability is king** - Better to exclude than force a bad comp
|
||||
7. **Document your choices** - Explain which metrics and why in notes section
|
||||
|
||||
---
|
||||
|
||||
## Output Checklist
|
||||
|
||||
Before delivering a comp analysis, verify:
|
||||
- [ ] All companies are truly comparable
|
||||
- [ ] Data is from consistent time periods
|
||||
- [ ] Units are clearly labeled (millions/billions)
|
||||
- [ ] Formulas reference cells, not hardcoded values
|
||||
- [ ] **All hard-coded input cells have comments with either: (1) exact data source with citation, OR (2) clear assumption with explanation**
|
||||
- [ ] **Hyperlinks added where relevant** (SEC EDGAR filings, Bloomberg pages, research reports)
|
||||
- [ ] Statistics include at least 5 metrics (Max, 75th, Med, 25th, Min)
|
||||
- [ ] Notes section documents sources and methodology
|
||||
- [ ] Visual formatting follows conventions (blue = input, black = formula)
|
||||
- [ ] Sanity checks pass (margins logical, multiples reasonable)
|
||||
- [ ] Date stamp is current ("As of [Date]")
|
||||
- [ ] Formula auditing shows no errors (#DIV/0!, #REF!, #N/A)
|
||||
|
||||
---
|
||||
|
||||
## Continuous Improvement
|
||||
|
||||
After completing a comp analysis, ask:
|
||||
1. Did the statistics reveal unexpected insights?
|
||||
2. Were there any data gaps that limited analysis?
|
||||
3. Did stakeholders ask for metrics you didn't include?
|
||||
4. How long did it take vs. how long should it take?
|
||||
5. What would make this more useful next time?
|
||||
|
||||
The best comp analyses evolve with each iteration. Save templates, learn from feedback, and refine the structure based on what decision-makers actually use.
|
||||
|
||||
|
||||
## Data sources — MCP first, web fallback
|
||||
|
||||
Many passages below say "use the S&P Kensho MCP / Daloopa MCP / FactSet MCP". Those are commercial financial-data MCPs from the original Cowork plugin context. In Hermes:
|
||||
|
||||
- **If you have any structured financial-data MCP configured** (Hermes supports MCP — see `native-mcp` skill), prefer it for point-in-time comps, precedent transactions, and filings.
|
||||
- **Otherwise**, fall back to:
|
||||
- `web_search` / `web_extract` against SEC EDGAR (`https://www.sec.gov/cgi-bin/browse-edgar`) for US filings
|
||||
- Company IR pages for press releases, earnings decks
|
||||
- `browser_navigate` for interactive data portals
|
||||
- User-provided data (explicitly ask when the context doesn't have it)
|
||||
- **Never fabricate**. If a multiple, precedent, or filing number can't be sourced, flag the cell as `[UNSOURCED]` and surface it to the user.
|
||||
|
||||
## Attribution
|
||||
|
||||
This skill is adapted from Anthropic's Claude for Financial Services plugin suite (Apache-2.0). The Office-JS / Cowork live-Excel paths have been removed; this version targets headless openpyxl via the `excel-author` skill's conventions. Original: https://github.com/anthropics/financial-services
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,40 @@
|
||||
# DCF Model Troubleshooting Guide
|
||||
|
||||
**When to read this file:** If recalc.py shows errors OR valuation results seem unreasonable OR case selector not working properly.
|
||||
|
||||
## Model Returns Error Values
|
||||
|
||||
### #REF! Errors
|
||||
- Usually caused by formulas referencing wrong rows after headers were inserted
|
||||
- Solution: Rebuild with correct row references, or start over following layout planning
|
||||
- Prevention: Define all row positions BEFORE writing formulas
|
||||
|
||||
### #DIV/0! Errors
|
||||
- Division by zero or empty cells
|
||||
- Solution: Add IF statements to handle zeros: `=IF([Divisor]=0,0,[Numerator]/[Divisor])`
|
||||
|
||||
### #VALUE! Errors
|
||||
- Wrong data type in calculation (text instead of number)
|
||||
- Solution: Verify all inputs are formatted as numbers
|
||||
|
||||
## Valuation Seems Unreasonable
|
||||
|
||||
### Implied price far too high
|
||||
- Check terminal value isn't >80% of EV
|
||||
- Verify terminal growth < WACC
|
||||
- Review if growth assumptions are realistic
|
||||
- Consider if margins are too optimistic
|
||||
|
||||
### Implied price far too low
|
||||
- Verify net debt vs net cash is correct
|
||||
- Check if WACC is too high
|
||||
- Review if projections are too conservative
|
||||
- Consider if terminal growth is too low
|
||||
|
||||
## Case Selector Not Working
|
||||
|
||||
### Consolidation column not updating when switching scenarios
|
||||
- Verify case selector cell contains 1, 2, or 3
|
||||
- Check INDEX/OFFSET formulas reference correct row range and selector cell
|
||||
- Ensure absolute references ($B$6) are used for selector
|
||||
- Test by manually changing the selector cell and verifying projection values update
|
||||
@@ -0,0 +1,7 @@
|
||||
# DCF Model Builder - Python Dependencies
|
||||
|
||||
# Excel file handling
|
||||
openpyxl>=3.0.0
|
||||
|
||||
# HTTP requests
|
||||
requests>=2.28.0
|
||||
+291
@@ -0,0 +1,291 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
DCF Model Validation Script
|
||||
Validates Excel DCF models for formula errors and common DCF mistakes
|
||||
"""
|
||||
|
||||
import sys
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class DCFModelValidator:
|
||||
"""Validates DCF models for errors and quality issues"""
|
||||
|
||||
def __init__(self, excel_path: str):
|
||||
try:
|
||||
import openpyxl
|
||||
except ImportError:
|
||||
raise ImportError("openpyxl not installed. Run: pip install openpyxl")
|
||||
|
||||
self.excel_path = excel_path
|
||||
self.openpyxl = openpyxl
|
||||
|
||||
if not Path(excel_path).exists():
|
||||
raise FileNotFoundError(f"File not found: {excel_path}")
|
||||
|
||||
self.workbook_formulas = openpyxl.load_workbook(excel_path, data_only=False)
|
||||
self.workbook_values = openpyxl.load_workbook(excel_path, data_only=True)
|
||||
self.errors = []
|
||||
self.warnings = []
|
||||
self.info = []
|
||||
|
||||
def validate_all(self) -> dict:
|
||||
"""
|
||||
Run all validation checks
|
||||
|
||||
Returns:
|
||||
Dict with validation results
|
||||
"""
|
||||
from datetime import datetime
|
||||
|
||||
self.check_sheet_structure()
|
||||
self.check_formula_errors()
|
||||
self.check_dcf_logic()
|
||||
|
||||
results = {
|
||||
'file': self.excel_path,
|
||||
'validation_date': datetime.now().isoformat(),
|
||||
'status': 'PASS' if len(self.errors) == 0 else 'FAIL',
|
||||
'error_count': len(self.errors),
|
||||
'warning_count': len(self.warnings),
|
||||
'errors': self.errors,
|
||||
'warnings': self.warnings,
|
||||
'info': self.info
|
||||
}
|
||||
|
||||
return results
|
||||
|
||||
def check_sheet_structure(self):
|
||||
"""Verify required sheets exist"""
|
||||
required_sheets = ['DCF', 'WACC', 'Sensitivity']
|
||||
sheet_names = self.workbook_values.sheetnames
|
||||
|
||||
for sheet in required_sheets:
|
||||
if sheet not in sheet_names:
|
||||
self.warnings.append(f"Recommended sheet missing: {sheet}")
|
||||
else:
|
||||
self.info.append(f"Found sheet: {sheet}")
|
||||
|
||||
def check_formula_errors(self):
|
||||
"""Check for Excel formula errors in all sheets"""
|
||||
excel_errors = ['#VALUE!', '#DIV/0!', '#REF!', '#NAME?', '#NULL!', '#NUM!', '#N/A']
|
||||
error_details = {err: [] for err in excel_errors}
|
||||
total_errors = 0
|
||||
total_formulas = 0
|
||||
|
||||
for sheet_name in self.workbook_values.sheetnames:
|
||||
ws_values = self.workbook_values[sheet_name]
|
||||
ws_formulas = self.workbook_formulas[sheet_name]
|
||||
|
||||
for row in ws_values.iter_rows():
|
||||
for cell in row:
|
||||
formula_cell = ws_formulas[cell.coordinate]
|
||||
|
||||
# Count formulas
|
||||
if formula_cell.value and isinstance(formula_cell.value, str) and formula_cell.value.startswith('='):
|
||||
total_formulas += 1
|
||||
|
||||
# Check for errors
|
||||
if cell.value is not None and isinstance(cell.value, str):
|
||||
for err in excel_errors:
|
||||
if err in cell.value:
|
||||
location = f"{sheet_name}!{cell.coordinate}"
|
||||
error_details[err].append(location)
|
||||
total_errors += 1
|
||||
self.errors.append(f"{err} at {location}")
|
||||
break
|
||||
|
||||
# Add summary info
|
||||
self.info.append(f"Total formulas: {total_formulas}")
|
||||
if total_errors == 0:
|
||||
self.info.append("✓ No formula errors found")
|
||||
else:
|
||||
self.errors.append(f"Total formula errors: {total_errors}")
|
||||
|
||||
return error_details, total_errors
|
||||
|
||||
def check_dcf_logic(self):
|
||||
"""Validate DCF-specific logic and calculations"""
|
||||
self._check_terminal_growth_vs_wacc()
|
||||
self._check_wacc_range()
|
||||
self._check_terminal_value_proportion()
|
||||
|
||||
def _check_terminal_growth_vs_wacc(self):
|
||||
"""Critical check: Terminal growth must be less than WACC"""
|
||||
try:
|
||||
dcf_sheet = self.workbook_values['DCF']
|
||||
|
||||
terminal_growth = None
|
||||
wacc = None
|
||||
|
||||
# Search for terminal growth and WACC values
|
||||
for row in dcf_sheet.iter_rows(max_row=100, max_col=20):
|
||||
for cell in row:
|
||||
if cell.value and isinstance(cell.value, str):
|
||||
cell_str = cell.value.lower()
|
||||
if 'terminal' in cell_str and 'growth' in cell_str:
|
||||
# Look for value in adjacent cells
|
||||
for offset in range(1, 5):
|
||||
adjacent = dcf_sheet.cell(cell.row, cell.column + offset).value
|
||||
if isinstance(adjacent, (int, float)) and 0 < adjacent < 1:
|
||||
terminal_growth = adjacent
|
||||
break
|
||||
if 'wacc' in cell_str and wacc is None:
|
||||
for offset in range(1, 5):
|
||||
adjacent = dcf_sheet.cell(cell.row, cell.column + offset).value
|
||||
if isinstance(adjacent, (int, float)) and 0 < adjacent < 1:
|
||||
wacc = adjacent
|
||||
break
|
||||
|
||||
if terminal_growth is not None and wacc is not None:
|
||||
if terminal_growth >= wacc:
|
||||
self.errors.append(
|
||||
f"CRITICAL: Terminal growth ({terminal_growth:.2%}) >= WACC ({wacc:.2%}). "
|
||||
"This creates infinite value and is mathematically invalid."
|
||||
)
|
||||
else:
|
||||
self.info.append(
|
||||
f"✓ Terminal growth ({terminal_growth:.2%}) < WACC ({wacc:.2%})"
|
||||
)
|
||||
else:
|
||||
self.warnings.append("Could not locate terminal growth and WACC values")
|
||||
|
||||
except KeyError:
|
||||
self.warnings.append("DCF sheet not found")
|
||||
except Exception as e:
|
||||
self.warnings.append(f"Could not validate terminal growth vs WACC: {str(e)}")
|
||||
|
||||
def _check_wacc_range(self):
|
||||
"""Check if WACC is in reasonable range"""
|
||||
try:
|
||||
wacc_sheet = self.workbook_values.get('WACC') or self.workbook_values['DCF']
|
||||
wacc = None
|
||||
|
||||
for row in wacc_sheet.iter_rows(max_row=100, max_col=20):
|
||||
for cell in row:
|
||||
if cell.value and isinstance(cell.value, str):
|
||||
if 'wacc' in cell.value.lower():
|
||||
for offset in range(1, 5):
|
||||
adjacent = wacc_sheet.cell(cell.row, cell.column + offset).value
|
||||
if isinstance(adjacent, (int, float)) and 0 < adjacent < 1:
|
||||
wacc = adjacent
|
||||
break
|
||||
|
||||
if wacc is not None:
|
||||
if wacc < 0.05 or wacc > 0.20:
|
||||
self.warnings.append(
|
||||
f"WACC ({wacc:.2%}) is outside typical range (5%-20%). Verify calculation."
|
||||
)
|
||||
else:
|
||||
self.info.append(f"✓ WACC ({wacc:.2%}) in reasonable range")
|
||||
else:
|
||||
self.warnings.append("Could not locate WACC value")
|
||||
|
||||
except Exception as e:
|
||||
self.warnings.append(f"Could not validate WACC range: {str(e)}")
|
||||
|
||||
def _check_terminal_value_proportion(self):
|
||||
"""Check if terminal value is reasonable proportion of enterprise value"""
|
||||
try:
|
||||
dcf_sheet = self.workbook_values['DCF']
|
||||
|
||||
terminal_value = None
|
||||
enterprise_value = None
|
||||
|
||||
for row in dcf_sheet.iter_rows(max_row=200, max_col=20):
|
||||
for cell in row:
|
||||
if cell.value and isinstance(cell.value, str):
|
||||
cell_str = cell.value.lower()
|
||||
if 'terminal' in cell_str and 'value' in cell_str and 'pv' in cell_str:
|
||||
for offset in range(1, 5):
|
||||
adjacent = dcf_sheet.cell(cell.row, cell.column + offset).value
|
||||
if isinstance(adjacent, (int, float)) and adjacent > 0:
|
||||
terminal_value = adjacent
|
||||
break
|
||||
if 'enterprise' in cell_str and 'value' in cell_str:
|
||||
for offset in range(1, 5):
|
||||
adjacent = dcf_sheet.cell(cell.row, cell.column + offset).value
|
||||
if isinstance(adjacent, (int, float)) and adjacent > 0:
|
||||
enterprise_value = adjacent
|
||||
break
|
||||
|
||||
if terminal_value is not None and enterprise_value is not None and enterprise_value > 0:
|
||||
proportion = terminal_value / enterprise_value
|
||||
if proportion > 0.80:
|
||||
self.warnings.append(
|
||||
f"Terminal value is {proportion:.1%} of EV (typically should be 50-70%). "
|
||||
"Model may be over-reliant on terminal assumptions."
|
||||
)
|
||||
elif proportion < 0.40:
|
||||
self.warnings.append(
|
||||
f"Terminal value is {proportion:.1%} of EV (typically should be 50-70%). "
|
||||
"Check if terminal assumptions are too conservative."
|
||||
)
|
||||
else:
|
||||
self.info.append(f"✓ Terminal value is {proportion:.1%} of EV")
|
||||
else:
|
||||
self.warnings.append("Could not locate terminal value and enterprise value")
|
||||
|
||||
except Exception as e:
|
||||
self.warnings.append(f"Could not validate terminal value proportion: {str(e)}")
|
||||
|
||||
|
||||
|
||||
def validate_dcf_model(excel_path: str) -> dict:
|
||||
"""
|
||||
Validate a DCF model Excel file
|
||||
|
||||
Args:
|
||||
excel_path: Path to Excel DCF model
|
||||
|
||||
Returns:
|
||||
Dict with validation results
|
||||
"""
|
||||
validator = DCFModelValidator(excel_path)
|
||||
return validator.validate_all()
|
||||
|
||||
|
||||
def main():
|
||||
"""Command-line interface"""
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: python validate_dcf.py <excel_file> [output.json]")
|
||||
print("\nValidates DCF model for:")
|
||||
print(" - Formula errors (#REF!, #DIV/0!, etc.)")
|
||||
print(" - Terminal growth < WACC (critical)")
|
||||
print(" - WACC in reasonable range (5-20%)")
|
||||
print(" - Terminal value proportion of EV (40-80%)")
|
||||
print("\nReturns JSON with errors, warnings, and info")
|
||||
print("\nExample: python validate_dcf.py model.xlsx")
|
||||
print("Example: python validate_dcf.py model.xlsx results.json")
|
||||
sys.exit(1)
|
||||
|
||||
excel_file = sys.argv[1]
|
||||
output_file = sys.argv[2] if len(sys.argv) > 2 else None
|
||||
|
||||
try:
|
||||
results = validate_dcf_model(excel_file)
|
||||
|
||||
# Print results
|
||||
print(json.dumps(results, indent=2))
|
||||
|
||||
# Save to file if requested
|
||||
if output_file:
|
||||
with open(output_file, 'w') as f:
|
||||
json.dump(results, f, indent=2)
|
||||
|
||||
# Exit with error code if validation failed
|
||||
sys.exit(0 if results['status'] == 'PASS' else 1)
|
||||
|
||||
except Exception as e:
|
||||
error_result = {
|
||||
'file': excel_file,
|
||||
'status': 'ERROR',
|
||||
'error': str(e)
|
||||
}
|
||||
print(json.dumps(error_result, indent=2))
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,244 @@
|
||||
---
|
||||
name: excel-author
|
||||
description: Build auditable Excel workbooks headless with openpyxl — blue/black/green cell conventions, formulas over hardcodes, named ranges, balance checks, sensitivity tables. Use for financial models, audit outputs, reconciliations.
|
||||
version: 1.0.0
|
||||
author: Anthropic (adapted by Nous Research)
|
||||
license: Apache-2.0
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [excel, openpyxl, finance, spreadsheet, modeling]
|
||||
related_skills: [pptx-author, dcf-model, comps-analysis, lbo-model, 3-statement-model]
|
||||
---
|
||||
|
||||
# excel-author
|
||||
|
||||
Produce an .xlsx file on disk using `openpyxl`. Follow the banker-grade conventions below so the model is auditable, flexible, and reviewable by someone other than the person who built it.
|
||||
|
||||
Adapted from Anthropic's `xlsx-author` and `audit-xls` skills in the [anthropics/financial-services](https://github.com/anthropics/financial-services) repo. The MCP / Office-JS / Cowork-specific branches of the originals are dropped — this skill assumes headless Python.
|
||||
|
||||
## Output contract
|
||||
|
||||
- Write to `./out/<name>.xlsx`. Create `./out/` if it does not exist.
|
||||
- Return the relative path in your final message so downstream tools can pick it up.
|
||||
- One logical model per file. Do not append to an existing workbook unless explicitly asked.
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
pip install "openpyxl>=3.0"
|
||||
```
|
||||
|
||||
## Core conventions (non-negotiable)
|
||||
|
||||
### Blue / black / green cell color
|
||||
- **Blue** (`Font(color="0000FF")`) — hardcoded input a human entered. Revenue drivers, WACC inputs, terminal growth, market data.
|
||||
- **Black** (default) — formula. Every derived cell is a live Excel formula.
|
||||
- **Green** (`Font(color="006100")`) — link to another sheet or external file.
|
||||
|
||||
A reviewer can then scan the sheet and immediately see what's an assumption vs. what's computed.
|
||||
|
||||
### Formulas over hardcodes
|
||||
Every calculation cell MUST be a formula string, never a number computed in Python and pasted as a value.
|
||||
|
||||
```python
|
||||
# WRONG — silent bug waiting to happen
|
||||
ws["D20"] = revenue_prior_year * (1 + growth)
|
||||
|
||||
# CORRECT — flexes when the user changes the assumption
|
||||
ws["D20"] = "=D19*(1+$B$8)"
|
||||
```
|
||||
|
||||
The only hardcoded numbers permitted:
|
||||
1. Raw historical inputs (actual revenues, reported EBITDA, etc.)
|
||||
2. Assumption drivers the user is meant to flex (growth rates, WACC inputs, terminal g)
|
||||
3. Current market data (share price, debt balance) — with a cell comment documenting source + date
|
||||
|
||||
If you catch yourself computing a value in Python and writing the result, stop.
|
||||
|
||||
### Named ranges for cross-sheet references
|
||||
Use named ranges for any figure referenced from another sheet, a deck, or a memo.
|
||||
|
||||
```python
|
||||
from openpyxl.workbook.defined_name import DefinedName
|
||||
wb.defined_names["WACC"] = DefinedName("WACC", attr_text="Inputs!$C$8")
|
||||
# then elsewhere:
|
||||
calc["D30"] = "=D29/WACC"
|
||||
```
|
||||
|
||||
### Balance checks tab
|
||||
Include a `Checks` tab that ties everything and surfaces TRUE/FALSE:
|
||||
- Balance sheet balances (assets = liabilities + equity)
|
||||
- Cash flow ties to period-over-period cash change on the BS
|
||||
- Sum-of-parts ties to consolidated totals
|
||||
- No rogue hardcodes inside calc ranges
|
||||
|
||||
Example:
|
||||
```python
|
||||
checks = wb.create_sheet("Checks")
|
||||
checks["A2"] = "BS balances"
|
||||
checks["B2"] = "=IS!D20-IS!D21-IS!D22"
|
||||
checks["C2"] = "=ABS(B2)<0.01" # TRUE/FALSE
|
||||
```
|
||||
|
||||
### Cell comments on every hardcoded input
|
||||
Add the comment AS you create the cell, not later.
|
||||
|
||||
```python
|
||||
from openpyxl.comments import Comment
|
||||
ws["C2"] = 1_250_000_000
|
||||
ws["C2"].font = Font(color="0000FF")
|
||||
ws["C2"].comment = Comment("Source: 10-K FY2024, p.47, revenue line", "analyst")
|
||||
```
|
||||
|
||||
Format: `Source: [System/Document], [Date], [Reference], [URL if applicable]`.
|
||||
|
||||
Never defer sourcing. Never write `TODO: add source`.
|
||||
|
||||
## Skeleton: typical financial model
|
||||
|
||||
```python
|
||||
from openpyxl import Workbook
|
||||
from openpyxl.styles import Font, PatternFill, Alignment, Border, Side
|
||||
from openpyxl.comments import Comment
|
||||
from openpyxl.utils import get_column_letter
|
||||
from pathlib import Path
|
||||
|
||||
BLUE = Font(color="0000FF")
|
||||
BLACK = Font(color="000000")
|
||||
GREEN = Font(color="006100")
|
||||
BOLD = Font(bold=True)
|
||||
HEADER_FILL = PatternFill("solid", fgColor="1F4E79")
|
||||
HEADER_FONT = Font(color="FFFFFF", bold=True)
|
||||
|
||||
wb = Workbook()
|
||||
|
||||
# --- Inputs tab ---
|
||||
inp = wb.active
|
||||
inp.title = "Inputs"
|
||||
inp["A1"] = "MARKET DATA & KEY INPUTS"
|
||||
inp["A1"].font = HEADER_FONT
|
||||
inp["A1"].fill = HEADER_FILL
|
||||
inp.merge_cells("A1:C1")
|
||||
|
||||
inp["B3"] = "Revenue FY2024"
|
||||
inp["C3"] = 1_250_000_000
|
||||
inp["C3"].font = BLUE
|
||||
inp["C3"].comment = Comment("Source: 10-K FY2024 p.47", "model")
|
||||
|
||||
inp["B4"] = "Growth Rate"
|
||||
inp["C4"] = 0.12
|
||||
inp["C4"].font = BLUE
|
||||
|
||||
# --- Calc tab ---
|
||||
calc = wb.create_sheet("DCF")
|
||||
calc["B2"] = "Projected Revenue"
|
||||
calc["C2"] = "=Inputs!C3*(1+Inputs!C4)" # formula, black
|
||||
|
||||
# --- Checks tab ---
|
||||
chk = wb.create_sheet("Checks")
|
||||
chk["A2"] = "BS balances"
|
||||
chk["B2"] = "=ABS(BS!D20-BS!D21-BS!D22)<0.01"
|
||||
|
||||
Path("./out").mkdir(exist_ok=True)
|
||||
wb.save("./out/model.xlsx")
|
||||
```
|
||||
|
||||
## Section headers with merged cells
|
||||
|
||||
openpyxl quirk: when you merge, set the value on the top-left cell and style the full range separately.
|
||||
|
||||
```python
|
||||
ws["A7"] = "CASH FLOW PROJECTION"
|
||||
ws["A7"].font = HEADER_FONT
|
||||
ws.merge_cells("A7:H7")
|
||||
for col in range(1, 9): # A..H
|
||||
ws.cell(row=7, column=col).fill = HEADER_FILL
|
||||
```
|
||||
|
||||
## Sensitivity tables
|
||||
|
||||
Build with loops, not hardcoded formulas per cell. Rules:
|
||||
|
||||
- **Odd number of rows/cols** (5×5 or 7×7) — guarantees a true center cell.
|
||||
- **Center cell = base case.** The middle row/col header must equal the model's actual WACC and terminal g so the center output equals the base-case implied share price. That's the sanity check.
|
||||
- **Highlight the center cell** with medium-blue fill (`"BDD7EE"`) and bold.
|
||||
- Populate every cell with a full recalculation formula — never an approximation.
|
||||
|
||||
```python
|
||||
# 5x5 WACC (rows) x terminal growth (cols) sensitivity
|
||||
wacc_axis = [0.08, 0.085, 0.09, 0.095, 0.10] # center row = base 9.0%
|
||||
term_axis = [0.02, 0.025, 0.03, 0.035, 0.04] # center col = base 3.0%
|
||||
|
||||
start_row = 40
|
||||
ws.cell(row=start_row, column=1).value = "Implied Share Price ($)"
|
||||
ws.cell(row=start_row, column=1).font = BOLD
|
||||
|
||||
for j, g in enumerate(term_axis):
|
||||
ws.cell(row=start_row+1, column=2+j).value = g
|
||||
ws.cell(row=start_row+1, column=2+j).font = BLUE
|
||||
|
||||
for i, w in enumerate(wacc_axis):
|
||||
r = start_row + 2 + i
|
||||
ws.cell(row=r, column=1).value = w
|
||||
ws.cell(row=r, column=1).font = BLUE
|
||||
for j, g in enumerate(term_axis):
|
||||
c = 2 + j
|
||||
# Full DCF recalc formula (simplified for illustration).
|
||||
# In a real model this references the full projection block.
|
||||
ws.cell(row=r, column=c).value = (
|
||||
f"=SUMPRODUCT(FCF_range,1/(1+{w})^year_offset) + "
|
||||
f"FCF_terminal*(1+{g})/({w}-{g})/(1+{w})^terminal_year"
|
||||
)
|
||||
|
||||
# Highlight center cell (base case)
|
||||
center = ws.cell(row=start_row+2+len(wacc_axis)//2,
|
||||
column=2+len(term_axis)//2)
|
||||
center.fill = PatternFill("solid", fgColor="BDD7EE")
|
||||
center.font = BOLD
|
||||
```
|
||||
|
||||
## Recalculating before delivery
|
||||
|
||||
openpyxl writes formula strings but does not compute them. Excel recalculates on open, but downstream consumers (auto-check scripts, CI) need computed values.
|
||||
|
||||
Run LibreOffice or a dedicated recalc step before delivery:
|
||||
|
||||
```bash
|
||||
# LibreOffice headless recalc
|
||||
libreoffice --headless --calc --convert-to xlsx ./out/model.xlsx --outdir ./out/
|
||||
```
|
||||
|
||||
Or use a Python recalc helper (see `scripts/recalc.py` in this skill).
|
||||
|
||||
## Model layout planning
|
||||
|
||||
Before writing any formula:
|
||||
1. Define ALL section row positions
|
||||
2. Write ALL headers and labels
|
||||
3. Write ALL section dividers and blank rows
|
||||
4. THEN write formulas using the locked row positions
|
||||
|
||||
This prevents the cascading-formula-breakage pattern where inserting a header row after formulas are written shifts every downstream reference.
|
||||
|
||||
## Verify step-by-step with the user
|
||||
|
||||
For large models (DCFs, 3-statement, LBO), stop and show the user intermediate artifacts before continuing. Catching a wrong margin assumption before you've built downstream sensitivity tables saves an hour.
|
||||
|
||||
Checkpoint pattern:
|
||||
- After Inputs block → show raw inputs, confirm before projecting
|
||||
- After Revenue projections → confirm top line + growth
|
||||
- After FCF build → confirm the full schedule
|
||||
- After WACC → confirm inputs
|
||||
- After valuation → confirm the equity bridge
|
||||
- THEN build sensitivity tables
|
||||
|
||||
## When NOT to use this skill
|
||||
|
||||
- Users in a live Excel session with an Office MCP available — drive their live workbook instead.
|
||||
- Pure tabular data export with no formulas — `csv` or `pandas.to_excel` is simpler.
|
||||
- Dashboards / charts with heavy interactivity — use a real BI tool.
|
||||
|
||||
## Attribution
|
||||
|
||||
Conventions (blue/black/green, formulas-over-hardcodes, named ranges, sensitivity rules) adapted from Anthropic's Claude for Financial Services plugin suite, Apache-2.0 licensed. Original: https://github.com/anthropics/financial-services/tree/main/plugins/vertical-plugins/financial-analysis/skills/xlsx-author
|
||||
@@ -0,0 +1,88 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Recalculate an .xlsx file's formulas using LibreOffice headless.
|
||||
|
||||
Usage: python recalc.py <path.xlsx> [timeout_seconds]
|
||||
|
||||
openpyxl writes formula strings but does not compute them. Downstream scripts
|
||||
that open the file with data_only=True get None for every formula cell until
|
||||
something has actually calculated the workbook. Excel does this on open;
|
||||
headless pipelines need LibreOffice (or similar) to do it explicitly.
|
||||
|
||||
Exits 0 on success (workbook recomputed and resaved in place), non-zero on
|
||||
failure. Writes status JSON to stdout either way.
|
||||
"""
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def find_libreoffice() -> str | None:
|
||||
for cmd in ("libreoffice", "soffice"):
|
||||
path = shutil.which(cmd)
|
||||
if path:
|
||||
return path
|
||||
return None
|
||||
|
||||
|
||||
def recalc(xlsx_path: str, timeout: int = 60) -> dict:
|
||||
src = Path(xlsx_path).resolve()
|
||||
if not src.exists():
|
||||
return {"status": "error", "error": f"File not found: {src}"}
|
||||
|
||||
lo = find_libreoffice()
|
||||
if lo is None:
|
||||
return {
|
||||
"status": "error",
|
||||
"error": "libreoffice not found on PATH — install it or recalc in a real Excel session",
|
||||
}
|
||||
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
try:
|
||||
subprocess.run(
|
||||
[
|
||||
lo,
|
||||
"--headless",
|
||||
"--calc",
|
||||
"--convert-to",
|
||||
"xlsx",
|
||||
str(src),
|
||||
"--outdir",
|
||||
td,
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
timeout=timeout,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
return {"status": "error", "error": f"libreoffice timed out after {timeout}s"}
|
||||
except subprocess.CalledProcessError as e:
|
||||
return {
|
||||
"status": "error",
|
||||
"error": f"libreoffice exited {e.returncode}: {e.stderr.decode(errors='replace')[:500]}",
|
||||
}
|
||||
|
||||
produced = Path(td) / src.name
|
||||
if not produced.exists():
|
||||
return {"status": "error", "error": "libreoffice did not produce output file"}
|
||||
|
||||
shutil.copy(produced, src)
|
||||
|
||||
return {"status": "success", "file": str(src)}
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: python recalc.py <path.xlsx> [timeout_seconds]", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
timeout = int(sys.argv[2]) if len(sys.argv) > 2 else 60
|
||||
result = recalc(sys.argv[1], timeout=timeout)
|
||||
print(json.dumps(result, indent=2))
|
||||
sys.exit(0 if result["status"] == "success" else 1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,291 @@
|
||||
---
|
||||
name: lbo-model
|
||||
description: Build leveraged buyout models in Excel — sources & uses, debt schedule, cash sweep, exit multiple, IRR/MOIC sensitivity. Pairs with excel-author. Use for PE screening, sponsor-case valuation, or illustrative LBO in a pitch.
|
||||
version: 1.0.0
|
||||
author: Anthropic (adapted by Nous Research)
|
||||
license: Apache-2.0
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [finance, valuation, lbo, private-equity, excel, openpyxl, modeling]
|
||||
related_skills: [excel-author, pptx-author, dcf-model, 3-statement-model]
|
||||
---
|
||||
|
||||
## Environment
|
||||
|
||||
This skill assumes **headless openpyxl** — you are producing an .xlsx file on disk.
|
||||
Follow the `excel-author` skill's conventions for cell coloring, formulas, named ranges, and sensitivity tables.
|
||||
Recalculate before delivery: `python /path/to/excel-author/scripts/recalc.py ./out/model.xlsx`.
|
||||
|
||||
---
|
||||
|
||||
## TEMPLATE REQUIREMENT
|
||||
|
||||
**This skill uses templates for LBO models. Always check for an attached template file first.**
|
||||
|
||||
Before starting any LBO model:
|
||||
1. **If a template file is attached/provided**: Use that template's structure exactly - copy it and populate with the user's data
|
||||
2. **If no template is attached**: Ask the user: *"Do you have a specific LBO template you'd like me to use? If not, I can use the standard template which includes Sources & Uses, Operating Model, Debt Schedule, and Returns Analysis."*
|
||||
3. **If using the standard template**: Copy `examples/LBO_Model.xlsx` as your starting point and populate it with the user's assumptions
|
||||
|
||||
**IMPORTANT**: When a file like `LBO_Model.xlsx` is attached, you MUST use it as your template - do not build from scratch. Even if the template seems complex or has more features than needed, copy it and adapt it to the user's requirements. Never decide to "build from scratch" when a template is provided.
|
||||
|
||||
---
|
||||
|
||||
## CRITICAL INSTRUCTIONS — READ FIRST
|
||||
|
||||
Use Python/openpyxl. Write formula strings (`ws["D20"] = "=B5*B6"`), then run the `excel-author` skill's `recalc.py` helper before delivery.
|
||||
|
||||
### Core Principles
|
||||
* **Every calculation must be an Excel formula** - NEVER compute values in Python and hardcode results into cells. When using openpyxl, write `cell.value = "=B5*B6"` (formula string), NOT `cell.value = 1250` (computed result). The model must be dynamic and update when inputs change.
|
||||
* **Use the template structure** - Follow the organization in `examples/LBO_Model.xlsx` or the user's provided template. Do not invent your own layout.
|
||||
* **Use proper cell references** - All formulas should reference the appropriate cells. Never type numbers that should come from other cells.
|
||||
* **Maintain sign convention consistency** - Follow whatever sign convention the template uses (some use negative for outflows, some use positive). Be consistent throughout.
|
||||
* **Work section by section, verify with user at each step** - Complete one section fully, show the user what was built, run the section's verification checks, and get confirmation BEFORE moving to the next section. Do NOT build the entire model end-to-end and then present it — later sections depend on earlier ones, so catching a mistake in Sources & Uses after the returns are already built means rework everywhere.
|
||||
|
||||
### Formula Color Conventions
|
||||
* **Blue (0000FF)**: Hardcoded inputs - typed numbers that don't reference other cells
|
||||
* **Black (000000)**: Formulas with calculations - any formula using operators or functions (`=B4*B5`, `=SUM()`, `=-MAX(0,B4)`)
|
||||
* **Purple (800080)**: Links to cells on the **same tab** - direct references with no calculation (`=B9`, `=B45`)
|
||||
* **Green (008000)**: Links to cells on **different tabs** - cross-sheet references (`=Assumptions!B5`, `='Operating Model'!C10`)
|
||||
|
||||
### Fill Color Palette — Professional Blues & Greys (Default unless user/template specifies otherwise)
|
||||
* **Keep it minimal** — only use blues and greys for cell fills. Do NOT introduce greens, yellows, reds, or multiple accents. A professional LBO model uses restraint.
|
||||
* **Default fill palette:**
|
||||
* **Section headers** (Sources & Uses, Operating Model, etc.): Dark blue `#1F4E79` with white bold text
|
||||
* **Column headers** (Year 1, Year 2, etc.): Light blue `#D9E1F2` with black bold text
|
||||
* **Input cells**: Light grey `#F2F2F2` (or just white) — the blue *font* is the signal, fill is secondary
|
||||
* **Formula/calculated cells**: White, no fill
|
||||
* **Key outputs** (IRR, MOIC, Exit Equity): Medium blue `#BDD7EE` with black bold text
|
||||
* **That's the whole palette.** 3 blues + 1 grey + white. If the template uses its own colors, follow the template instead.
|
||||
* Note: The blue/black/purple/green **font** colors above are for distinguishing inputs vs formulas vs links. Those are separate from the **fill** palette here — both work together.
|
||||
|
||||
### Number Formatting Standards
|
||||
* **Currency**: `$#,##0;($#,##0);"-"` or `$#,##0.0` depending on template
|
||||
* **Percentages**: `0.0%` (one decimal)
|
||||
* **Multiples**: `0.0"x"` (one decimal)
|
||||
* **MOIC/Detailed Ratios**: `0.00"x"` (two decimals for precision)
|
||||
* **All numeric cells**: Right-aligned
|
||||
|
||||
---
|
||||
|
||||
### Clarify Requirements First
|
||||
|
||||
Before filling any formulas:
|
||||
|
||||
* **Examine the template structure** - Identify all sections, understand the timeline (which columns are which periods), note any existing formulas
|
||||
* **Ask the user if anything is unclear** - If the template structure, calculation methods, or requirements are ambiguous, ask before proceeding
|
||||
* **Confirm key assumptions** - Any key inputs, calculation preferences, or specific requirements
|
||||
* **ONLY AFTER understanding the template**, proceed to fill in formulas
|
||||
|
||||
---
|
||||
|
||||
## TEMPLATE ANALYSIS PHASE - DO THIS FIRST
|
||||
|
||||
Before filling any formulas, examine the template thoroughly:
|
||||
|
||||
1. **Map the structure** - Identify where each section lives and how they relate to each other. Note which sections feed into others.
|
||||
|
||||
2. **Understand the timeline** - Which columns represent which periods? Is there a "Closing" or "Pro Forma" column? Where does the projection period start?
|
||||
|
||||
3. **Identify input vs formula cells** - Templates often use color coding, borders, or shading to indicate which cells need inputs vs formulas. Respect these conventions.
|
||||
|
||||
4. **Read existing labels carefully** - The row labels tell you exactly what calculation is expected. Don't assume - read what the template is asking for.
|
||||
|
||||
5. **Check for existing formulas** - Some templates come partially filled. Don't overwrite working formulas unless specifically asked.
|
||||
|
||||
6. **Note template-specific conventions** - Sign conventions, subtotal structures, how sections are organized, whether there are separate tabs for different components, etc.
|
||||
|
||||
---
|
||||
|
||||
## FILLING FORMULAS - GENERAL APPROACH
|
||||
|
||||
For each cell that needs a formula, follow this hierarchy:
|
||||
|
||||
### Step 1: Check the Template
|
||||
* Does the cell already have a formula? If yes, verify it's correct and move on.
|
||||
* Is there a comment or note indicating the expected calculation?
|
||||
* Does the row/column label make the calculation obvious?
|
||||
* Do neighboring cells show a pattern you should follow?
|
||||
|
||||
### Step 2: Check the User's Instructions
|
||||
* Did the user specify a particular calculation method?
|
||||
* Are there stated assumptions that affect this formula?
|
||||
* Any special requirements mentioned?
|
||||
|
||||
### Step 3: Apply Standard Practice
|
||||
* If neither template nor user specifies, use standard LBO modeling conventions
|
||||
* Document any assumptions you make
|
||||
* If genuinely uncertain, ask the user
|
||||
|
||||
---
|
||||
|
||||
## COMMON PROBLEM AREAS
|
||||
|
||||
The following calculation patterns frequently cause issues across LBO models. Pay special attention when you encounter these:
|
||||
|
||||
### Balancing Sections
|
||||
* When two sections must equal (e.g., Sources = Uses), one item is typically the "plug" (balancing figure)
|
||||
* Identify which item is the plug and calculate it as the difference
|
||||
|
||||
### Tax Calculations
|
||||
* Tax formulas should only reference the relevant income line and tax rate
|
||||
* Should NOT reference unrelated sections (e.g., debt schedules)
|
||||
* Consider whether losses create tax shields or are simply ignored
|
||||
|
||||
### Interest and Circular References
|
||||
* Interest calculations can create circularity if they reference balances affected by cash flows
|
||||
* Use **Beginning Balance** (not average or ending) to break circular references
|
||||
* Pattern: Interest → Cash Flow → Paydown → Ending Balance (if interest uses ending balance, this circles back)
|
||||
|
||||
### Debt Paydown / Cash Sweeps
|
||||
* When multiple debt tranches exist, there's usually a priority order
|
||||
* Cash sweep should respect the priority waterfall
|
||||
* Balances cannot go negative - use MAX or MIN functions appropriately
|
||||
|
||||
### Returns Calculations (IRR/MOIC)
|
||||
* Cash flows must have correct signs: Investment = negative, Proceeds = positive
|
||||
* If using XIRR, need corresponding dates
|
||||
* If using IRR, cash flows should be in consecutive periods
|
||||
* MOIC = Total Proceeds / Total Investment
|
||||
|
||||
### Sensitivity Tables
|
||||
* **Use ODD dimensions** (5×5 or 7×7) — never 4×4 or 6×6. Odd dimensions guarantee a true center cell.
|
||||
* **Center cell = base case.** Build the row and column axis values symmetrically around the model's actual assumptions (e.g., if base entry multiple = 10.0x, axis = `[8.0x, 9.0x, 10.0x, 11.0x, 12.0x]`). The center cell's IRR/MOIC MUST then equal the model's actual IRR/MOIC output — this is the proof the table is wired correctly.
|
||||
* **Highlight the center cell** — medium-blue fill (`#BDD7EE`) + bold font so the base case is visually anchored.
|
||||
* Excel's DATA TABLE function may not work with openpyxl — instead write explicit formulas that reference row/column headers
|
||||
* Each cell should show a DIFFERENT value — if all same, formulas aren't varying correctly
|
||||
* Use mixed references (e.g., `$A5` for row input, `B$4` for column input)
|
||||
|
||||
---
|
||||
|
||||
## VERIFICATION CHECKLIST - RUN AFTER COMPLETION
|
||||
|
||||
### Run Formula Validation
|
||||
```bash
|
||||
python /path/to/excel-author/scripts/recalc.py model.xlsx
|
||||
```
|
||||
Must return success with zero errors.
|
||||
|
||||
### Section Balancing
|
||||
- [ ] Any sections that must balance (Sources/Uses, Assets/Liabilities) balance exactly
|
||||
- [ ] Plug items are calculated correctly as the balancing figure
|
||||
- [ ] Amounts that should match across sections are consistent
|
||||
|
||||
### Income/Operating Projections
|
||||
- [ ] Revenue/top-line builds correctly from drivers or growth rates
|
||||
- [ ] All cost and expense items calculated appropriately
|
||||
- [ ] Subtotals and totals sum correctly
|
||||
- [ ] Margins and ratios are reasonable
|
||||
- [ ] Links to assumptions are correct
|
||||
|
||||
### Balance Sheet (if applicable)
|
||||
- [ ] Assets = Liabilities + Equity (must balance)
|
||||
- [ ] All items link to appropriate schedules or roll-forwards
|
||||
- [ ] Beginning balances = prior period ending balances
|
||||
- [ ] Check row included and shows zero
|
||||
|
||||
### Cash Flow (if applicable)
|
||||
- [ ] Starts with correct income figure
|
||||
- [ ] Non-cash items added/subtracted appropriately
|
||||
- [ ] Working capital changes have correct signs
|
||||
- [ ] Ending Cash = Beginning Cash + Net Cash Flow
|
||||
- [ ] Cash balances are consistent across statements
|
||||
|
||||
### Supporting Schedules
|
||||
- [ ] Roll-forward schedules balance (Beginning + Changes = Ending)
|
||||
- [ ] Schedules link correctly to main statements
|
||||
- [ ] Calculated items use appropriate drivers
|
||||
- [ ] All periods are calculated consistently
|
||||
|
||||
### Debt/Financing Schedules (if applicable)
|
||||
- [ ] Beginning balances tie to sources or prior period
|
||||
- [ ] Interest calculated on appropriate balance (typically beginning)
|
||||
- [ ] Paydowns respect cash availability and priority
|
||||
- [ ] Ending balances cannot be negative
|
||||
- [ ] Totals sum tranches correctly
|
||||
|
||||
### Returns/Output Analysis
|
||||
- [ ] Exit/terminal values calculated correctly
|
||||
- [ ] All relevant adjustments included
|
||||
- [ ] Cash flow signs are correct (negative for investment, positive for proceeds)
|
||||
- [ ] IRR/MOIC formulas reference complete ranges
|
||||
- [ ] Results are reasonable for the scenario
|
||||
|
||||
### Sensitivity Tables (if applicable)
|
||||
- [ ] Grid dimensions are ODD (5×5 or 7×7) — there is a true center cell
|
||||
- [ ] Row and column axis values are symmetric around the base case (`[base-2Δ, base-Δ, base, base+Δ, base+2Δ]`)
|
||||
- [ ] Center cell output equals the model's actual IRR/MOIC — confirms the table is wired correctly
|
||||
- [ ] Center cell is highlighted (medium-blue fill `#BDD7EE`, bold font)
|
||||
- [ ] Row and column headers contain appropriate input values
|
||||
- [ ] Each data cell contains a formula (not hardcoded)
|
||||
- [ ] Each data cell shows a DIFFERENT value
|
||||
- [ ] Values move in expected directions (higher exit multiple → higher IRR, etc.)
|
||||
|
||||
### Formatting
|
||||
- [ ] Hardcoded inputs are blue (0000FF)
|
||||
- [ ] Calculated formulas are black (000000)
|
||||
- [ ] Same-tab links are purple (800080)
|
||||
- [ ] Cross-tab links are green (008000)
|
||||
- [ ] All numbers are right-aligned
|
||||
- [ ] Appropriate number formats applied throughout
|
||||
- [ ] No cells show error values (#REF!, #DIV/0!, #VALUE!, #NAME?)
|
||||
|
||||
### Logical Sanity Checks
|
||||
- [ ] Numbers are reasonable order of magnitude
|
||||
- [ ] Trends make sense (growth, decline, stabilization as expected)
|
||||
- [ ] No obviously wrong values (negative where should be positive, impossible percentages, etc.)
|
||||
- [ ] Key outputs are within reasonable ranges for the type of analysis
|
||||
|
||||
---
|
||||
|
||||
## COMMON ERRORS TO AVOID
|
||||
|
||||
| Error | What Goes Wrong | How to Fix |
|
||||
|-------|-----------------|------------|
|
||||
| Hardcoding calculated values | Model doesn't update when inputs change | Always use formulas that reference source cells |
|
||||
| Wrong cell references after copying | Formulas point to wrong cells | Verify all links, use appropriate $ anchoring |
|
||||
| Circular reference errors | Model can't calculate | Use beginning balances for interest-type calcs, break the circle |
|
||||
| Sections don't balance | Totals that should match don't | Ensure one item is the plug (calculated as difference) |
|
||||
| Negative balances where impossible | Paying/using more than available | Use MAX(0, ...) or MIN functions appropriately |
|
||||
| IRR/return errors | Wrong signs or incomplete ranges | Check cash flow signs and ensure formula covers all periods |
|
||||
| Sensitivity table shows same value | Formula not varying with inputs | Check cell references - need mixed references ($A5, B$4) |
|
||||
| Roll-forwards don't tie | Beginning ≠ prior ending | Verify links between periods |
|
||||
| Inconsistent sign conventions | Additions become subtractions or vice versa | Follow template's convention consistently throughout |
|
||||
|
||||
---
|
||||
|
||||
## WORKING WITH THE USER — SECTION-BY-SECTION CHECKPOINTS
|
||||
|
||||
* **If the template structure is unclear**, ask before proceeding
|
||||
* **If the user's requirements conflict with the template**, confirm their preference
|
||||
* **After completing each major section**, STOP and verify with the user before continuing:
|
||||
- **After Sources & Uses** → show the balanced table, confirm the plug is correct, get sign-off before building the operating model
|
||||
- **After Operating Model / Projections** → show the projected P&L, confirm growth rates and margins look right, get sign-off before the debt schedule
|
||||
- **After Debt Schedule** → show beginning/ending balances and interest, confirm the waterfall logic, get sign-off before returns
|
||||
- **After Returns (IRR/MOIC)** → show the cash flow series and outputs, confirm signs and ranges, get sign-off before sensitivity tables
|
||||
- **After Sensitivity Tables** → show that each cell varies, confirm the base case lands where expected
|
||||
* **If errors are found during verification**, fix them before moving to the next section
|
||||
* **Show your work** - explain key formulas or assumptions when helpful
|
||||
* **Never present a completed model without having checked in at each section** — it's faster to catch a wrong cell reference at the source than to trace it backwards from a broken IRR
|
||||
|
||||
---
|
||||
|
||||
**This skill produces investment banking-quality LBO models by filling templates with correct formulas, proper formatting, and validated calculations. The skill adapts to any template structure while ensuring financial accuracy and professional presentation standards.**
|
||||
|
||||
|
||||
## Data sources — MCP first, web fallback
|
||||
|
||||
Many passages below say "use the S&P Kensho MCP / Daloopa MCP / FactSet MCP". Those are commercial financial-data MCPs from the original Cowork plugin context. In Hermes:
|
||||
|
||||
- **If you have any structured financial-data MCP configured** (Hermes supports MCP — see `native-mcp` skill), prefer it for point-in-time comps, precedent transactions, and filings.
|
||||
- **Otherwise**, fall back to:
|
||||
- `web_search` / `web_extract` against SEC EDGAR (`https://www.sec.gov/cgi-bin/browse-edgar`) for US filings
|
||||
- Company IR pages for press releases, earnings decks
|
||||
- `browser_navigate` for interactive data portals
|
||||
- User-provided data (explicitly ask when the context doesn't have it)
|
||||
- **Never fabricate**. If a multiple, precedent, or filing number can't be sourced, flag the cell as `[UNSOURCED]` and surface it to the user.
|
||||
|
||||
## Attribution
|
||||
|
||||
This skill is adapted from Anthropic's Claude for Financial Services plugin suite (Apache-2.0). The Office-JS / Cowork live-Excel paths have been removed; this version targets headless openpyxl via the `excel-author` skill's conventions. Original: https://github.com/anthropics/financial-services
|
||||
@@ -0,0 +1,144 @@
|
||||
---
|
||||
name: merger-model
|
||||
description: Build accretion/dilution (merger) models in Excel — pro-forma P&L, synergies, financing mix, EPS impact. Pairs with excel-author. Use for M&A pitches, board materials, or deal evaluation.
|
||||
version: 1.0.0
|
||||
author: Anthropic (adapted by Nous Research)
|
||||
license: Apache-2.0
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [finance, m-and-a, merger, accretion-dilution, excel, openpyxl, modeling, investment-banking]
|
||||
related_skills: [excel-author, pptx-author, dcf-model, 3-statement-model]
|
||||
---
|
||||
|
||||
## Environment
|
||||
|
||||
This skill assumes **headless openpyxl** — you are producing an .xlsx file on disk.
|
||||
Follow the `excel-author` skill's conventions for cell coloring, formulas, named ranges, and sensitivity tables.
|
||||
Recalculate before delivery: `python /path/to/excel-author/scripts/recalc.py ./out/model.xlsx`.
|
||||
|
||||
# Merger Model
|
||||
|
||||
Build accretion/dilution analysis for M&A transactions. Models pro forma EPS impact, synergy sensitivities, and purchase price allocation. Use when evaluating a potential acquisition, preparing merger consequences analysis for a pitch, or advising on deal terms.
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 1: Gather Inputs
|
||||
|
||||
**Acquirer:**
|
||||
- Company name, current share price, shares outstanding
|
||||
- LTM and NTM EPS (GAAP and adjusted)
|
||||
- P/E multiple
|
||||
- Pre-tax cost of debt, tax rate
|
||||
- Cash on balance sheet, existing debt
|
||||
|
||||
**Target:**
|
||||
- Company name, current share price, shares outstanding (if public)
|
||||
- LTM and NTM EPS or net income
|
||||
- Enterprise value or equity value
|
||||
|
||||
**Deal Terms:**
|
||||
- Offer price per share (or premium to current)
|
||||
- Consideration mix: % cash vs. % stock
|
||||
- New debt raised to fund cash portion
|
||||
- Expected synergies (revenue and cost) and phase-in timeline
|
||||
- Transaction fees and financing costs
|
||||
- Expected close date
|
||||
|
||||
### Step 2: Purchase Price Analysis
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Offer price per share | |
|
||||
| Premium to current | |
|
||||
| Equity value | |
|
||||
| Plus: net debt assumed | |
|
||||
| Enterprise value | |
|
||||
| EV / EBITDA implied | |
|
||||
| P/E implied | |
|
||||
|
||||
### Step 3: Sources & Uses
|
||||
|
||||
| Sources | $ | Uses | $ |
|
||||
|---------|---|------|---|
|
||||
| New debt | | Equity purchase price | |
|
||||
| Cash on hand | | Refinance target debt | |
|
||||
| New equity issued | | Transaction fees | |
|
||||
| | | Financing fees | |
|
||||
| **Total** | | **Total** | |
|
||||
|
||||
### Step 4: Pro Forma EPS (Accretion / Dilution)
|
||||
|
||||
Calculate year-by-year (Year 1-3):
|
||||
|
||||
| | Standalone | Pro Forma | Accretion/(Dilution) |
|
||||
|---|-----------|-----------|---------------------|
|
||||
| Acquirer net income | | | |
|
||||
| Target net income | | | |
|
||||
| Synergies (after tax) | | | |
|
||||
| Foregone interest on cash (after tax) | | | |
|
||||
| New debt interest (after tax) | | | |
|
||||
| Intangible amortization (after tax) | | | |
|
||||
| Pro forma net income | | | |
|
||||
| Pro forma shares | | | |
|
||||
| **Pro forma EPS** | | | |
|
||||
| **Accretion / (Dilution) %** | | | |
|
||||
|
||||
### Step 5: Sensitivity Analysis
|
||||
|
||||
**Accretion/Dilution vs. Synergies and Offer Premium:**
|
||||
|
||||
| | $0M syn | $25M syn | $50M syn | $75M syn | $100M syn |
|
||||
|---|---------|----------|----------|----------|-----------|
|
||||
| 15% premium | | | | | |
|
||||
| 20% premium | | | | | |
|
||||
| 25% premium | | | | | |
|
||||
| 30% premium | | | | | |
|
||||
|
||||
**Accretion/Dilution vs. Cash/Stock Mix:**
|
||||
|
||||
| | 100% cash | 75/25 | 50/50 | 25/75 | 100% stock |
|
||||
|---|-----------|-------|-------|-------|------------|
|
||||
| Year 1 | | | | | |
|
||||
| Year 2 | | | | | |
|
||||
|
||||
### Step 6: Breakeven Synergies
|
||||
|
||||
Calculate the minimum synergies needed for the deal to be EPS-neutral in Year 1.
|
||||
|
||||
### Step 7: Output
|
||||
|
||||
- Excel workbook with:
|
||||
- Assumptions tab
|
||||
- Sources & uses
|
||||
- Pro forma income statement
|
||||
- Accretion/dilution summary
|
||||
- Sensitivity tables
|
||||
- Breakeven analysis
|
||||
- One-page merger consequences summary for pitch book
|
||||
|
||||
## Important Notes
|
||||
|
||||
- Always show both GAAP and adjusted (cash) EPS where relevant
|
||||
- Stock deals: use acquirer's current price for exchange ratio, note dilution from new shares
|
||||
- Include purchase price allocation — goodwill and intangible amortization matter for GAAP EPS
|
||||
- Synergy phase-in is critical — Year 1 is often only 25-50% of run-rate synergies
|
||||
- Don't forget foregone interest income on cash used and new interest expense on debt raised
|
||||
- Tax rate on synergies and interest adjustments should match the acquirer's marginal rate
|
||||
|
||||
|
||||
## Data sources — MCP first, web fallback
|
||||
|
||||
Many passages below say "use the S&P Kensho MCP / Daloopa MCP / FactSet MCP". Those are commercial financial-data MCPs from the original Cowork plugin context. In Hermes:
|
||||
|
||||
- **If you have any structured financial-data MCP configured** (Hermes supports MCP — see `native-mcp` skill), prefer it for point-in-time comps, precedent transactions, and filings.
|
||||
- **Otherwise**, fall back to:
|
||||
- `web_search` / `web_extract` against SEC EDGAR (`https://www.sec.gov/cgi-bin/browse-edgar`) for US filings
|
||||
- Company IR pages for press releases, earnings decks
|
||||
- `browser_navigate` for interactive data portals
|
||||
- User-provided data (explicitly ask when the context doesn't have it)
|
||||
- **Never fabricate**. If a multiple, precedent, or filing number can't be sourced, flag the cell as `[UNSOURCED]` and surface it to the user.
|
||||
|
||||
## Attribution
|
||||
|
||||
This skill is adapted from Anthropic's Claude for Financial Services plugin suite (Apache-2.0). The Office-JS / Cowork live-Excel paths have been removed; this version targets headless openpyxl via the `excel-author` skill's conventions. Original: https://github.com/anthropics/financial-services
|
||||
@@ -0,0 +1,173 @@
|
||||
---
|
||||
name: pptx-author
|
||||
description: Build PowerPoint decks headless with python-pptx. Pairs with excel-author for model-backed decks where every number traces to a workbook cell. Use for pitch decks, IC memos, earnings notes.
|
||||
version: 1.0.0
|
||||
author: Anthropic (adapted by Nous Research)
|
||||
license: Apache-2.0
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [powerpoint, pptx, python-pptx, presentation, finance]
|
||||
related_skills: [excel-author, powerpoint]
|
||||
---
|
||||
|
||||
# pptx-author
|
||||
|
||||
Produce a .pptx file on disk using `python-pptx`. Use when you need to deliver a deck as a file artifact, not drive a live PowerPoint session.
|
||||
|
||||
Adapted from Anthropic's `pptx-author` and `pitch-deck` skills in [anthropics/financial-services](https://github.com/anthropics/financial-services). The MCP / Office-JS branches of the originals are dropped — this assumes headless Python.
|
||||
|
||||
For the broader, already-shipped PowerPoint authoring skill (slides, speaker notes, embeds, media), see the built-in `powerpoint` skill. This skill is a lighter-weight pattern tuned for model-backed decks (pitch decks, IC memos, earnings notes) where every number must trace to a source workbook.
|
||||
|
||||
## Output contract
|
||||
|
||||
- Write to `./out/<name>.pptx`. Create `./out/` if it does not exist.
|
||||
- Return the relative path in your final message.
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
pip install "python-pptx>=0.6"
|
||||
```
|
||||
|
||||
## Core conventions
|
||||
|
||||
### One idea per slide
|
||||
Title states the takeaway; body supports it. A slide titled "Q3 Revenue" is weak; "Revenue growth accelerated to 14% Y/Y in Q3" is strong.
|
||||
|
||||
### Every number traces to the model
|
||||
If a figure on a slide came from `./out/model.xlsx`, footnote the sheet and cell.
|
||||
|
||||
```
|
||||
Revenue: $1,250M (Source: model.xlsx, Inputs!C3)
|
||||
```
|
||||
|
||||
Never transcribe numbers from memory or from a summary — open the workbook, read the named range, and bind the deck value to it programmatically when you can.
|
||||
|
||||
### Use the firm template when one is mounted
|
||||
If `./templates/firm-template.pptx` exists, load it so the deck inherits branded colors, fonts, and master layouts.
|
||||
|
||||
```python
|
||||
from pptx import Presentation
|
||||
from pathlib import Path
|
||||
|
||||
template = Path("./templates/firm-template.pptx")
|
||||
prs = Presentation(str(template)) if template.exists() else Presentation()
|
||||
```
|
||||
|
||||
### Charts: PNG-from-model beats native pptx charts
|
||||
When fidelity matters (the model's chart styling must match the deck exactly), render the chart to PNG from the source workbook and embed the image. Native `pptx.chart` charts are fragile and often don't match firm conventions.
|
||||
|
||||
```python
|
||||
from pptx.util import Inches
|
||||
slide.shapes.add_picture("./out/charts/football_field.png",
|
||||
Inches(1), Inches(2),
|
||||
width=Inches(8))
|
||||
```
|
||||
|
||||
### No external sends
|
||||
This skill writes a file. It never emails, uploads, or posts. Orchestration layers handle delivery.
|
||||
|
||||
## Skeleton
|
||||
|
||||
```python
|
||||
from pptx import Presentation
|
||||
from pptx.util import Inches, Pt
|
||||
from pptx.dml.color import RGBColor
|
||||
from pathlib import Path
|
||||
|
||||
template = Path("./templates/firm-template.pptx")
|
||||
prs = Presentation(str(template)) if template.exists() else Presentation()
|
||||
|
||||
# Title slide
|
||||
slide = prs.slides.add_slide(prs.slide_layouts[0])
|
||||
slide.shapes.title.text = "Project Aurora — Strategic Alternatives"
|
||||
slide.placeholders[1].text = "Preliminary Discussion Materials"
|
||||
|
||||
# Valuation summary slide (title-only layout)
|
||||
slide = prs.slides.add_slide(prs.slide_layouts[5])
|
||||
slide.shapes.title.text = "Valuation implies $38–$52 per share across methodologies"
|
||||
|
||||
# Add a table bound to model outputs
|
||||
rows, cols = 5, 4
|
||||
tbl_shape = slide.shapes.add_table(rows, cols,
|
||||
Inches(0.5), Inches(1.5),
|
||||
Inches(9), Inches(3))
|
||||
tbl = tbl_shape.table
|
||||
headers = ["Methodology", "Low ($)", "Mid ($)", "High ($)"]
|
||||
for c, h in enumerate(headers):
|
||||
tbl.cell(0, c).text = h
|
||||
|
||||
# In a real deck, read these from the model workbook with openpyxl
|
||||
data = [
|
||||
("Trading comps", "35", "41", "48"),
|
||||
("Precedent M&A", "39", "45", "52"),
|
||||
("DCF (base)", "36", "43", "51"),
|
||||
("LBO (10% IRR)", "33", "38", "44"),
|
||||
]
|
||||
for r, row in enumerate(data, start=1):
|
||||
for c, val in enumerate(row):
|
||||
tbl.cell(r, c).text = val
|
||||
|
||||
# Embed a chart rendered from the model
|
||||
slide = prs.slides.add_slide(prs.slide_layouts[5])
|
||||
slide.shapes.title.text = "Football field — current price $42"
|
||||
slide.shapes.add_picture("./out/charts/football_field.png",
|
||||
Inches(1), Inches(1.8), width=Inches(8))
|
||||
|
||||
Path("./out").mkdir(exist_ok=True)
|
||||
prs.save("./out/pitch-aurora.pptx")
|
||||
```
|
||||
|
||||
## Binding deck numbers to the source workbook
|
||||
|
||||
Read named ranges or specific cells from your Excel model so deck numbers never drift.
|
||||
|
||||
```python
|
||||
from openpyxl import load_workbook
|
||||
|
||||
wb = load_workbook("./out/model.xlsx", data_only=True)
|
||||
def nr(name):
|
||||
"""Resolve a named range to its current computed value."""
|
||||
rng = wb.defined_names[name]
|
||||
sheet, coord = next(rng.destinations)
|
||||
return wb[sheet][coord].value
|
||||
|
||||
revenue_fy24 = nr("RevenueFY24")
|
||||
implied_mid = nr("ImpliedSharePriceBase")
|
||||
```
|
||||
|
||||
Then build deck content using those values:
|
||||
```python
|
||||
slide.shapes.title.text = f"Implied share price of ${implied_mid:.2f} (base case)"
|
||||
```
|
||||
|
||||
Remember to recalculate the workbook before reading it — openpyxl only sees computed values if something has already calculated the sheet. Run the recalc helper in the `excel-author` skill first, or open/save through a real Excel session.
|
||||
|
||||
## Slide-type checklist for pitch decks
|
||||
|
||||
A typical banking pitch deck follows this structure. Not prescriptive, but useful as a starting skeleton:
|
||||
|
||||
1. Cover / title
|
||||
2. Disclaimer
|
||||
3. Table of contents
|
||||
4. Situation overview
|
||||
5. Company snapshot (the target)
|
||||
6. Market / sector context
|
||||
7. Valuation summary (football field) — the money slide
|
||||
8. Trading comps detail
|
||||
9. Precedent transactions detail
|
||||
10. DCF summary
|
||||
11. Illustrative LBO / sponsor case
|
||||
12. Process considerations
|
||||
13. Appendix
|
||||
|
||||
## When NOT to use this skill
|
||||
|
||||
- Users in a live PowerPoint session with an Office MCP available — drive their live doc instead.
|
||||
- Non-financial slideware (quarterly all-hands, marketing decks) — use the broader `powerpoint` skill.
|
||||
- Decks with heavy animation, transitions, or speaker notes — use the broader `powerpoint` skill.
|
||||
|
||||
## Attribution
|
||||
|
||||
Conventions adapted from Anthropic's Claude for Financial Services plugin suite, Apache-2.0 licensed. Original: https://github.com/anthropics/financial-services/tree/main/plugins/agent-plugins/pitch-agent/skills/pptx-author
|
||||
@@ -0,0 +1,95 @@
|
||||
---
|
||||
name: stocks
|
||||
description: Stock quotes, history, search, compare, crypto via Yahoo.
|
||||
version: 0.1.0
|
||||
author: Mibay (Mibayy), Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Stocks, Finance, Market, Crypto, Investing]
|
||||
category: finance
|
||||
related_skills: [dcf-model, comps-analysis, lbo-model]
|
||||
---
|
||||
|
||||
# Stocks Skill
|
||||
|
||||
Read-only market data via Yahoo Finance. Five commands: `quote`, `search`,
|
||||
`history`, `compare`, `crypto`. Python stdlib only — no API key, no pip
|
||||
installs. Yahoo's endpoint is unofficial and may rate-limit or change.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks for a current stock price (AAPL, TSLA, MSFT, ...)
|
||||
- User wants to look up a ticker by company name
|
||||
- User wants OHLCV history or performance over a date range
|
||||
- User wants to compare several tickers side by side
|
||||
- User asks for a crypto price (BTC, ETH, SOL, ...)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Python 3.8+ stdlib only. Optional: set `ALPHA_VANTAGE_KEY` to enrich
|
||||
`market_cap`, `pe_ratio`, and 52-week levels when Yahoo's crumb-protected
|
||||
fields come back null. Free key: https://www.alphavantage.co/support/#api-key
|
||||
|
||||
## How to Run
|
||||
|
||||
Invoke through the `terminal` tool. Once installed:
|
||||
|
||||
```
|
||||
SCRIPT=~/.hermes/skills/finance/stocks/scripts/stocks_client.py
|
||||
python3 $SCRIPT quote AAPL
|
||||
```
|
||||
|
||||
All output is JSON on stdout — pipe through `jq` if you want to slice it.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
```
|
||||
python3 $SCRIPT quote AAPL
|
||||
python3 $SCRIPT quote AAPL MSFT GOOGL TSLA
|
||||
python3 $SCRIPT search "Tesla"
|
||||
python3 $SCRIPT history NVDA --range 6mo
|
||||
python3 $SCRIPT compare AAPL MSFT GOOGL
|
||||
python3 $SCRIPT crypto BTC ETH SOL
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
### `quote SYMBOL [SYMBOL2 ...]`
|
||||
|
||||
Current price, change, change%, volume, 52-week high/low.
|
||||
|
||||
### `search QUERY`
|
||||
|
||||
Find tickers by company name. Returns top 5: symbol, name, exchange, type.
|
||||
|
||||
### `history SYMBOL [--range RANGE]`
|
||||
|
||||
Daily OHLCV plus stats (min, max, avg, total return %). Ranges: `1mo`,
|
||||
`3mo`, `6mo`, `1y`, `5y`. Default: `1mo`.
|
||||
|
||||
### `compare SYMBOL1 SYMBOL2 [...]`
|
||||
|
||||
Side-by-side: price, change%, 52-week performance.
|
||||
|
||||
### `crypto SYMBOL [SYMBOL2 ...]`
|
||||
|
||||
Crypto prices. Pass `BTC` (the script appends `-USD` automatically).
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Yahoo Finance's API is unofficial. Endpoints can change or rate-limit
|
||||
without notice — if requests start failing, that's why.
|
||||
- `market_cap` and `pe_ratio` may return null on `quote` when Yahoo's
|
||||
crumb session isn't established. Set `ALPHA_VANTAGE_KEY` to backfill.
|
||||
- Add a small delay between bulk requests to avoid rate-limiting.
|
||||
- This is read-only — no order placement, no account integration.
|
||||
|
||||
## Verification
|
||||
|
||||
```
|
||||
python3 ~/.hermes/skills/finance/stocks/scripts/stocks_client.py quote AAPL
|
||||
```
|
||||
|
||||
Returns a JSON object with `symbol: "AAPL"` and a numeric `price` field.
|
||||
+755
@@ -0,0 +1,755 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
stocks_client.py - Stock market data CLI tool for the Hermes Agent project.
|
||||
Zero external dependencies - Python stdlib only.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from datetime import datetime, timezone
|
||||
from http.cookiejar import CookieJar
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
USER_AGENT = "Mozilla/5.0 (compatible; HermesAgent/1.0)"
|
||||
YF_BASE = "https://query1.finance.yahoo.com"
|
||||
YF_BASE2 = "https://query2.finance.yahoo.com"
|
||||
AV_BASE = "https://www.alphavantage.co/query"
|
||||
|
||||
MAX_RETRIES = 3
|
||||
BACKOFF_BASE = 1.5 # seconds
|
||||
|
||||
# Global cookie jar + opener (handles Yahoo Finance session cookies)
|
||||
_cookie_jar = CookieJar()
|
||||
_opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(_cookie_jar))
|
||||
_crumb: str | None = None
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Utilities
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def print_json(data: dict | list) -> None:
|
||||
print(json.dumps(data, indent=2, ensure_ascii=False))
|
||||
|
||||
|
||||
def fmt_price(value) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
return f"{float(value):.2f}"
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def fmt_large(value) -> str | None:
|
||||
"""Format large numbers with B/T suffix."""
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
v = float(value)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
if abs(v) >= 1e12:
|
||||
return f"{v / 1e12:.2f}T"
|
||||
if abs(v) >= 1e9:
|
||||
return f"{v / 1e9:.2f}B"
|
||||
if abs(v) >= 1e6:
|
||||
return f"{v / 1e6:.2f}M"
|
||||
return str(int(v))
|
||||
|
||||
|
||||
def fmt_pct(value) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
return f"{float(value):.2f}%"
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def safe_get(d: dict, *keys, default=None):
|
||||
"""Safely traverse nested dict."""
|
||||
cur = d
|
||||
for k in keys:
|
||||
if not isinstance(cur, dict):
|
||||
return default
|
||||
cur = cur.get(k, default)
|
||||
if cur is None:
|
||||
return default
|
||||
return cur
|
||||
|
||||
|
||||
def ts_to_date(ts) -> str | None:
|
||||
"""Convert Unix timestamp to ISO date string."""
|
||||
if ts is None:
|
||||
return None
|
||||
try:
|
||||
return datetime.fromtimestamp(int(ts), tz=timezone.utc).strftime("%Y-%m-%d")
|
||||
except (OSError, ValueError, TypeError):
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# HTTP layer with retry + exponential backoff
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _build_request(url: str, headers: dict | None = None) -> urllib.request.Request:
|
||||
req = urllib.request.Request(url)
|
||||
req.add_header("User-Agent", USER_AGENT)
|
||||
req.add_header("Accept", "application/json, */*")
|
||||
req.add_header("Accept-Language", "en-US,en;q=0.9")
|
||||
if headers:
|
||||
for k, v in headers.items():
|
||||
req.add_header(k, v)
|
||||
return req
|
||||
|
||||
|
||||
def fetch_url(url: str, headers: dict | None = None, retries: int = MAX_RETRIES) -> dict | list | None:
|
||||
"""Fetch a URL, parse JSON, retry on transient errors."""
|
||||
last_err = None
|
||||
for attempt in range(retries):
|
||||
try:
|
||||
req = _build_request(url, headers)
|
||||
with _opener.open(req, timeout=15) as resp:
|
||||
raw = resp.read()
|
||||
return json.loads(raw.decode("utf-8", errors="replace"))
|
||||
except urllib.error.HTTPError as e:
|
||||
last_err = e
|
||||
if e.code in {404, 400}:
|
||||
break # no point retrying
|
||||
wait = BACKOFF_BASE ** attempt
|
||||
time.sleep(wait)
|
||||
except urllib.error.URLError as e:
|
||||
last_err = e
|
||||
wait = BACKOFF_BASE ** attempt
|
||||
time.sleep(wait)
|
||||
except json.JSONDecodeError as e:
|
||||
last_err = e
|
||||
break
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Yahoo Finance crumb / cookie management
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _fetch_crumb() -> str | None:
|
||||
"""
|
||||
Yahoo Finance v8 requires a crumb + consent cookie.
|
||||
We hit the consent page once to grab cookies, then fetch the crumb.
|
||||
"""
|
||||
global _crumb
|
||||
if _crumb is not None:
|
||||
return _crumb
|
||||
|
||||
# Step 1: touch Yahoo Finance to get cookies
|
||||
try:
|
||||
req = _build_request("https://finance.yahoo.com/")
|
||||
with _opener.open(req, timeout=10) as resp:
|
||||
resp.read()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Step 2: fetch crumb
|
||||
crumb_url = f"{YF_BASE}/v1/test/getcrumb"
|
||||
try:
|
||||
req = _build_request(crumb_url)
|
||||
with _opener.open(req, timeout=10) as resp:
|
||||
crumb_raw = resp.read().decode("utf-8").strip()
|
||||
if crumb_raw and crumb_raw != "":
|
||||
_crumb = crumb_raw
|
||||
return _crumb
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def yf_url(path: str, params: dict | None = None) -> str:
|
||||
"""Build a Yahoo Finance URL, injecting crumb if available."""
|
||||
crumb = _fetch_crumb()
|
||||
if params is None:
|
||||
params = {}
|
||||
if crumb:
|
||||
params["crumb"] = crumb
|
||||
qs = urllib.parse.urlencode(params)
|
||||
base = f"{YF_BASE}{path}"
|
||||
return f"{base}?{qs}" if qs else base
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Yahoo Finance API calls
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def yf_chart(symbol: str, interval: str = "1d", range_: str = "1d") -> dict | None:
|
||||
params = {"interval": interval, "range": range_}
|
||||
crumb = _fetch_crumb()
|
||||
if crumb:
|
||||
params["crumb"] = crumb
|
||||
qs = urllib.parse.urlencode(params)
|
||||
url = f"{YF_BASE}/v8/finance/chart/{urllib.parse.quote(symbol)}?{qs}"
|
||||
data = fetch_url(url)
|
||||
if data is None:
|
||||
# fallback to query2
|
||||
url2 = f"{YF_BASE2}/v8/finance/chart/{urllib.parse.quote(symbol)}?{qs}"
|
||||
data = fetch_url(url2)
|
||||
return data
|
||||
|
||||
|
||||
def yf_search(query: str, count: int = 5) -> dict | None:
|
||||
params = {"q": query, "quotesCount": count, "newsCount": 0}
|
||||
crumb = _fetch_crumb()
|
||||
if crumb:
|
||||
params["crumb"] = crumb
|
||||
qs = urllib.parse.urlencode(params)
|
||||
url = f"{YF_BASE}/v1/finance/search?{qs}"
|
||||
data = fetch_url(url)
|
||||
if data is None:
|
||||
url2 = f"{YF_BASE2}/v1/finance/search?{qs}"
|
||||
data = fetch_url(url2)
|
||||
return data
|
||||
|
||||
|
||||
def yf_quote_summary(symbol: str) -> dict | None:
|
||||
"""Fetch detailed quote summary (quoteSummary) for PE, market cap, etc."""
|
||||
modules = "summaryDetail,defaultKeyStatistics,price"
|
||||
params = {"modules": modules}
|
||||
crumb = _fetch_crumb()
|
||||
if crumb:
|
||||
params["crumb"] = crumb
|
||||
qs = urllib.parse.urlencode(params)
|
||||
url = f"{YF_BASE}/v11/finance/quoteSummary/{urllib.parse.quote(symbol)}?{qs}"
|
||||
data = fetch_url(url)
|
||||
if data is None:
|
||||
url2 = f"{YF_BASE2}/v11/finance/quoteSummary/{urllib.parse.quote(symbol)}?{qs}"
|
||||
data = fetch_url(url2)
|
||||
return data
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Alpha Vantage (optional, requires API key)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def av_overview(symbol: str) -> dict | None:
|
||||
key = os.environ.get("ALPHA_VANTAGE_KEY")
|
||||
if not key:
|
||||
return None
|
||||
params = {"function": "OVERVIEW", "symbol": symbol, "apikey": key}
|
||||
qs = urllib.parse.urlencode(params)
|
||||
url = f"{AV_BASE}?{qs}"
|
||||
data = fetch_url(url)
|
||||
if isinstance(data, dict) and data.get("Symbol"):
|
||||
return data
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Data extraction helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def extract_quote_from_chart(symbol: str, chart_data: dict) -> dict:
|
||||
"""Extract current quote info from v8 chart response."""
|
||||
result = {
|
||||
"symbol": symbol.upper(),
|
||||
"price": None,
|
||||
"change": None,
|
||||
"change_pct": None,
|
||||
"volume": None,
|
||||
"market_cap": None,
|
||||
"pe_ratio": None,
|
||||
"52w_high": None,
|
||||
"52w_low": None,
|
||||
"currency": None,
|
||||
"exchange": None,
|
||||
"short_name": None,
|
||||
}
|
||||
|
||||
chart = safe_get(chart_data, "chart", "result")
|
||||
if not chart or not isinstance(chart, list) or len(chart) == 0:
|
||||
return result
|
||||
|
||||
r = chart[0]
|
||||
meta = r.get("meta", {})
|
||||
|
||||
result["currency"] = meta.get("currency")
|
||||
result["exchange"] = meta.get("exchangeName")
|
||||
result["short_name"] = meta.get("shortName") or meta.get("longName")
|
||||
|
||||
# Price
|
||||
price = meta.get("regularMarketPrice") or meta.get("chartPreviousClose")
|
||||
result["price"] = fmt_price(price)
|
||||
|
||||
# Change
|
||||
prev_close = meta.get("previousClose") or meta.get("chartPreviousClose")
|
||||
if price and prev_close:
|
||||
chg = float(price) - float(prev_close)
|
||||
chg_pct = (chg / float(prev_close)) * 100
|
||||
result["change"] = fmt_price(chg)
|
||||
result["change_pct"] = fmt_pct(chg_pct)
|
||||
|
||||
result["volume"] = meta.get("regularMarketVolume")
|
||||
result["52w_high"] = fmt_price(meta.get("fiftyTwoWeekHigh"))
|
||||
result["52w_low"] = fmt_price(meta.get("fiftyTwoWeekLow"))
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def extract_quote_summary_fields(qs_data: dict) -> dict:
|
||||
"""Extract PE, market cap, etc. from quoteSummary response."""
|
||||
out = {
|
||||
"market_cap": None,
|
||||
"pe_ratio": None,
|
||||
"52w_high": None,
|
||||
"52w_low": None,
|
||||
"volume": None,
|
||||
"short_name": None,
|
||||
}
|
||||
|
||||
result = safe_get(qs_data, "quoteSummary", "result")
|
||||
if not result or not isinstance(result, list) or len(result) == 0:
|
||||
return out
|
||||
|
||||
r = result[0]
|
||||
|
||||
# price module
|
||||
price_mod = r.get("price", {})
|
||||
out["market_cap"] = fmt_large(safe_get(price_mod, "marketCap", "raw"))
|
||||
out["short_name"] = price_mod.get("shortName") or price_mod.get("longName")
|
||||
|
||||
# summaryDetail
|
||||
sd = r.get("summaryDetail", {})
|
||||
pe_raw = safe_get(sd, "trailingPE", "raw")
|
||||
out["pe_ratio"] = fmt_price(pe_raw) if pe_raw else None
|
||||
out["52w_high"] = fmt_price(safe_get(sd, "fiftyTwoWeekHigh", "raw"))
|
||||
out["52w_low"] = fmt_price(safe_get(sd, "fiftyTwoWeekLow", "raw"))
|
||||
out["volume"] = safe_get(sd, "volume", "raw") or safe_get(sd, "regularMarketVolume", "raw")
|
||||
|
||||
# defaultKeyStatistics
|
||||
ks = r.get("defaultKeyStatistics", {})
|
||||
if out["pe_ratio"] is None:
|
||||
pe_raw = safe_get(ks, "trailingEps", "raw")
|
||||
# can't compute PE from EPS alone without price, skip
|
||||
|
||||
return out
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Command: quote
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def cmd_quote(symbols: list[str]) -> None:
|
||||
results = []
|
||||
|
||||
for sym in symbols:
|
||||
sym = sym.upper().strip()
|
||||
entry = {"symbol": sym, "data_source": "Yahoo Finance"}
|
||||
|
||||
# Fetch chart for price data
|
||||
chart_data = yf_chart(sym, interval="1d", range_="1d")
|
||||
if chart_data:
|
||||
q = extract_quote_from_chart(sym, chart_data)
|
||||
entry.update(q)
|
||||
|
||||
# Fetch quoteSummary for enriched data
|
||||
qs_data = yf_quote_summary(sym)
|
||||
if qs_data:
|
||||
qs_fields = extract_quote_summary_fields(qs_data)
|
||||
# Prefer quoteSummary values if chart didn't have them
|
||||
for field in ("market_cap", "pe_ratio", "52w_high", "52w_low", "volume", "short_name"):
|
||||
if entry.get(field) is None and qs_fields.get(field) is not None:
|
||||
entry[field] = qs_fields[field]
|
||||
elif field == "market_cap" and qs_fields.get(field) is not None:
|
||||
# Always prefer formatted market cap from quoteSummary
|
||||
entry[field] = qs_fields[field]
|
||||
|
||||
# Optionally enrich with Alpha Vantage
|
||||
av_key = os.environ.get("ALPHA_VANTAGE_KEY")
|
||||
if av_key:
|
||||
av_data = av_overview(sym)
|
||||
if av_data:
|
||||
entry["data_source"] = "Yahoo Finance + Alpha Vantage"
|
||||
if entry.get("pe_ratio") is None:
|
||||
pe = av_data.get("PERatio")
|
||||
entry["pe_ratio"] = pe if pe and pe != "None" and pe != "-" else None
|
||||
if entry.get("market_cap") is None:
|
||||
mc = av_data.get("MarketCapitalization")
|
||||
entry["market_cap"] = fmt_large(mc)
|
||||
if entry.get("52w_high") is None:
|
||||
entry["52w_high"] = av_data.get("52WeekHigh")
|
||||
if entry.get("52w_low") is None:
|
||||
entry["52w_low"] = av_data.get("52WeekLow")
|
||||
|
||||
results.append(entry)
|
||||
|
||||
if len(results) == 1:
|
||||
print_json(results[0])
|
||||
else:
|
||||
print_json(results)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Command: search
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def cmd_search(query: str) -> None:
|
||||
data = yf_search(query, count=5)
|
||||
if not data:
|
||||
print_json({"error": "Search failed or no results", "query": query, "data_source": "Yahoo Finance"})
|
||||
return
|
||||
|
||||
quotes = data.get("quotes") or []
|
||||
if not quotes:
|
||||
print_json({"error": "No matches found", "query": query, "data_source": "Yahoo Finance"})
|
||||
return
|
||||
|
||||
results = []
|
||||
for q in quotes[:5]:
|
||||
results.append({
|
||||
"symbol": q.get("symbol"),
|
||||
"name": q.get("longname") or q.get("shortname"),
|
||||
"exchange": q.get("exchange") or q.get("exchDisp"),
|
||||
"type": q.get("quoteType"),
|
||||
"sector": q.get("sector"),
|
||||
})
|
||||
|
||||
output = {
|
||||
"query": query,
|
||||
"matches": results,
|
||||
"data_source": "Yahoo Finance",
|
||||
}
|
||||
print_json(output)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Command: history
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def cmd_history(symbol: str, range_: str = "1mo") -> None:
|
||||
valid_ranges = ("1mo", "3mo", "6mo", "1y", "5y")
|
||||
if range_ not in valid_ranges:
|
||||
print_json({"error": f"Invalid range '{range_}'. Valid: {', '.join(valid_ranges)}"})
|
||||
return
|
||||
|
||||
sym = symbol.upper().strip()
|
||||
chart_data = yf_chart(sym, interval="1d", range_=range_)
|
||||
|
||||
if not chart_data:
|
||||
print_json({"error": f"Failed to fetch history for {sym}", "data_source": "Yahoo Finance"})
|
||||
return
|
||||
|
||||
chart = safe_get(chart_data, "chart", "result")
|
||||
if not chart or not isinstance(chart, list) or len(chart) == 0:
|
||||
err = safe_get(chart_data, "chart", "error", "description") or "Unknown error"
|
||||
print_json({"error": err, "symbol": sym, "data_source": "Yahoo Finance"})
|
||||
return
|
||||
|
||||
r = chart[0]
|
||||
timestamps = r.get("timestamp") or []
|
||||
indicators = r.get("indicators", {})
|
||||
quote_list = indicators.get("quote") or [{}]
|
||||
ohlcv = quote_list[0] if quote_list else {}
|
||||
|
||||
opens = ohlcv.get("open") or []
|
||||
closes = ohlcv.get("close") or []
|
||||
highs = ohlcv.get("high") or []
|
||||
lows = ohlcv.get("low") or []
|
||||
volumes = ohlcv.get("volume") or []
|
||||
|
||||
history = []
|
||||
for i, ts in enumerate(timestamps):
|
||||
def _v(lst, idx):
|
||||
try:
|
||||
val = lst[idx]
|
||||
return round(val, 2) if val is not None else None
|
||||
except IndexError:
|
||||
return None
|
||||
|
||||
entry = {
|
||||
"date": ts_to_date(ts),
|
||||
"open": _v(opens, i),
|
||||
"close": _v(closes, i),
|
||||
"high": _v(highs, i),
|
||||
"low": _v(lows, i),
|
||||
"volume": _v(volumes, i),
|
||||
}
|
||||
history.append(entry)
|
||||
|
||||
# Stats
|
||||
valid_closes = [c["close"] for c in history if c["close"] is not None]
|
||||
stats = {}
|
||||
if valid_closes:
|
||||
stats["min"] = fmt_price(min(valid_closes))
|
||||
stats["max"] = fmt_price(max(valid_closes))
|
||||
stats["avg"] = fmt_price(sum(valid_closes) / len(valid_closes))
|
||||
if len(valid_closes) >= 2:
|
||||
total_return = ((valid_closes[-1] - valid_closes[0]) / valid_closes[0]) * 100
|
||||
stats["total_return_pct"] = fmt_pct(total_return)
|
||||
else:
|
||||
stats["total_return_pct"] = None
|
||||
|
||||
meta = r.get("meta", {})
|
||||
output = {
|
||||
"symbol": sym,
|
||||
"range": range_,
|
||||
"currency": meta.get("currency"),
|
||||
"exchange": meta.get("exchangeName"),
|
||||
"data_points": len(history),
|
||||
"stats": stats,
|
||||
"history": history,
|
||||
"data_source": "Yahoo Finance",
|
||||
}
|
||||
print_json(output)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Command: compare
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def cmd_compare(symbols: list[str]) -> None:
|
||||
if len(symbols) < 2:
|
||||
print_json({"error": "compare requires at least 2 symbols"})
|
||||
return
|
||||
|
||||
comparisons = []
|
||||
|
||||
for sym in symbols:
|
||||
sym = sym.upper().strip()
|
||||
entry = {
|
||||
"symbol": sym,
|
||||
"name": None,
|
||||
"price": None,
|
||||
"change_pct": None,
|
||||
"market_cap": None,
|
||||
"pe_ratio": None,
|
||||
"52w_high": None,
|
||||
"52w_low": None,
|
||||
"52w_performance_pct": None,
|
||||
}
|
||||
|
||||
# Chart data
|
||||
chart_data = yf_chart(sym, interval="1d", range_="1d")
|
||||
if chart_data:
|
||||
q = extract_quote_from_chart(sym, chart_data)
|
||||
entry["name"] = q.get("short_name")
|
||||
entry["price"] = q.get("price")
|
||||
entry["change_pct"] = q.get("change_pct")
|
||||
entry["52w_high"] = q.get("52w_high")
|
||||
entry["52w_low"] = q.get("52w_low")
|
||||
|
||||
# quoteSummary for enrichment
|
||||
qs_data = yf_quote_summary(sym)
|
||||
if qs_data:
|
||||
qs = extract_quote_summary_fields(qs_data)
|
||||
if qs.get("market_cap"):
|
||||
entry["market_cap"] = qs["market_cap"]
|
||||
if qs.get("pe_ratio"):
|
||||
entry["pe_ratio"] = qs["pe_ratio"]
|
||||
if entry["52w_high"] is None and qs.get("52w_high"):
|
||||
entry["52w_high"] = qs["52w_high"]
|
||||
if entry["52w_low"] is None and qs.get("52w_low"):
|
||||
entry["52w_low"] = qs["52w_low"]
|
||||
if entry["name"] is None and qs.get("short_name"):
|
||||
entry["name"] = qs["short_name"]
|
||||
|
||||
# 52w performance: (current - 52w_low) / (52w_high - 52w_low)
|
||||
try:
|
||||
price_f = float(entry["price"]) if entry["price"] else None
|
||||
high_f = float(entry["52w_high"]) if entry["52w_high"] else None
|
||||
low_f = float(entry["52w_low"]) if entry["52w_low"] else None
|
||||
if price_f and low_f and price_f > 0 and low_f > 0:
|
||||
perf = ((price_f - low_f) / low_f) * 100
|
||||
entry["52w_performance_pct"] = fmt_pct(perf)
|
||||
except (ValueError, TypeError, ZeroDivisionError):
|
||||
pass
|
||||
|
||||
comparisons.append(entry)
|
||||
|
||||
output = {
|
||||
"comparison": comparisons,
|
||||
"symbols": [s.upper() for s in symbols],
|
||||
"data_source": "Yahoo Finance",
|
||||
}
|
||||
print_json(output)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Command: crypto
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def cmd_crypto(symbol: str, vs: str = "USD") -> None:
|
||||
sym = symbol.upper().strip()
|
||||
vs = vs.upper().strip()
|
||||
|
||||
# If user already passed BTC-USD, keep as-is; otherwise append
|
||||
if "-" not in sym:
|
||||
ticker = f"{sym}-{vs}"
|
||||
else:
|
||||
ticker = sym
|
||||
|
||||
chart_data = yf_chart(ticker, interval="1d", range_="1d")
|
||||
|
||||
if not chart_data:
|
||||
print_json({
|
||||
"error": f"Failed to fetch crypto data for {ticker}",
|
||||
"symbol": ticker,
|
||||
"data_source": "Yahoo Finance",
|
||||
})
|
||||
return
|
||||
|
||||
chart = safe_get(chart_data, "chart", "result")
|
||||
if not chart or not isinstance(chart, list) or len(chart) == 0:
|
||||
err = safe_get(chart_data, "chart", "error", "description") or "Symbol not found"
|
||||
print_json({"error": err, "symbol": ticker, "data_source": "Yahoo Finance"})
|
||||
return
|
||||
|
||||
r = chart[0]
|
||||
meta = r.get("meta", {})
|
||||
|
||||
price = meta.get("regularMarketPrice") or meta.get("chartPreviousClose")
|
||||
prev_close = meta.get("previousClose") or meta.get("chartPreviousClose")
|
||||
|
||||
change = None
|
||||
change_pct = None
|
||||
if price and prev_close:
|
||||
try:
|
||||
chg = float(price) - float(prev_close)
|
||||
chg_pct = (chg / float(prev_close)) * 100
|
||||
change = fmt_price(chg)
|
||||
change_pct = fmt_pct(chg_pct)
|
||||
except (TypeError, ValueError, ZeroDivisionError):
|
||||
pass
|
||||
|
||||
# 24h stats from indicators
|
||||
indicators = r.get("indicators", {})
|
||||
quote_list = indicators.get("quote") or [{}]
|
||||
ohlcv = quote_list[0] if quote_list else {}
|
||||
highs = [h for h in (ohlcv.get("high") or []) if h is not None]
|
||||
lows = [l for l in (ohlcv.get("low") or []) if l is not None]
|
||||
volumes = [v for v in (ohlcv.get("volume") or []) if v is not None]
|
||||
|
||||
output = {
|
||||
"symbol": ticker,
|
||||
"base": sym if "-" not in sym else sym.split("-")[0],
|
||||
"quote_currency": vs,
|
||||
"price": fmt_price(price),
|
||||
"change": change,
|
||||
"change_pct": change_pct,
|
||||
"day_high": fmt_price(max(highs)) if highs else None,
|
||||
"day_low": fmt_price(min(lows)) if lows else None,
|
||||
"volume": fmt_large(sum(volumes)) if volumes else None,
|
||||
"52w_high": fmt_price(meta.get("fiftyTwoWeekHigh")),
|
||||
"52w_low": fmt_price(meta.get("fiftyTwoWeekLow")),
|
||||
"exchange": meta.get("exchangeName"),
|
||||
"short_name": meta.get("shortName") or meta.get("longName"),
|
||||
"data_source": "Yahoo Finance",
|
||||
}
|
||||
print_json(output)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI entry point
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="stocks_client",
|
||||
description="Stock & crypto market data CLI — Hermes Agent",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog="""
|
||||
Examples:
|
||||
stocks_client.py quote AAPL MSFT GOOGL
|
||||
stocks_client.py search "Tesla"
|
||||
stocks_client.py history AAPL --range 3mo
|
||||
stocks_client.py compare AAPL MSFT GOOGL AMZN
|
||||
stocks_client.py crypto BTC
|
||||
stocks_client.py crypto ETH --vs EUR
|
||||
ALPHA_VANTAGE_KEY=yourkey stocks_client.py quote AAPL
|
||||
""",
|
||||
)
|
||||
|
||||
sub = parser.add_subparsers(dest="command", required=True)
|
||||
|
||||
# quote
|
||||
p_quote = sub.add_parser("quote", help="Get current quote for one or more symbols")
|
||||
p_quote.add_argument("symbols", nargs="+", metavar="SYMBOL", help="Stock ticker symbol(s)")
|
||||
|
||||
# search
|
||||
p_search = sub.add_parser("search", help="Search for stocks by name or symbol")
|
||||
p_search.add_argument("query", help="Search query (company name or partial symbol)")
|
||||
|
||||
# history
|
||||
p_history = sub.add_parser("history", help="Price history for a symbol")
|
||||
p_history.add_argument("symbol", metavar="SYMBOL", help="Stock ticker symbol")
|
||||
p_history.add_argument(
|
||||
"--range",
|
||||
dest="range_",
|
||||
default="1mo",
|
||||
choices=["1mo", "3mo", "6mo", "1y", "5y"],
|
||||
help="Date range (default: 1mo)",
|
||||
)
|
||||
|
||||
# compare
|
||||
p_compare = sub.add_parser("compare", help="Compare multiple stocks side by side")
|
||||
p_compare.add_argument("symbols", nargs="+", metavar="SYMBOL", help="At least 2 stock symbols")
|
||||
|
||||
# crypto
|
||||
p_crypto = sub.add_parser("crypto", help="Crypto price (BTC, ETH, SOL, etc.)")
|
||||
p_crypto.add_argument("symbol", metavar="SYMBOL", help="Crypto symbol (e.g. BTC, ETH, SOL)")
|
||||
p_crypto.add_argument(
|
||||
"--vs",
|
||||
default="USD",
|
||||
metavar="CURRENCY",
|
||||
help="Quote currency (default: USD)",
|
||||
)
|
||||
|
||||
return parser
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = build_parser()
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
if args.command == "quote":
|
||||
cmd_quote(args.symbols)
|
||||
elif args.command == "search":
|
||||
cmd_search(args.query)
|
||||
elif args.command == "history":
|
||||
cmd_history(args.symbol, range_=args.range_)
|
||||
elif args.command == "compare":
|
||||
cmd_compare(args.symbols)
|
||||
elif args.command == "crypto":
|
||||
cmd_crypto(args.symbol, vs=args.vs)
|
||||
else:
|
||||
parser.print_help()
|
||||
sys.exit(1)
|
||||
except KeyboardInterrupt:
|
||||
print_json({"error": "Interrupted by user"})
|
||||
sys.exit(130)
|
||||
except Exception as e:
|
||||
print_json({"error": f"Unexpected error: {e}", "type": type(e).__name__})
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1 @@
|
||||
Health, wellness, and biometric integration skills — BCI wearables, neurofeedback, sleep tracking, and cognitive state monitoring.
|
||||
@@ -0,0 +1,256 @@
|
||||
---
|
||||
name: fitness-nutrition
|
||||
description: >
|
||||
Gym workout planner and nutrition tracker. Search 690+ exercises by muscle,
|
||||
equipment, or category via wger. Look up macros and calories for 380,000+
|
||||
foods via USDA FoodData Central. Compute BMI, TDEE, one-rep max, macro
|
||||
splits, and body fat — pure Python, no pip installs. Built for anyone
|
||||
chasing gains, cutting weight, or just trying to eat better.
|
||||
platforms: [linux, macos, windows]
|
||||
version: 1.0.0
|
||||
authors:
|
||||
- haileymarshall
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [health, fitness, nutrition, gym, workout, diet, exercise]
|
||||
category: health
|
||||
prerequisites:
|
||||
commands: [curl, python3]
|
||||
required_environment_variables:
|
||||
- name: USDA_API_KEY
|
||||
prompt: "USDA FoodData Central API key (free)"
|
||||
help: "Get one free at https://fdc.nal.usda.gov/api-key-signup/ — or skip to use DEMO_KEY with lower rate limits"
|
||||
required_for: "higher rate limits on food/nutrition lookups (DEMO_KEY works without signup)"
|
||||
optional: true
|
||||
---
|
||||
|
||||
# Fitness & Nutrition
|
||||
|
||||
Expert fitness coach and sports nutritionist skill. Two data sources
|
||||
plus offline calculators — everything a gym-goer needs in one place.
|
||||
|
||||
**Data sources (all free, no pip dependencies):**
|
||||
|
||||
- **wger** (https://wger.de/api/v2/) — open exercise database, 690+ exercises with muscles, equipment, images. Public endpoints need zero authentication.
|
||||
- **USDA FoodData Central** (https://api.nal.usda.gov/fdc/v1/) — US government nutrition database, 380,000+ foods. `DEMO_KEY` works instantly; free signup for higher limits.
|
||||
|
||||
**Offline calculators (pure stdlib Python):**
|
||||
|
||||
- BMI, TDEE (Mifflin-St Jeor), one-rep max (Epley/Brzycki/Lombardi), macro splits, body fat % (US Navy method)
|
||||
|
||||
---
|
||||
|
||||
## When to Use
|
||||
|
||||
Trigger this skill when the user asks about:
|
||||
- Exercises, workouts, gym routines, muscle groups, workout splits
|
||||
- Food macros, calories, protein content, meal planning, calorie counting
|
||||
- Body composition: BMI, body fat, TDEE, caloric surplus/deficit
|
||||
- One-rep max estimates, training percentages, progressive overload
|
||||
- Macro ratios for cutting, bulking, or maintenance
|
||||
|
||||
---
|
||||
|
||||
## Procedure
|
||||
|
||||
### Exercise Lookup (wger API)
|
||||
|
||||
All wger public endpoints return JSON and require no auth. Always add
|
||||
`format=json` and `language=2` (English) to exercise queries.
|
||||
|
||||
**Step 1 — Identify what the user wants:**
|
||||
|
||||
- By muscle → use `/api/v2/exercise/?muscles={id}&language=2&status=2&format=json`
|
||||
- By category → use `/api/v2/exercise/?category={id}&language=2&status=2&format=json`
|
||||
- By equipment → use `/api/v2/exercise/?equipment={id}&language=2&status=2&format=json`
|
||||
- By name → use `/api/v2/exercise/search/?term={query}&language=english&format=json`
|
||||
- Full details → use `/api/v2/exerciseinfo/{exercise_id}/?format=json`
|
||||
|
||||
**Step 2 — Reference IDs (so you don't need extra API calls):**
|
||||
|
||||
Exercise categories:
|
||||
|
||||
| ID | Category |
|
||||
|----|-------------|
|
||||
| 8 | Arms |
|
||||
| 9 | Legs |
|
||||
| 10 | Abs |
|
||||
| 11 | Chest |
|
||||
| 12 | Back |
|
||||
| 13 | Shoulders |
|
||||
| 14 | Calves |
|
||||
| 15 | Cardio |
|
||||
|
||||
Muscles:
|
||||
|
||||
| ID | Muscle | ID | Muscle |
|
||||
|----|---------------------------|----|-------------------------|
|
||||
| 1 | Biceps brachii | 2 | Anterior deltoid |
|
||||
| 3 | Serratus anterior | 4 | Pectoralis major |
|
||||
| 5 | Obliquus externus | 6 | Gastrocnemius |
|
||||
| 7 | Rectus abdominis | 8 | Gluteus maximus |
|
||||
| 9 | Trapezius | 10 | Quadriceps femoris |
|
||||
| 11 | Biceps femoris | 12 | Latissimus dorsi |
|
||||
| 13 | Brachialis | 14 | Triceps brachii |
|
||||
| 15 | Soleus | | |
|
||||
|
||||
Equipment:
|
||||
|
||||
| ID | Equipment |
|
||||
|----|----------------|
|
||||
| 1 | Barbell |
|
||||
| 3 | Dumbbell |
|
||||
| 4 | Gym mat |
|
||||
| 5 | Swiss Ball |
|
||||
| 6 | Pull-up bar |
|
||||
| 7 | none (bodyweight) |
|
||||
| 8 | Bench |
|
||||
| 9 | Incline bench |
|
||||
| 10 | Kettlebell |
|
||||
|
||||
**Step 3 — Fetch and present results:**
|
||||
|
||||
```bash
|
||||
# Search exercises by name
|
||||
QUERY="$1"
|
||||
ENCODED=$(python3 -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1]))" "$QUERY")
|
||||
curl -s "https://wger.de/api/v2/exercise/search/?term=${ENCODED}&language=english&format=json" \
|
||||
| python3 -c "
|
||||
import json,sys
|
||||
data=json.load(sys.stdin)
|
||||
for s in data.get('suggestions',[])[:10]:
|
||||
d=s.get('data',{})
|
||||
print(f\" ID {d.get('id','?'):>4} | {d.get('name','N/A'):<35} | Category: {d.get('category','N/A')}\")
|
||||
"
|
||||
```
|
||||
|
||||
```bash
|
||||
# Get full details for a specific exercise
|
||||
EXERCISE_ID="$1"
|
||||
curl -s "https://wger.de/api/v2/exerciseinfo/${EXERCISE_ID}/?format=json" \
|
||||
| python3 -c "
|
||||
import json,sys,html,re
|
||||
data=json.load(sys.stdin)
|
||||
trans=[t for t in data.get('translations',[]) if t.get('language')==2]
|
||||
t=trans[0] if trans else data.get('translations',[{}])[0]
|
||||
desc=re.sub('<[^>]+>','',html.unescape(t.get('description','N/A')))
|
||||
print(f\"Exercise : {t.get('name','N/A')}\")
|
||||
print(f\"Category : {data.get('category',{}).get('name','N/A')}\")
|
||||
print(f\"Primary : {', '.join(m.get('name_en','') for m in data.get('muscles',[])) or 'N/A'}\")
|
||||
print(f\"Secondary : {', '.join(m.get('name_en','') for m in data.get('muscles_secondary',[])) or 'none'}\")
|
||||
print(f\"Equipment : {', '.join(e.get('name','') for e in data.get('equipment',[])) or 'bodyweight'}\")
|
||||
print(f\"How to : {desc[:500]}\")
|
||||
imgs=data.get('images',[])
|
||||
if imgs: print(f\"Image : {imgs[0].get('image','')}\")
|
||||
"
|
||||
```
|
||||
|
||||
```bash
|
||||
# List exercises filtering by muscle, category, or equipment
|
||||
# Combine filters as needed: ?muscles=4&equipment=1&language=2&status=2
|
||||
FILTER="$1" # e.g. "muscles=4" or "category=11" or "equipment=3"
|
||||
curl -s "https://wger.de/api/v2/exercise/?${FILTER}&language=2&status=2&limit=20&format=json" \
|
||||
| python3 -c "
|
||||
import json,sys
|
||||
data=json.load(sys.stdin)
|
||||
print(f'Found {data.get(\"count\",0)} exercises.')
|
||||
for ex in data.get('results',[]):
|
||||
print(f\" ID {ex['id']:>4} | muscles: {ex.get('muscles',[])} | equipment: {ex.get('equipment',[])}\")
|
||||
"
|
||||
```
|
||||
|
||||
### Nutrition Lookup (USDA FoodData Central)
|
||||
|
||||
Uses `USDA_API_KEY` env var if set, otherwise falls back to `DEMO_KEY`.
|
||||
DEMO_KEY = 30 requests/hour. Free signup key = 1,000 requests/hour.
|
||||
|
||||
```bash
|
||||
# Search foods by name
|
||||
FOOD="$1"
|
||||
API_KEY="${USDA_API_KEY:-DEMO_KEY}"
|
||||
ENCODED=$(python3 -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1]))" "$FOOD")
|
||||
curl -s "https://api.nal.usda.gov/fdc/v1/foods/search?api_key=${API_KEY}&query=${ENCODED}&pageSize=5&dataType=Foundation,SR%20Legacy" \
|
||||
| python3 -c "
|
||||
import json,sys
|
||||
data=json.load(sys.stdin)
|
||||
foods=data.get('foods',[])
|
||||
if not foods: print('No foods found.'); sys.exit()
|
||||
for f in foods:
|
||||
n={x['nutrientName']:x.get('value','?') for x in f.get('foodNutrients',[])}
|
||||
cal=n.get('Energy','?'); prot=n.get('Protein','?')
|
||||
fat=n.get('Total lipid (fat)','?'); carb=n.get('Carbohydrate, by difference','?')
|
||||
print(f\"{f.get('description','N/A')}\")
|
||||
print(f\" Per 100g: {cal} kcal | {prot}g protein | {fat}g fat | {carb}g carbs\")
|
||||
print(f\" FDC ID: {f.get('fdcId','N/A')}\")
|
||||
print()
|
||||
"
|
||||
```
|
||||
|
||||
```bash
|
||||
# Detailed nutrient profile by FDC ID
|
||||
FDC_ID="$1"
|
||||
API_KEY="${USDA_API_KEY:-DEMO_KEY}"
|
||||
curl -s "https://api.nal.usda.gov/fdc/v1/food/${FDC_ID}?api_key=${API_KEY}" \
|
||||
| python3 -c "
|
||||
import json,sys
|
||||
d=json.load(sys.stdin)
|
||||
print(f\"Food: {d.get('description','N/A')}\")
|
||||
print(f\"{'Nutrient':<40} {'Amount':>8} {'Unit'}\")
|
||||
print('-'*56)
|
||||
for x in sorted(d.get('foodNutrients',[]),key=lambda x:x.get('nutrient',{}).get('rank',9999)):
|
||||
nut=x.get('nutrient',{}); amt=x.get('amount',0)
|
||||
if amt and float(amt)>0:
|
||||
print(f\" {nut.get('name',''):<38} {amt:>8} {nut.get('unitName','')}\")
|
||||
"
|
||||
```
|
||||
|
||||
### Offline Calculators
|
||||
|
||||
Use the helper scripts in `scripts/` for batch operations,
|
||||
or run inline for single calculations:
|
||||
|
||||
- `python3 scripts/body_calc.py bmi <weight_kg> <height_cm>`
|
||||
- `python3 scripts/body_calc.py tdee <weight_kg> <height_cm> <age> <M|F> <activity 1-5>`
|
||||
- `python3 scripts/body_calc.py 1rm <weight> <reps>`
|
||||
- `python3 scripts/body_calc.py macros <tdee_kcal> <cut|maintain|bulk>`
|
||||
- `python3 scripts/body_calc.py bodyfat <M|F> <neck_cm> <waist_cm> [hip_cm] <height_cm>`
|
||||
|
||||
See `references/FORMULAS.md` for the science behind each formula.
|
||||
|
||||
---
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- wger exercise endpoint returns **all languages by default** — always add `language=2` for English
|
||||
- wger includes **unverified user submissions** — add `status=2` to only get approved exercises
|
||||
- USDA `DEMO_KEY` has **30 req/hour** — add `sleep 2` between batch requests or get a free key
|
||||
- USDA data is **per 100g** — remind users to scale to their actual portion size
|
||||
- BMI does not distinguish muscle from fat — high BMI in muscular people is not necessarily unhealthy
|
||||
- Body fat formulas are **estimates** (±3-5%) — recommend DEXA scans for precision
|
||||
- 1RM formulas lose accuracy above 10 reps — use sets of 3-5 for best estimates
|
||||
- wger's `exercise/search` endpoint uses `term` not `query` as the parameter name
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
After running exercise search: confirm results include exercise names, muscle groups, and equipment.
|
||||
After nutrition lookup: confirm per-100g macros are returned with kcal, protein, fat, carbs.
|
||||
After calculators: sanity-check outputs (e.g. TDEE should be 1500-3500 for most adults).
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Task | Source | Endpoint |
|
||||
|------|--------|----------|
|
||||
| Search exercises by name | wger | `GET /api/v2/exercise/search/?term=&language=english` |
|
||||
| Exercise details | wger | `GET /api/v2/exerciseinfo/{id}/` |
|
||||
| Filter by muscle | wger | `GET /api/v2/exercise/?muscles={id}&language=2&status=2` |
|
||||
| Filter by equipment | wger | `GET /api/v2/exercise/?equipment={id}&language=2&status=2` |
|
||||
| List categories | wger | `GET /api/v2/exercisecategory/` |
|
||||
| List muscles | wger | `GET /api/v2/muscle/` |
|
||||
| Search foods | USDA | `GET /fdc/v1/foods/search?query=&dataType=Foundation,SR Legacy` |
|
||||
| Food details | USDA | `GET /fdc/v1/food/{fdcId}` |
|
||||
| BMI / TDEE / 1RM / macros | offline | `python3 scripts/body_calc.py` |
|
||||
@@ -0,0 +1,100 @@
|
||||
# Formulas Reference
|
||||
|
||||
Scientific references for all calculators used in the fitness-nutrition skill.
|
||||
|
||||
## BMI (Body Mass Index)
|
||||
|
||||
**Formula:** BMI = weight (kg) / height (m)²
|
||||
|
||||
| Category | BMI Range |
|
||||
|-------------|------------|
|
||||
| Underweight | < 18.5 |
|
||||
| Normal | 18.5 – 24.9 |
|
||||
| Overweight | 25.0 – 29.9 |
|
||||
| Obese | 30.0+ |
|
||||
|
||||
**Limitation:** BMI does not distinguish muscle from fat. A muscular person
|
||||
can have a high BMI while being lean. Use body fat % for a better picture.
|
||||
|
||||
Reference: Quetelet, A. (1832). Keys et al., Int J Obes (1972).
|
||||
|
||||
## TDEE (Total Daily Energy Expenditure)
|
||||
|
||||
Uses the **Mifflin-St Jeor equation** — the most accurate BMR predictor for
|
||||
the general population according to the ADA (2005).
|
||||
|
||||
**BMR formulas:**
|
||||
|
||||
- Male: BMR = 10 × weight(kg) + 6.25 × height(cm) − 5 × age + 5
|
||||
- Female: BMR = 10 × weight(kg) + 6.25 × height(cm) − 5 × age − 161
|
||||
|
||||
**Activity multipliers:**
|
||||
|
||||
| Level | Description | Multiplier |
|
||||
|-------|--------------------------------|------------|
|
||||
| 1 | Sedentary (desk job) | 1.200 |
|
||||
| 2 | Lightly active (1-3 days/wk) | 1.375 |
|
||||
| 3 | Moderately active (3-5 days) | 1.550 |
|
||||
| 4 | Very active (6-7 days) | 1.725 |
|
||||
| 5 | Extremely active (2x/day) | 1.900 |
|
||||
|
||||
Reference: Mifflin et al., Am J Clin Nutr 51, 241-247 (1990).
|
||||
|
||||
## One-Rep Max (1RM)
|
||||
|
||||
Three validated formulas. Average of all three is most reliable.
|
||||
|
||||
- **Epley:** 1RM = w × (1 + r/30)
|
||||
- **Brzycki:** 1RM = w × 36 / (37 − r)
|
||||
- **Lombardi:** 1RM = w × r^0.1
|
||||
|
||||
All formulas are most accurate for r ≤ 10. Above 10 reps, error increases.
|
||||
|
||||
Reference: LeSuer et al., J Strength Cond Res 11(4), 211-213 (1997).
|
||||
|
||||
## Macro Splits
|
||||
|
||||
Recommended splits based on goal:
|
||||
|
||||
| Goal | Protein | Fat | Carbs | Calorie Offset |
|
||||
|-------------|---------|------|-------|----------------|
|
||||
| Fat loss | 40% | 30% | 30% | −500 kcal |
|
||||
| Maintenance | 30% | 30% | 40% | 0 |
|
||||
| Lean bulk | 30% | 25% | 45% | +400 kcal |
|
||||
|
||||
Protein targets for muscle growth: 1.6–2.2 g/kg body weight per day.
|
||||
Minimum fat intake: 0.5 g/kg to support hormone production.
|
||||
|
||||
Conversion: Protein = 4 kcal/g, Fat = 9 kcal/g, Carbs = 4 kcal/g.
|
||||
|
||||
Reference: Morton et al., Br J Sports Med 52, 376–384 (2018).
|
||||
|
||||
## Body Fat % (US Navy Method)
|
||||
|
||||
**Male:**
|
||||
|
||||
BF% = 86.010 × log₁₀(waist − neck) − 70.041 × log₁₀(height) + 36.76
|
||||
|
||||
**Female:**
|
||||
|
||||
BF% = 163.205 × log₁₀(waist + hip − neck) − 97.684 × log₁₀(height) − 78.387
|
||||
|
||||
All measurements in centimeters.
|
||||
|
||||
| Category | Male | Female |
|
||||
|--------------|--------|--------|
|
||||
| Essential | 2-5% | 10-13% |
|
||||
| Athletic | 6-13% | 14-20% |
|
||||
| Fitness | 14-17% | 21-24% |
|
||||
| Average | 18-24% | 25-31% |
|
||||
| Obese | 25%+ | 32%+ |
|
||||
|
||||
Accuracy: ±3-5% compared to DEXA. Measure at the navel (waist),
|
||||
at the Adam's apple (neck), and widest point (hip, females only).
|
||||
|
||||
Reference: Hodgdon & Beckett, Naval Health Research Center (1984).
|
||||
|
||||
## APIs
|
||||
|
||||
- wger: https://wger.de/api/v2/ — AGPL-3.0, exercise data is CC-BY-SA 3.0
|
||||
- USDA FoodData Central: https://api.nal.usda.gov/fdc/v1/ — public domain (CC0 1.0)
|
||||
@@ -0,0 +1,210 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
body_calc.py — All-in-one fitness calculator.
|
||||
|
||||
Subcommands:
|
||||
bmi <weight_kg> <height_cm>
|
||||
tdee <weight_kg> <height_cm> <age> <M|F> <activity 1-5>
|
||||
1rm <weight> <reps>
|
||||
macros <tdee_kcal> <cut|maintain|bulk>
|
||||
bodyfat <M|F> <neck_cm> <waist_cm> [hip_cm] <height_cm>
|
||||
|
||||
No external dependencies — stdlib only.
|
||||
"""
|
||||
import sys
|
||||
import math
|
||||
|
||||
|
||||
def bmi(weight_kg, height_cm):
|
||||
h = height_cm / 100
|
||||
val = weight_kg / (h * h)
|
||||
if val < 18.5:
|
||||
cat = "Underweight"
|
||||
elif val < 25:
|
||||
cat = "Normal weight"
|
||||
elif val < 30:
|
||||
cat = "Overweight"
|
||||
else:
|
||||
cat = "Obese"
|
||||
print(f"BMI: {val:.1f} — {cat}")
|
||||
print()
|
||||
print("Ranges:")
|
||||
print(f" Underweight : < 18.5")
|
||||
print(f" Normal : 18.5 – 24.9")
|
||||
print(f" Overweight : 25.0 – 29.9")
|
||||
print(f" Obese : 30.0+")
|
||||
|
||||
|
||||
def tdee(weight_kg, height_cm, age, sex, activity):
|
||||
if sex.upper() == "M":
|
||||
bmr = 10 * weight_kg + 6.25 * height_cm - 5 * age + 5
|
||||
else:
|
||||
bmr = 10 * weight_kg + 6.25 * height_cm - 5 * age - 161
|
||||
|
||||
multipliers = {
|
||||
1: ("Sedentary (desk job, no exercise)", 1.2),
|
||||
2: ("Lightly active (1-3 days/week)", 1.375),
|
||||
3: ("Moderately active (3-5 days/week)", 1.55),
|
||||
4: ("Very active (6-7 days/week)", 1.725),
|
||||
5: ("Extremely active (athlete + physical job)", 1.9),
|
||||
}
|
||||
|
||||
label, mult = multipliers.get(activity, ("Moderate", 1.55))
|
||||
total = bmr * mult
|
||||
|
||||
print(f"BMR (Mifflin-St Jeor): {bmr:.0f} kcal/day")
|
||||
print(f"Activity: {label} (x{mult})")
|
||||
print(f"TDEE: {total:.0f} kcal/day")
|
||||
print()
|
||||
print("Calorie targets:")
|
||||
print(f" Aggressive cut (-750): {total - 750:.0f} kcal/day")
|
||||
print(f" Fat loss (-500): {total - 500:.0f} kcal/day")
|
||||
print(f" Mild cut (-250): {total - 250:.0f} kcal/day")
|
||||
print(f" Maintenance : {total:.0f} kcal/day")
|
||||
print(f" Lean bulk (+250): {total + 250:.0f} kcal/day")
|
||||
print(f" Bulk (+500): {total + 500:.0f} kcal/day")
|
||||
|
||||
|
||||
def one_rep_max(weight, reps):
|
||||
if reps < 1:
|
||||
print("Error: reps must be at least 1.")
|
||||
sys.exit(1)
|
||||
if reps == 1:
|
||||
print(f"1RM = {weight:.1f} (actual single)")
|
||||
return
|
||||
|
||||
epley = weight * (1 + reps / 30)
|
||||
brzycki = weight * (36 / (37 - reps)) if reps < 37 else 0
|
||||
lombardi = weight * (reps ** 0.1)
|
||||
avg = (epley + brzycki + lombardi) / 3
|
||||
|
||||
print(f"Estimated 1RM ({weight} x {reps} reps):")
|
||||
print(f" Epley : {epley:.1f}")
|
||||
print(f" Brzycki : {brzycki:.1f}")
|
||||
print(f" Lombardi : {lombardi:.1f}")
|
||||
print(f" Average : {avg:.1f}")
|
||||
print()
|
||||
print("Training percentages off average 1RM:")
|
||||
for pct, rep_range in [
|
||||
(100, "1"), (95, "1-2"), (90, "3-4"), (85, "4-6"),
|
||||
(80, "6-8"), (75, "8-10"), (70, "10-12"),
|
||||
(65, "12-15"), (60, "15-20"),
|
||||
]:
|
||||
print(f" {pct:>3}% = {avg * pct / 100:>7.1f} (~{rep_range} reps)")
|
||||
|
||||
|
||||
def macros(tdee_kcal, goal):
|
||||
goal = goal.lower()
|
||||
if goal in {"cut", "lose", "deficit"}:
|
||||
cals = tdee_kcal - 500
|
||||
p, f, c = 0.40, 0.30, 0.30
|
||||
label = "Fat Loss (-500 kcal)"
|
||||
elif goal in {"bulk", "gain", "surplus"}:
|
||||
cals = tdee_kcal + 400
|
||||
p, f, c = 0.30, 0.25, 0.45
|
||||
label = "Lean Bulk (+400 kcal)"
|
||||
else:
|
||||
cals = tdee_kcal
|
||||
p, f, c = 0.30, 0.30, 0.40
|
||||
label = "Maintenance"
|
||||
|
||||
prot_g = cals * p / 4
|
||||
fat_g = cals * f / 9
|
||||
carb_g = cals * c / 4
|
||||
|
||||
print(f"Goal: {label}")
|
||||
print(f"Daily calories: {cals:.0f} kcal")
|
||||
print()
|
||||
print(f" Protein : {prot_g:>6.0f}g ({p * 100:.0f}%) = {prot_g * 4:.0f} kcal")
|
||||
print(f" Fat : {fat_g:>6.0f}g ({f * 100:.0f}%) = {fat_g * 9:.0f} kcal")
|
||||
print(f" Carbs : {carb_g:>6.0f}g ({c * 100:.0f}%) = {carb_g * 4:.0f} kcal")
|
||||
print()
|
||||
print(f"Per meal (3 meals): P {prot_g / 3:.0f}g | F {fat_g / 3:.0f}g | C {carb_g / 3:.0f}g")
|
||||
print(f"Per meal (4 meals): P {prot_g / 4:.0f}g | F {fat_g / 4:.0f}g | C {carb_g / 4:.0f}g")
|
||||
|
||||
|
||||
def bodyfat(sex, neck_cm, waist_cm, hip_cm, height_cm):
|
||||
sex = sex.upper()
|
||||
if sex == "M":
|
||||
if waist_cm <= neck_cm:
|
||||
print("Error: waist must be larger than neck."); sys.exit(1)
|
||||
bf = 86.010 * math.log10(waist_cm - neck_cm) - 70.041 * math.log10(height_cm) + 36.76
|
||||
else:
|
||||
if (waist_cm + hip_cm) <= neck_cm:
|
||||
print("Error: waist + hip must be larger than neck."); sys.exit(1)
|
||||
bf = 163.205 * math.log10(waist_cm + hip_cm - neck_cm) - 97.684 * math.log10(height_cm) - 78.387
|
||||
|
||||
print(f"Estimated body fat: {bf:.1f}%")
|
||||
|
||||
if sex == "M":
|
||||
ranges = [
|
||||
(6, "Essential fat (2-5%)"),
|
||||
(14, "Athletic (6-13%)"),
|
||||
(18, "Fitness (14-17%)"),
|
||||
(25, "Average (18-24%)"),
|
||||
]
|
||||
default = "Obese (25%+)"
|
||||
else:
|
||||
ranges = [
|
||||
(14, "Essential fat (10-13%)"),
|
||||
(21, "Athletic (14-20%)"),
|
||||
(25, "Fitness (21-24%)"),
|
||||
(32, "Average (25-31%)"),
|
||||
]
|
||||
default = "Obese (32%+)"
|
||||
|
||||
cat = default
|
||||
for threshold, label in ranges:
|
||||
if bf < threshold:
|
||||
cat = label
|
||||
break
|
||||
|
||||
print(f"Category: {cat}")
|
||||
print(f"Method: US Navy circumference formula")
|
||||
|
||||
|
||||
def usage():
|
||||
print(__doc__)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
usage()
|
||||
|
||||
cmd = sys.argv[1].lower()
|
||||
|
||||
try:
|
||||
if cmd == "bmi":
|
||||
bmi(float(sys.argv[2]), float(sys.argv[3]))
|
||||
|
||||
elif cmd == "tdee":
|
||||
tdee(
|
||||
float(sys.argv[2]), float(sys.argv[3]),
|
||||
int(sys.argv[4]), sys.argv[5], int(sys.argv[6]),
|
||||
)
|
||||
|
||||
elif cmd in {"1rm", "orm"}:
|
||||
one_rep_max(float(sys.argv[2]), int(sys.argv[3]))
|
||||
|
||||
elif cmd == "macros":
|
||||
macros(float(sys.argv[2]), sys.argv[3])
|
||||
|
||||
elif cmd == "bodyfat":
|
||||
sex = sys.argv[2]
|
||||
if sex.upper() == "M":
|
||||
bodyfat(sex, float(sys.argv[3]), float(sys.argv[4]), 0, float(sys.argv[5]))
|
||||
else:
|
||||
bodyfat(sex, float(sys.argv[3]), float(sys.argv[4]), float(sys.argv[5]), float(sys.argv[6]))
|
||||
|
||||
else:
|
||||
print(f"Unknown command: {cmd}")
|
||||
usage()
|
||||
|
||||
except (IndexError, ValueError) as e:
|
||||
print(f"Error: {e}")
|
||||
usage()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,85 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
nutrition_search.py — Search USDA FoodData Central for nutrition info.
|
||||
|
||||
Usage:
|
||||
python3 nutrition_search.py "chicken breast"
|
||||
python3 nutrition_search.py "rice" "eggs" "broccoli"
|
||||
echo -e "oats\\nbanana\\nwhey protein" | python3 nutrition_search.py -
|
||||
|
||||
Reads USDA_API_KEY from environment, falls back to DEMO_KEY.
|
||||
No external dependencies.
|
||||
"""
|
||||
import sys
|
||||
import os
|
||||
import json
|
||||
import time
|
||||
import urllib.request
|
||||
import urllib.parse
|
||||
|
||||
API_KEY = os.environ.get("USDA_API_KEY", "DEMO_KEY")
|
||||
BASE = "https://api.nal.usda.gov/fdc/v1"
|
||||
|
||||
|
||||
def search(query, max_results=3):
|
||||
encoded = urllib.parse.quote(query)
|
||||
url = (
|
||||
f"{BASE}/foods/search?api_key={API_KEY}"
|
||||
f"&query={encoded}&pageSize={max_results}"
|
||||
f"&dataType=Foundation,SR%20Legacy"
|
||||
)
|
||||
try:
|
||||
req = urllib.request.Request(url, headers={"Accept": "application/json"})
|
||||
with urllib.request.urlopen(req, timeout=15) as r:
|
||||
return json.loads(r.read())
|
||||
except Exception as e:
|
||||
print(f" API error: {e}", file=sys.stderr)
|
||||
return None
|
||||
|
||||
|
||||
def display(food):
|
||||
nutrients = {n["nutrientName"]: n.get("value", "?") for n in food.get("foodNutrients", [])}
|
||||
cal = nutrients.get("Energy", "?")
|
||||
prot = nutrients.get("Protein", "?")
|
||||
fat = nutrients.get("Total lipid (fat)", "?")
|
||||
carb = nutrients.get("Carbohydrate, by difference", "?")
|
||||
fib = nutrients.get("Fiber, total dietary", "?")
|
||||
sug = nutrients.get("Sugars, total including NLEA", "?")
|
||||
|
||||
print(f" {food.get('description', 'N/A')}")
|
||||
print(f" Calories : {cal} kcal")
|
||||
print(f" Protein : {prot}g")
|
||||
print(f" Fat : {fat}g")
|
||||
print(f" Carbs : {carb}g (fiber: {fib}g, sugar: {sug}g)")
|
||||
print(f" FDC ID : {food.get('fdcId', 'N/A')}")
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print(__doc__)
|
||||
sys.exit(1)
|
||||
|
||||
if sys.argv[1] == "-":
|
||||
queries = [line.strip() for line in sys.stdin if line.strip()]
|
||||
else:
|
||||
queries = sys.argv[1:]
|
||||
|
||||
for query in queries:
|
||||
print(f"\n--- {query.upper()} (per 100g) ---")
|
||||
data = search(query, max_results=2)
|
||||
if not data or not data.get("foods"):
|
||||
print(" No results found.")
|
||||
else:
|
||||
for food in data["foods"]:
|
||||
display(food)
|
||||
print()
|
||||
if len(queries) > 1:
|
||||
time.sleep(1) # respect rate limits
|
||||
|
||||
if API_KEY == "DEMO_KEY":
|
||||
print("\nTip: using DEMO_KEY (30 req/hr). Set USDA_API_KEY for 1000 req/hr.")
|
||||
print("Free signup: https://fdc.nal.usda.gov/api-key-signup/")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,459 @@
|
||||
---
|
||||
name: neuroskill-bci
|
||||
description: >
|
||||
Connect to a running NeuroSkill instance and incorporate the user's real-time
|
||||
cognitive and emotional state (focus, relaxation, mood, cognitive load, drowsiness,
|
||||
heart rate, HRV, sleep staging, and 40+ derived EXG scores) into responses.
|
||||
Requires a BCI wearable (Muse 2/S or OpenBCI) and the NeuroSkill desktop app
|
||||
running locally.
|
||||
platforms: [linux, macos, windows]
|
||||
version: 1.0.0
|
||||
author: Hermes Agent + Nous Research
|
||||
license: MIT
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [BCI, neurofeedback, health, focus, EEG, cognitive-state, biometrics, neuroskill]
|
||||
category: health
|
||||
related_skills: []
|
||||
---
|
||||
|
||||
# NeuroSkill BCI Integration
|
||||
|
||||
Connect Hermes to a running [NeuroSkill](https://neuroskill.com/) instance to read
|
||||
real-time brain and body metrics from a BCI wearable. Use this to give
|
||||
cognitively-aware responses, suggest interventions, and track mental performance
|
||||
over time.
|
||||
|
||||
> **⚠️ Research Use Only** — NeuroSkill is an open-source research tool. It is
|
||||
> NOT a medical device and has NOT been cleared by the FDA, CE, or any regulatory
|
||||
> body. Never use these metrics for clinical diagnosis or treatment.
|
||||
|
||||
See `references/metrics.md` for the full metric reference, `references/protocols.md`
|
||||
for intervention protocols, and `references/api.md` for the WebSocket/HTTP API.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Node.js 20+** installed (`node --version`)
|
||||
- **NeuroSkill desktop app** running with a connected BCI device
|
||||
- **BCI hardware**: Muse 2, Muse S, or OpenBCI (4-channel EEG + PPG + IMU via BLE)
|
||||
- `npx neuroskill status` returns data without errors
|
||||
|
||||
### Verify Setup
|
||||
```bash
|
||||
node --version # Must be 20+
|
||||
npx neuroskill status # Full system snapshot
|
||||
npx neuroskill status --json # Machine-parseable JSON
|
||||
```
|
||||
|
||||
If `npx neuroskill status` returns an error, tell the user:
|
||||
- Make sure the NeuroSkill desktop app is open
|
||||
- Ensure the BCI device is powered on and connected via Bluetooth
|
||||
- Check signal quality — green indicators in NeuroSkill (≥0.7 per electrode)
|
||||
- If `command not found`, install Node.js 20+
|
||||
|
||||
---
|
||||
|
||||
## CLI Reference: `npx neuroskill <command>`
|
||||
|
||||
All commands support `--json` (raw JSON, pipe-safe) and `--full` (human summary + JSON).
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `status` | Full system snapshot: device, scores, bands, ratios, sleep, history |
|
||||
| `session [N]` | Single session breakdown with first/second half trends (0=most recent) |
|
||||
| `sessions` | List all recorded sessions across all days |
|
||||
| `search` | ANN similarity search for neurally similar historical moments |
|
||||
| `compare` | A/B session comparison with metric deltas and trend analysis |
|
||||
| `sleep [N]` | Sleep stage classification (Wake/N1/N2/N3/REM) with analysis |
|
||||
| `label "text"` | Create a timestamped annotation at the current moment |
|
||||
| `search-labels "query"` | Semantic vector search over past labels |
|
||||
| `interactive "query"` | Cross-modal 4-layer graph search (text → EXG → labels) |
|
||||
| `listen` | Real-time event streaming (default 5s, set `--seconds N`) |
|
||||
| `umap` | 3D UMAP projection of session embeddings |
|
||||
| `calibrate` | Open calibration window and start a profile |
|
||||
| `timer` | Launch focus timer (Pomodoro/Deep Work/Short Focus presets) |
|
||||
| `notify "title" "body"` | Send an OS notification via the NeuroSkill app |
|
||||
| `raw '{json}'` | Raw JSON passthrough to the server |
|
||||
|
||||
### Global Flags
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--json` | Raw JSON output (no ANSI, pipe-safe) |
|
||||
| `--full` | Human summary + colorized JSON |
|
||||
| `--port <N>` | Override server port (default: auto-discover, usually 8375) |
|
||||
| `--ws` | Force WebSocket transport |
|
||||
| `--http` | Force HTTP transport |
|
||||
| `--k <N>` | Nearest neighbors count (search, search-labels) |
|
||||
| `--seconds <N>` | Duration for listen (default: 5) |
|
||||
| `--trends` | Show per-session metric trends (sessions) |
|
||||
| `--dot` | Graphviz DOT output (interactive) |
|
||||
|
||||
---
|
||||
|
||||
## 1. Checking Current State
|
||||
|
||||
### Get Live Metrics
|
||||
```bash
|
||||
npx neuroskill status --json
|
||||
```
|
||||
|
||||
**Always use `--json`** for reliable parsing. The default output is colorized
|
||||
human-readable text.
|
||||
|
||||
### Key Fields in the Response
|
||||
|
||||
The `scores` object contains all live metrics (0–1 scale unless noted):
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"scores": {
|
||||
"focus": 0.70, // β / (α + θ) — sustained attention
|
||||
"relaxation": 0.40, // α / (β + θ) — calm wakefulness
|
||||
"engagement": 0.60, // active mental investment
|
||||
"meditation": 0.52, // alpha + stillness + HRV coherence
|
||||
"mood": 0.55, // composite from FAA, TAR, BAR
|
||||
"cognitive_load": 0.33, // frontal θ / temporal α · f(FAA, TBR)
|
||||
"drowsiness": 0.10, // TAR + TBR + falling spectral centroid
|
||||
"hr": 68.2, // heart rate in bpm (from PPG)
|
||||
"snr": 14.3, // signal-to-noise ratio in dB
|
||||
"stillness": 0.88, // 0–1; 1 = perfectly still
|
||||
"faa": 0.042, // Frontal Alpha Asymmetry (+ = approach)
|
||||
"tar": 0.56, // Theta/Alpha Ratio
|
||||
"bar": 0.53, // Beta/Alpha Ratio
|
||||
"tbr": 1.06, // Theta/Beta Ratio (ADHD proxy)
|
||||
"apf": 10.1, // Alpha Peak Frequency in Hz
|
||||
"coherence": 0.614, // inter-hemispheric coherence
|
||||
"bands": {
|
||||
"rel_delta": 0.28, "rel_theta": 0.18,
|
||||
"rel_alpha": 0.32, "rel_beta": 0.17, "rel_gamma": 0.05
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Also includes: `device` (state, battery, firmware), `signal_quality` (per-electrode 0–1),
|
||||
`session` (duration, epochs), `embeddings`, `labels`, `sleep` summary, and `history`.
|
||||
|
||||
### Interpreting the Output
|
||||
|
||||
Parse the JSON and translate metrics into natural language. Never report raw
|
||||
numbers alone — always give them meaning:
|
||||
|
||||
**DO:**
|
||||
> "Your focus is solid right now at 0.70 — that's flow state territory. Heart
|
||||
> rate is steady at 68 bpm and your FAA is positive, which suggests good
|
||||
> approach motivation. Great time to tackle something complex."
|
||||
|
||||
**DON'T:**
|
||||
> "Focus: 0.70, Relaxation: 0.40, HR: 68"
|
||||
|
||||
Key interpretation thresholds (see `references/metrics.md` for the full guide):
|
||||
- **Focus > 0.70** → flow state territory, protect it
|
||||
- **Focus < 0.40** → suggest a break or protocol
|
||||
- **Drowsiness > 0.60** → fatigue warning, micro-sleep risk
|
||||
- **Relaxation < 0.30** → stress intervention needed
|
||||
- **Cognitive Load > 0.70 sustained** → mind dump or break
|
||||
- **TBR > 1.5** → theta-dominant, reduced executive control
|
||||
- **FAA < 0** → withdrawal/negative affect — consider FAA rebalancing
|
||||
- **SNR < 3 dB** → unreliable signal, suggest electrode repositioning
|
||||
|
||||
---
|
||||
|
||||
## 2. Session Analysis
|
||||
|
||||
### Single Session Breakdown
|
||||
```bash
|
||||
npx neuroskill session --json # most recent session
|
||||
npx neuroskill session 1 --json # previous session
|
||||
npx neuroskill session 0 --json | jq '{focus: .metrics.focus, trend: .trends.focus}'
|
||||
```
|
||||
|
||||
Returns full metrics with **first-half vs second-half trends** (`"up"`, `"down"`, `"flat"`).
|
||||
Use this to describe how a session evolved:
|
||||
|
||||
> "Your focus started at 0.64 and climbed to 0.76 by the end — a clear upward trend.
|
||||
> Cognitive load dropped from 0.38 to 0.28, suggesting the task became more automatic
|
||||
> as you settled in."
|
||||
|
||||
### List All Sessions
|
||||
```bash
|
||||
npx neuroskill sessions --json
|
||||
npx neuroskill sessions --trends # show per-session metric trends
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Historical Search
|
||||
|
||||
### Neural Similarity Search
|
||||
```bash
|
||||
npx neuroskill search --json # auto: last session, k=5
|
||||
npx neuroskill search --k 10 --json # 10 nearest neighbors
|
||||
npx neuroskill search --start <UTC> --end <UTC> --json
|
||||
```
|
||||
|
||||
Finds moments in history that are neurally similar using HNSW approximate
|
||||
nearest-neighbor search over 128-D ZUNA embeddings. Returns distance statistics,
|
||||
temporal distribution (hour of day), and top matching days.
|
||||
|
||||
Use this when the user asks:
|
||||
- "When was I last in a state like this?"
|
||||
- "Find my best focus sessions"
|
||||
- "When do I usually crash in the afternoon?"
|
||||
|
||||
### Semantic Label Search
|
||||
```bash
|
||||
npx neuroskill search-labels "deep focus" --k 10 --json
|
||||
npx neuroskill search-labels "stress" --json | jq '[.results[].EXG_metrics.tbr]'
|
||||
```
|
||||
|
||||
Searches label text using vector embeddings (Xenova/bge-small-en-v1.5). Returns
|
||||
matching labels with their associated EXG metrics at the time of labeling.
|
||||
|
||||
### Cross-Modal Graph Search
|
||||
```bash
|
||||
npx neuroskill interactive "deep focus" --json
|
||||
npx neuroskill interactive "deep focus" --dot | dot -Tsvg > graph.svg
|
||||
```
|
||||
|
||||
4-layer graph: query → text labels → EXG points → nearby labels. Use `--k-text`,
|
||||
`--k-EXG`, `--reach <minutes>` to tune.
|
||||
|
||||
---
|
||||
|
||||
## 4. Session Comparison
|
||||
```bash
|
||||
npx neuroskill compare --json # auto: last 2 sessions
|
||||
npx neuroskill compare --a-start <UTC> --a-end <UTC> --b-start <UTC> --b-end <UTC> --json
|
||||
```
|
||||
|
||||
Returns metric deltas with absolute change, percentage change, and direction for
|
||||
~50 metrics. Also includes `insights.improved[]` and `insights.declined[]` arrays,
|
||||
sleep staging for both sessions, and a UMAP job ID.
|
||||
|
||||
Interpret comparisons with context — mention trends, not just deltas:
|
||||
> "Yesterday you had two strong focus blocks (10am and 2pm). Today you've had one
|
||||
> starting around 11am that's still going. Your overall engagement is higher today
|
||||
> but there have been more stress spikes — your stress index jumped 15% and
|
||||
> FAA dipped negative more often."
|
||||
|
||||
```bash
|
||||
# Sort metrics by improvement percentage
|
||||
npx neuroskill compare --json | jq '.insights.deltas | to_entries | sort_by(.value.pct) | reverse'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Sleep Data
|
||||
```bash
|
||||
npx neuroskill sleep --json # last 24 hours
|
||||
npx neuroskill sleep 0 --json # most recent sleep session
|
||||
npx neuroskill sleep --start <UTC> --end <UTC> --json
|
||||
```
|
||||
|
||||
Returns epoch-by-epoch sleep staging (5-second windows) with analysis:
|
||||
- **Stage codes**: 0=Wake, 1=N1, 2=N2, 3=N3 (deep), 4=REM
|
||||
- **Analysis**: efficiency_pct, onset_latency_min, rem_latency_min, bout counts
|
||||
- **Healthy targets**: N3 15–25%, REM 20–25%, efficiency >85%, onset <20 min
|
||||
|
||||
```bash
|
||||
npx neuroskill sleep --json | jq '.summary | {n3: .n3_epochs, rem: .rem_epochs}'
|
||||
npx neuroskill sleep --json | jq '.analysis.efficiency_pct'
|
||||
```
|
||||
|
||||
Use this when the user mentions sleep, tiredness, or recovery.
|
||||
|
||||
---
|
||||
|
||||
## 6. Labeling Moments
|
||||
```bash
|
||||
npx neuroskill label "breakthrough"
|
||||
npx neuroskill label "studying algorithms"
|
||||
npx neuroskill label "post-meditation"
|
||||
npx neuroskill label --json "focus block start" # returns label_id
|
||||
```
|
||||
|
||||
Auto-label moments when:
|
||||
- User reports a breakthrough or insight
|
||||
- User starts a new task type (e.g., "switching to code review")
|
||||
- User completes a significant protocol
|
||||
- User asks you to mark the current moment
|
||||
- A notable state transition occurs (entering/leaving flow)
|
||||
|
||||
Labels are stored in a database and indexed for later retrieval via `search-labels`
|
||||
and `interactive` commands.
|
||||
|
||||
---
|
||||
|
||||
## 7. Real-Time Streaming
|
||||
```bash
|
||||
npx neuroskill listen --seconds 30 --json
|
||||
npx neuroskill listen --seconds 5 --json | jq '[.[] | select(.event == "scores")]'
|
||||
```
|
||||
|
||||
Streams live WebSocket events (EXG, PPG, IMU, scores, labels) for the specified
|
||||
duration. Requires WebSocket connection (not available with `--http`).
|
||||
|
||||
Use this for continuous monitoring scenarios or to observe metric changes in real-time
|
||||
during a protocol.
|
||||
|
||||
---
|
||||
|
||||
## 8. UMAP Visualization
|
||||
```bash
|
||||
npx neuroskill umap --json # auto: last 2 sessions
|
||||
npx neuroskill umap --a-start <UTC> --a-end <UTC> --b-start <UTC> --b-end <UTC> --json
|
||||
```
|
||||
|
||||
GPU-accelerated 3D UMAP projection of ZUNA embeddings. The `separation_score`
|
||||
indicates how neurally distinct two sessions are:
|
||||
- **> 1.5** → Sessions are neurally distinct (different brain states)
|
||||
- **< 0.5** → Similar brain states across both sessions
|
||||
|
||||
---
|
||||
|
||||
## 9. Proactive State Awareness
|
||||
|
||||
### Session Start Check
|
||||
At the beginning of a session, optionally run a status check if the user mentions
|
||||
they're wearing their device or asks about their state:
|
||||
```bash
|
||||
npx neuroskill status --json
|
||||
```
|
||||
|
||||
Inject a brief state summary:
|
||||
> "Quick check-in: focus is building at 0.62, relaxation is good at 0.55, and your
|
||||
> FAA is positive — approach motivation is engaged. Looks like a solid start."
|
||||
|
||||
### When to Proactively Mention State
|
||||
|
||||
Mention cognitive state **only** when:
|
||||
- User explicitly asks ("How am I doing?", "Check my focus")
|
||||
- User reports difficulty concentrating, stress, or fatigue
|
||||
- A critical threshold is crossed (drowsiness > 0.70, focus < 0.30 sustained)
|
||||
- User is about to do something cognitively demanding and asks for readiness
|
||||
|
||||
**Do NOT** interrupt flow state to report metrics. If focus > 0.75, protect the
|
||||
session — silence is the correct response.
|
||||
|
||||
---
|
||||
|
||||
## 10. Suggesting Protocols
|
||||
|
||||
When metrics indicate a need, suggest a protocol from `references/protocols.md`.
|
||||
Always ask before starting — never interrupt flow state:
|
||||
|
||||
> "Your focus has been declining for the past 15 minutes and TBR is climbing past
|
||||
> 1.5 — signs of theta dominance and mental fatigue. Want me to walk you through
|
||||
> a Theta-Beta Neurofeedback Anchor? It's a 90-second exercise that uses rhythmic
|
||||
> counting and breath to suppress theta and lift beta."
|
||||
|
||||
Key triggers:
|
||||
- **Focus < 0.40, TBR > 1.5** → Theta-Beta Neurofeedback Anchor or Box Breathing
|
||||
- **Relaxation < 0.30, stress_index high** → Cardiac Coherence or 4-7-8 Breathing
|
||||
- **Cognitive Load > 0.70 sustained** → Cognitive Load Offload (mind dump)
|
||||
- **Drowsiness > 0.60** → Ultradian Reset or Wake Reset
|
||||
- **FAA < 0 (negative)** → FAA Rebalancing
|
||||
- **Flow State (focus > 0.75, engagement > 0.70)** → Do NOT interrupt
|
||||
- **High stillness + headache_index** → Neck Release Sequence
|
||||
- **Low RMSSD (< 25ms)** → Vagal Toning
|
||||
|
||||
---
|
||||
|
||||
## 11. Additional Tools
|
||||
|
||||
### Focus Timer
|
||||
```bash
|
||||
npx neuroskill timer --json
|
||||
```
|
||||
Launches the Focus Timer window with Pomodoro (25/5), Deep Work (50/10), or
|
||||
Short Focus (15/5) presets.
|
||||
|
||||
### Calibration
|
||||
```bash
|
||||
npx neuroskill calibrate
|
||||
npx neuroskill calibrate --profile "Eyes Open"
|
||||
```
|
||||
Opens the calibration window. Useful when signal quality is poor or the user
|
||||
wants to establish a personalized baseline.
|
||||
|
||||
### OS Notifications
|
||||
```bash
|
||||
npx neuroskill notify "Break Time" "Your focus has been declining for 20 minutes"
|
||||
```
|
||||
|
||||
### Raw JSON Passthrough
|
||||
```bash
|
||||
npx neuroskill raw '{"command":"status"}' --json
|
||||
```
|
||||
For any server command not yet mapped to a CLI subcommand.
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Error | Likely Cause | Fix |
|
||||
|-------|-------------|-----|
|
||||
| `npx neuroskill status` hangs | NeuroSkill app not running | Open NeuroSkill desktop app |
|
||||
| `device.state: "disconnected"` | BCI device not connected | Check Bluetooth, device battery |
|
||||
| All scores return 0 | Poor electrode contact | Reposition headband, moisten electrodes |
|
||||
| `signal_quality` values < 0.7 | Loose electrodes | Adjust fit, clean electrode contacts |
|
||||
| SNR < 3 dB | Noisy signal | Minimize head movement, check environment |
|
||||
| `command not found: npx` | Node.js not installed | Install Node.js 20+ |
|
||||
|
||||
---
|
||||
|
||||
## Example Interactions
|
||||
|
||||
**"How am I doing right now?"**
|
||||
```bash
|
||||
npx neuroskill status --json
|
||||
```
|
||||
→ Interpret scores naturally, mentioning focus, relaxation, mood, and any notable
|
||||
ratios (FAA, TBR). Suggest an action only if metrics indicate a need.
|
||||
|
||||
**"I can't concentrate"**
|
||||
```bash
|
||||
npx neuroskill status --json
|
||||
```
|
||||
→ Check if metrics confirm it (high theta, low beta, rising TBR, high drowsiness).
|
||||
→ If confirmed, suggest an appropriate protocol from `references/protocols.md`.
|
||||
→ If metrics look fine, the issue may be motivational rather than neurological.
|
||||
|
||||
**"Compare my focus today vs yesterday"**
|
||||
```bash
|
||||
npx neuroskill compare --json
|
||||
```
|
||||
→ Interpret trends, not just numbers. Mention what improved, what declined, and
|
||||
possible causes.
|
||||
|
||||
**"When was I last in a flow state?"**
|
||||
```bash
|
||||
npx neuroskill search-labels "flow" --json
|
||||
npx neuroskill search --json
|
||||
```
|
||||
→ Report timestamps, associated metrics, and what the user was doing (from labels).
|
||||
|
||||
**"How did I sleep?"**
|
||||
```bash
|
||||
npx neuroskill sleep --json
|
||||
```
|
||||
→ Report sleep architecture (N3%, REM%, efficiency), compare to healthy targets,
|
||||
and note any issues (high wake epochs, low REM).
|
||||
|
||||
**"Mark this moment — I just had a breakthrough"**
|
||||
```bash
|
||||
npx neuroskill label "breakthrough"
|
||||
```
|
||||
→ Confirm label saved. Optionally note the current metrics to remember the state.
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- [NeuroSkill Paper — arXiv:2603.03212](https://arxiv.org/abs/2603.03212) (Kosmyna & Hauptmann, MIT Media Lab)
|
||||
- [NeuroSkill Desktop App](https://github.com/NeuroSkill-com/skill) (GPLv3)
|
||||
- [NeuroLoop CLI Companion](https://github.com/NeuroSkill-com/neuroloop) (GPLv3)
|
||||
- [MIT Media Lab Project](https://www.media.mit.edu/projects/neuroskill/overview/)
|
||||
@@ -0,0 +1,286 @@
|
||||
# NeuroSkill WebSocket & HTTP API Reference
|
||||
|
||||
NeuroSkill runs a local server (default port **8375**) discoverable via mDNS
|
||||
(`_skill._tcp`). It exposes both WebSocket and HTTP endpoints.
|
||||
|
||||
---
|
||||
|
||||
## Server Discovery
|
||||
|
||||
```bash
|
||||
# Auto-discovery (built into the CLI — usually just works)
|
||||
npx neuroskill status --json
|
||||
|
||||
# Manual port discovery
|
||||
NEURO_PORT=$(lsof -i -n -P | grep neuroskill | grep LISTEN | awk '{print $9}' | cut -d: -f2 | head -1)
|
||||
echo "NeuroSkill on port: $NEURO_PORT"
|
||||
```
|
||||
|
||||
The CLI auto-discovers the port. Use `--port <N>` to override.
|
||||
|
||||
---
|
||||
|
||||
## HTTP REST Endpoints
|
||||
|
||||
### Universal Command Tunnel
|
||||
```bash
|
||||
# POST / — accepts any command as JSON
|
||||
curl -s -X POST http://127.0.0.1:8375/ \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"command":"status"}'
|
||||
```
|
||||
|
||||
### Convenience Endpoints
|
||||
| Method | Endpoint | Description |
|
||||
|--------|----------|-------------|
|
||||
| GET | `/v1/status` | System status |
|
||||
| GET | `/v1/sessions` | List sessions |
|
||||
| POST | `/v1/label` | Create label |
|
||||
| POST | `/v1/search` | ANN search |
|
||||
| POST | `/v1/compare` | A/B comparison |
|
||||
| POST | `/v1/sleep` | Sleep staging |
|
||||
| POST | `/v1/notify` | OS notification |
|
||||
| POST | `/v1/say` | Text-to-speech |
|
||||
| POST | `/v1/calibrate` | Open calibration |
|
||||
| POST | `/v1/timer` | Open focus timer |
|
||||
| GET | `/v1/dnd` | Get DND status |
|
||||
| POST | `/v1/dnd` | Force DND on/off |
|
||||
| GET | `/v1/calibrations` | List calibration profiles |
|
||||
| POST | `/v1/calibrations` | Create profile |
|
||||
| GET | `/v1/calibrations/{id}` | Get profile |
|
||||
| PATCH | `/v1/calibrations/{id}` | Update profile |
|
||||
| DELETE | `/v1/calibrations/{id}` | Delete profile |
|
||||
|
||||
---
|
||||
|
||||
## WebSocket Events (Broadcast)
|
||||
|
||||
Connect to `ws://127.0.0.1:8375/` to receive real-time events:
|
||||
|
||||
### EXG (Raw EEG Samples)
|
||||
```json
|
||||
{"event": "EXG", "electrode": 0, "samples": [12.3, -4.1, ...], "timestamp": 1740412800.512}
|
||||
```
|
||||
|
||||
### PPG (Photoplethysmography)
|
||||
```json
|
||||
{"event": "PPG", "channel": 0, "samples": [...], "timestamp": 1740412800.512}
|
||||
```
|
||||
|
||||
### IMU (Inertial Measurement Unit)
|
||||
```json
|
||||
{"event": "IMU", "ax": 0.01, "ay": -0.02, "az": 9.81, "gx": 0.1, "gy": -0.05, "gz": 0.02}
|
||||
```
|
||||
|
||||
### Scores (Computed Metrics)
|
||||
```json
|
||||
{
|
||||
"event": "scores",
|
||||
"focus": 0.70, "relaxation": 0.40, "engagement": 0.60,
|
||||
"rel_delta": 0.28, "rel_theta": 0.18, "rel_alpha": 0.32,
|
||||
"rel_beta": 0.17, "hr": 68.2, "snr": 14.3
|
||||
}
|
||||
```
|
||||
|
||||
### EXG Bands (Spectral Analysis)
|
||||
```json
|
||||
{"event": "EXG-bands", "channels": [...], "faa": 0.12}
|
||||
```
|
||||
|
||||
### Labels
|
||||
```json
|
||||
{"event": "label", "label_id": 42, "text": "meditation start", "created_at": 1740413100}
|
||||
```
|
||||
|
||||
### Device Status
|
||||
```json
|
||||
{"event": "muse-status", "state": "connected"}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## JSON Response Formats
|
||||
|
||||
### `status`
|
||||
```jsonc
|
||||
{
|
||||
"command": "status", "ok": true,
|
||||
"device": {
|
||||
"state": "connected", // "connected" | "connecting" | "disconnected"
|
||||
"name": "Muse-A1B2",
|
||||
"battery": 73,
|
||||
"firmware": "1.3.4",
|
||||
"EXG_samples": 195840,
|
||||
"ppg_samples": 30600,
|
||||
"imu_samples": 122400
|
||||
},
|
||||
"session": {
|
||||
"start_utc": 1740412800,
|
||||
"duration_secs": 1847,
|
||||
"n_epochs": 369
|
||||
},
|
||||
"signal_quality": {
|
||||
"tp9": 0.95, "af7": 0.88, "af8": 0.91, "tp10": 0.97
|
||||
},
|
||||
"scores": {
|
||||
"focus": 0.70, "relaxation": 0.40, "engagement": 0.60,
|
||||
"meditation": 0.52, "mood": 0.55, "cognitive_load": 0.33,
|
||||
"drowsiness": 0.10, "hr": 68.2, "snr": 14.3, "stillness": 0.88,
|
||||
"bands": { "rel_delta": 0.28, "rel_theta": 0.18, "rel_alpha": 0.32, "rel_beta": 0.17, "rel_gamma": 0.05 },
|
||||
"faa": 0.042, "tar": 0.56, "bar": 0.53, "tbr": 1.06,
|
||||
"apf": 10.1, "coherence": 0.614, "mu_suppression": 0.031
|
||||
},
|
||||
"embeddings": { "today": 342, "total": 14820, "recording_days": 31 },
|
||||
"labels": { "total": 58, "recent": [{"id": 42, "text": "meditation start", "created_at": 1740413100}] },
|
||||
"sleep": { "total_epochs": 1054, "wake_epochs": 134, "n1_epochs": 89, "n2_epochs": 421, "n3_epochs": 298, "rem_epochs": 112, "epoch_secs": 5 },
|
||||
"history": { "total_sessions": 63, "recording_days": 31, "current_streak_days": 7, "total_recording_hours": 94.2, "longest_session_min": 187, "avg_session_min": 89 }
|
||||
}
|
||||
```
|
||||
|
||||
### `sessions`
|
||||
```jsonc
|
||||
{
|
||||
"command": "sessions", "ok": true,
|
||||
"sessions": [
|
||||
{ "day": "20260224", "start_utc": 1740412800, "end_utc": 1740415510, "n_epochs": 541 },
|
||||
{ "day": "20260223", "start_utc": 1740380100, "end_utc": 1740382665, "n_epochs": 513 }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### `session` (single session breakdown)
|
||||
```jsonc
|
||||
{
|
||||
"ok": true,
|
||||
"metrics": { "focus": 0.70, "relaxation": 0.40, "n_epochs": 541 /* ... ~50 metrics */ },
|
||||
"first": { "focus": 0.64 /* first-half averages */ },
|
||||
"second": { "focus": 0.76 /* second-half averages */ },
|
||||
"trends": { "focus": "up", "relaxation": "down" /* "up" | "down" | "flat" */ }
|
||||
}
|
||||
```
|
||||
|
||||
### `compare` (A/B comparison)
|
||||
```jsonc
|
||||
{
|
||||
"command": "compare", "ok": true,
|
||||
"insights": {
|
||||
"deltas": {
|
||||
"focus": { "a": 0.62, "b": 0.71, "abs": 0.09, "pct": 14.5, "direction": "up" },
|
||||
"relaxation": { "a": 0.45, "b": 0.38, "abs": -0.07, "pct": -15.6, "direction": "down" }
|
||||
},
|
||||
"improved": ["focus", "engagement"],
|
||||
"declined": ["relaxation"]
|
||||
},
|
||||
"sleep_a": { /* sleep summary for session A */ },
|
||||
"sleep_b": { /* sleep summary for session B */ },
|
||||
"umap": { "job_id": "abc123" }
|
||||
}
|
||||
```
|
||||
|
||||
### `search` (ANN similarity)
|
||||
```jsonc
|
||||
{
|
||||
"command": "search", "ok": true,
|
||||
"result": {
|
||||
"results": [{
|
||||
"neighbors": [{ "distance": 0.12, "metadata": {"device": "Muse-A1B2", "date": "20260223"} }]
|
||||
}],
|
||||
"analysis": {
|
||||
"distance_stats": { "mean": 0.15, "min": 0.08, "max": 0.42 },
|
||||
"temporal_distribution": { /* hour-of-day distribution */ },
|
||||
"top_days": [["20260223", 5], ["20260222", 3]]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `sleep` (sleep staging)
|
||||
```jsonc
|
||||
{
|
||||
"command": "sleep", "ok": true,
|
||||
"summary": { "total_epochs": 1054, "wake_epochs": 134, "n1_epochs": 89, "n2_epochs": 421, "n3_epochs": 298, "rem_epochs": 112, "epoch_secs": 5 },
|
||||
"analysis": { "efficiency_pct": 87.3, "onset_latency_min": 12.5, "rem_latency_min": 65.0, "bouts": { /* wake/n3/rem bout counts and durations */ } },
|
||||
"epochs": [{ "utc": 1740380100, "stage": 0, "rel_delta": 0.15, "rel_theta": 0.22, "rel_alpha": 0.38, "rel_beta": 0.20 }]
|
||||
}
|
||||
```
|
||||
|
||||
### `label`
|
||||
```json
|
||||
{"command": "label", "ok": true, "label_id": 42}
|
||||
```
|
||||
|
||||
### `search-labels` (semantic search)
|
||||
```jsonc
|
||||
{
|
||||
"command": "search-labels", "ok": true,
|
||||
"results": [{
|
||||
"text": "deep focus block",
|
||||
"EXG_metrics": { "focus": 0.82, "relaxation": 0.35, "engagement": 0.75, "hr": 65.0, "mood": 0.60 },
|
||||
"EXG_start": 1740412800, "EXG_end": 1740412805,
|
||||
"created_at": 1740412802,
|
||||
"similarity": 0.92
|
||||
}]
|
||||
}
|
||||
```
|
||||
|
||||
### `umap` (3D projection)
|
||||
```jsonc
|
||||
{
|
||||
"command": "umap", "ok": true,
|
||||
"result": {
|
||||
"points": [{ "x": 1.23, "y": -0.45, "z": 2.01, "session": "a", "utc": 1740412800 }],
|
||||
"analysis": {
|
||||
"separation_score": 1.84,
|
||||
"inter_cluster_distance": 2.31,
|
||||
"intra_spread_a": 0.82, "intra_spread_b": 0.94,
|
||||
"centroid_a": [1.23, -0.45, 2.01],
|
||||
"centroid_b": [-0.87, 1.34, -1.22]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Useful `jq` Snippets
|
||||
|
||||
```bash
|
||||
# Get just focus score
|
||||
npx neuroskill status --json | jq '.scores.focus'
|
||||
|
||||
# Get all band powers
|
||||
npx neuroskill status --json | jq '.scores.bands'
|
||||
|
||||
# Check device battery
|
||||
npx neuroskill status --json | jq '.device.battery'
|
||||
|
||||
# Get signal quality
|
||||
npx neuroskill status --json | jq '.signal_quality'
|
||||
|
||||
# Find improving metrics after a session
|
||||
npx neuroskill session 0 --json | jq '[.trends | to_entries[] | select(.value == "up") | .key]'
|
||||
|
||||
# Sort comparison deltas by improvement
|
||||
npx neuroskill compare --json | jq '.insights.deltas | to_entries | sort_by(.value.pct) | reverse'
|
||||
|
||||
# Get sleep efficiency
|
||||
npx neuroskill sleep --json | jq '.analysis.efficiency_pct'
|
||||
|
||||
# Find closest neural match
|
||||
npx neuroskill search --json | jq '[.result.results[].neighbors[]] | sort_by(.distance) | .[0]'
|
||||
|
||||
# Extract TBR from labeled stress moments
|
||||
npx neuroskill search-labels "stress" --json | jq '[.results[].EXG_metrics.tbr]'
|
||||
|
||||
# Get session timestamps for manual compare
|
||||
npx neuroskill sessions --json | jq '{start: .sessions[0].start_utc, end: .sessions[0].end_utc}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Storage
|
||||
|
||||
- **Local database**: `~/.skill/YYYYMMDD/` (SQLite + HNSW index)
|
||||
- **ZUNA embeddings**: 128-D vectors, 5-second epochs
|
||||
- **Labels**: Stored in SQLite, indexed with bge-small-en-v1.5 embeddings
|
||||
- **All data is local** — nothing is sent to external servers
|
||||
@@ -0,0 +1,220 @@
|
||||
# NeuroSkill Metric Definitions & Interpretation Guide
|
||||
|
||||
> **⚠️ Research Use Only:** All metrics are experimental and derived from
|
||||
> consumer-grade hardware (Muse 2/S). They are not FDA/CE-cleared and must not
|
||||
> be used for medical diagnosis or treatment.
|
||||
|
||||
---
|
||||
|
||||
## Hardware & Signal Acquisition
|
||||
|
||||
NeuroSkill is validated for **Muse 2** and **Muse S** headbands (with OpenBCI
|
||||
support in the desktop app), streaming at **256 Hz** (EEG) and **64 Hz** (PPG).
|
||||
|
||||
### Electrode Positions (International 10-20 System)
|
||||
| Channel | Electrode | Position | Primary Signals |
|
||||
|---------|-----------|----------|-----------------|
|
||||
| CH1 | TP9 | Left Mastoid | Auditory cortex, verbal memory, jaw-clench artifact |
|
||||
| CH2 | AF7 | Left Prefrontal | Executive function, approach motivation, eye blinks |
|
||||
| CH3 | AF8 | Right Prefrontal | Emotional regulation, vigilance, eye blinks |
|
||||
| CH4 | TP10 | Right Mastoid | Prosody, spatial hearing, non-verbal cognition |
|
||||
|
||||
### Preprocessing Pipeline
|
||||
1. **Filtering**: High-pass (0.5 Hz), Low-pass (50/60 Hz), Notch filter
|
||||
2. **Spectral Analysis**: Hann-windowed FFT (512-sample window), Welch periodogram
|
||||
3. **GPU acceleration**: ~125ms latency via `gpu_fft`
|
||||
|
||||
---
|
||||
|
||||
## EEG Frequency Bands
|
||||
|
||||
Relative power values (sum ≈ 1.0 across all bands):
|
||||
|
||||
| Band | Range (Hz) | High Means | Low Means |
|
||||
|------|-----------|------------|-----------|
|
||||
| **Delta (δ)** | 1–4 | Deep sleep (N3), high-amplitude artifacts | Awake, alert |
|
||||
| **Theta (θ)** | 4–8 | Drowsiness, REM onset, creative ideation, cognitive load | Alert, focused |
|
||||
| **Alpha (α)** | 8–13 | Relaxed wakefulness, "alpha blocking" during effort | Active thinking, anxiety |
|
||||
| **Beta (β)** | 13–30 | Active concentration, problem-solving, alertness | Relaxed, unfocused |
|
||||
| **Gamma (γ)** | 30–50 | Higher-order processing, perceptual binding, memory | Baseline |
|
||||
|
||||
### JSON Field Names
|
||||
```json
|
||||
"bands": {
|
||||
"rel_delta": 0.28, "rel_theta": 0.18, "rel_alpha": 0.32,
|
||||
"rel_beta": 0.17, "rel_gamma": 0.05
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Core Composite Scores (0–1 Scale)
|
||||
|
||||
### Focus
|
||||
- **Formula**: σ(β / (α + θ)) — beta dominance over slow waves, sigmoid-mapped
|
||||
- **> 0.70**: Deep concentration, flow state, task absorption
|
||||
- **0.40–0.69**: Moderate attention, some mind-wandering
|
||||
- **< 0.40**: Distracted, fatigued, difficulty concentrating
|
||||
|
||||
### Relaxation
|
||||
- **Formula**: σ(α / (β + θ)) — alpha dominance, sigmoid-mapped
|
||||
- **> 0.70**: Calm, stress-free, parasympathetic dominant
|
||||
- **0.40–0.69**: Mild tension present
|
||||
- **< 0.30**: Stressed, anxious, sympathetic dominant
|
||||
|
||||
### Engagement
|
||||
- **0–1 scale**: Active mental investment and motivation
|
||||
- **> 0.70**: Mentally invested, motivated, active processing
|
||||
- **0.40–0.69**: Passive participation
|
||||
- **< 0.30**: Bored, disengaged, autopilot mode
|
||||
|
||||
### Meditation
|
||||
- **Composite**: Combines alpha elevation, physical stillness (IMU), and HRV coherence
|
||||
- **> 0.70**: Deep meditative state
|
||||
- **< 0.30**: Active, non-meditative
|
||||
|
||||
### Mood
|
||||
- **Composite**: Derived from FAA, TAR, and BAR
|
||||
- **> 0.60**: Positive affect, approach motivation
|
||||
- **< 0.40**: Low mood, withdrawal tendency
|
||||
|
||||
### Cognitive Load
|
||||
- **Formula**: (P_θ_frontal / P_α_temporal) · f(FAA, TBR) — working memory usage
|
||||
- **> 0.70**: Working memory near capacity, complex processing
|
||||
- **0.40–0.69**: Moderate mental effort
|
||||
- **< 0.40**: Task is easy or automatic
|
||||
- **Interpretation**: High load + high focus = productive struggle. High load + low focus = overwhelmed.
|
||||
|
||||
### Drowsiness
|
||||
- **Composite**: Weighted TAR + TBR + falling Spectral Centroid
|
||||
- **> 0.60**: Sleep pressure building, micro-sleep risk
|
||||
- **0.30–0.59**: Mild fatigue
|
||||
- **< 0.30**: Alert
|
||||
|
||||
---
|
||||
|
||||
## EEG Ratios & Spectral Indices
|
||||
|
||||
| Metric | Formula | Interpretation |
|
||||
|--------|---------|----------------|
|
||||
| **FAA** | ln(P_α_AF8) − ln(P_α_AF7) | Frontal Alpha Asymmetry. Positive = approach/positive affect. Negative = withdrawal/depression. |
|
||||
| **TAR** | P_θ / P_α | Theta/Alpha Ratio. > 1.5 = drowsiness or mind-wandering. |
|
||||
| **BAR** | P_β / P_α | Beta/Alpha Ratio. > 1.5 = alert, engaged cognition. Can also indicate anxiety. |
|
||||
| **TBR** | P_θ / P_β | Theta/Beta Ratio. ADHD biomarker. Healthy ≈ 1.0, elevated > 1.5, clinical > 3.0. |
|
||||
| **APF** | argmax_f PSD(f) in [7.5, 12.5] Hz | Alpha Peak Frequency. Typical 8–12 Hz. Higher = faster cognitive processing. Slows with age/fatigue. |
|
||||
| **SNR** | 10 · log₁₀(P_signal / P_noise) | Signal-to-Noise Ratio. > 10 dB = clean, 3–10 dB = usable, < 3 dB = unreliable. |
|
||||
| **Coherence** | Inter-hemispheric coherence (0–1) | Cortical connectivity between hemispheres. |
|
||||
| **Mu Suppression** | Motor cortex suppression index | Low values during movement or motor imagery. |
|
||||
|
||||
---
|
||||
|
||||
## Complexity & Nonlinear Metrics
|
||||
|
||||
| Metric | Description | Healthy Range |
|
||||
|--------|-------------|---------------|
|
||||
| **Permutation Entropy (PE)** | Temporal complexity. Near 1 = maximally irregular. | Consciousness marker |
|
||||
| **Higuchi Fractal Dimension (HFD)** | Waveform self-similarity. | Waking: 1.3–1.8; higher = complex |
|
||||
| **DFA Exponent** | Long-range correlations. | Healthy: 0.6–0.9 |
|
||||
| **PSE** | Power Spectral Entropy. Near 1.0 = white noise. | Lower = organized brain state |
|
||||
| **PAC θ-γ** | Phase-Amplitude Coupling, theta-gamma. | Working memory mechanism |
|
||||
| **BPS** | Band-Power Slope (1/f spectral exponent). | Steeper = inhibition-dominated |
|
||||
|
||||
---
|
||||
|
||||
## Consciousness Metrics
|
||||
|
||||
Derived from the nonlinear metrics above:
|
||||
|
||||
| Metric | Scale | Interpretation |
|
||||
|--------|-------|----------------|
|
||||
| **LZC** | 0–100 | Lempel-Ziv Complexity proxy (PE + HFD). > 60 = wakefulness. |
|
||||
| **Wakefulness** | 0–100 | Inverse drowsiness composite. |
|
||||
| **Integration** | 0–100 | Cortical integration (Coherence × PAC × Spectral Entropy). |
|
||||
|
||||
Status thresholds: ≥ 50 Green, 25–50 Yellow, < 25 Red.
|
||||
|
||||
---
|
||||
|
||||
## Cardiac & Autonomic Metrics (from PPG)
|
||||
|
||||
| Metric | Description | Normal / Green Range |
|
||||
|--------|-------------|---------------------|
|
||||
| **HR** | Heart rate (bpm) | 55–90 (green), 45–110 (yellow), else red |
|
||||
| **RMSSD** | Primary vagal tone marker (ms) | > 50 ms healthy, < 20 ms stress |
|
||||
| **SDNN** | HRV time-domain variability (ms) | Higher = better |
|
||||
| **pNN50** | Parasympathetic indicator (%) | Higher = more parasympathetic activity |
|
||||
| **LF/HF Ratio** | Sympatho-vagal balance | > 2.0 = stress, < 0.5 = relaxation |
|
||||
| **Stress Index** | Baevsky SI: AMo / (2 × MxDMn × Mo) | 0–100 composite. > 200 raw = strong stress |
|
||||
| **SpO₂ Estimate** | Blood oxygen saturation (uncalibrated) | 95–100% normal (research only) |
|
||||
| **Respiratory Rate** | Breaths per minute | 12–20 normal |
|
||||
|
||||
---
|
||||
|
||||
## Motion & Artifact Detection
|
||||
|
||||
| Metric | Description |
|
||||
|--------|-------------|
|
||||
| **Stillness** | 0–1 (1 = perfectly still). From IMU accelerometer/gyroscope. |
|
||||
| **Blink Count** | Eye blinks detected (large spikes in AF7/AF8). Normal: 15–20/min. |
|
||||
| **Jaw Clench Count** | High-frequency EMG bursts (> 30 Hz) at TP9/TP10. |
|
||||
| **Nod Count** | Head nods detected via IMU. |
|
||||
| **Shake Count** | Head shakes detected via IMU. |
|
||||
| **Head Pitch/Roll** | Head orientation from IMU. |
|
||||
|
||||
---
|
||||
|
||||
## Signal Quality (Per Electrode)
|
||||
|
||||
| Electrode | Range | Interpretation |
|
||||
|-----------|-------|----------------|
|
||||
| **TP9** | 0–1 | ≥ 0.9 = good, ≥ 0.7 = acceptable, < 0.7 = poor |
|
||||
| **AF7** | 0–1 | Same thresholds |
|
||||
| **AF8** | 0–1 | Same thresholds |
|
||||
| **TP10** | 0–1 | Same thresholds |
|
||||
|
||||
If any electrode is below 0.7, recommend the user adjust the headband fit or
|
||||
moisten the electrode contacts.
|
||||
|
||||
---
|
||||
|
||||
## Sleep Staging
|
||||
|
||||
Based on 5-second epochs using relative band-power ratios and AASM heuristics:
|
||||
|
||||
| Stage | Code | EEG Signature | Function |
|
||||
|-------|------|---------------|----------|
|
||||
| Wake | 0 | Alpha-dominant, BAR > 0.8 | Conscious awareness |
|
||||
| N1 | 1 | Alpha → Theta transition | Light sleep onset |
|
||||
| N2 | 2 | Sleep spindles, K-complexes | Memory consolidation |
|
||||
| N3 (Deep) | 3 | Delta > 20% of epoch, DTR > 2 | Deep restorative sleep |
|
||||
| REM | 4 | Active EEG, high Theta, low Delta | Emotional processing, dreaming |
|
||||
|
||||
### Healthy Adult Targets (~8h Sleep)
|
||||
- **N3 (Deep)**: 15–25% of total sleep
|
||||
- **REM**: 20–25%
|
||||
- **Sleep Efficiency**: > 85%
|
||||
- **Sleep Onset Latency**: < 20 min
|
||||
|
||||
---
|
||||
|
||||
## Composite State Patterns
|
||||
|
||||
| Pattern | Key Metrics | Interpretation |
|
||||
|---------|-------------|----------------|
|
||||
| **Flow State** | Focus > 0.75, Engagement > 0.70, Cognitive Load 0.50–0.70, HR steady | Optimal performance zone — protect it |
|
||||
| **Mental Fatigue** | Focus < 0.40, Drowsiness > 0.60, TBR > 1.5, Theta elevated | Rest or break needed |
|
||||
| **Anxiety** | Relaxation < 0.30, HR elevated, high Beta, high BAR, stress_index high | Calming intervention helpful |
|
||||
| **Peak Alert** | Focus > 0.80, Engagement > 0.70, Drowsiness < 0.20 | Best time for hard tasks |
|
||||
| **Recovery** | Relaxation > 0.70, HRV (RMSSD) rising, Alpha dominant | Integration, light tasks only |
|
||||
| **Creative Mode** | High Theta, high Alpha, low Beta, moderate focus | Ideation — don't force structure |
|
||||
| **Withdrawal** | FAA < 0, low Mood, low Engagement | Approach motivation needed |
|
||||
|
||||
---
|
||||
|
||||
## ZUNA Embeddings
|
||||
|
||||
NeuroSkill uses the **ZUNA Neural Encoder** to convert 5-second EEG epochs into
|
||||
**128-dimensional vectors** stored in an HNSW index:
|
||||
- **Search**: Sub-millisecond approximate nearest-neighbor queries
|
||||
- **UMAP**: GPU-accelerated 3D projection for visual comparison
|
||||
- **Storage**: Local SQLite + HNSW index in `~/.skill/YYYYMMDD/`
|
||||
@@ -0,0 +1,452 @@
|
||||
# NeuroSkill Guided Protocols
|
||||
|
||||
Over 70 mind-body practices triggered by specific biometric (EXG) signals. These
|
||||
are sourced from NeuroLoop's protocol repertoire and are designed to be suggested
|
||||
when the system detects specific cognitive or physiological states.
|
||||
|
||||
> **⚠️ Contraindication**: Wim Hof and hyperventilation-style breathwork are
|
||||
> unsuitable for epilepsy_risk > 30, known cardiac conditions, or pregnancy.
|
||||
|
||||
---
|
||||
|
||||
## When to Suggest Protocols
|
||||
|
||||
**Always ask before starting.** Match ONE protocol to the single most salient
|
||||
metric signal. Explain the metric connection to the user.
|
||||
|
||||
| User State | Recommended Protocol |
|
||||
|------------|---------------------|
|
||||
| Focus < 0.40, TBR > 1.5 | Theta-Beta Neurofeedback Anchor or Box Breathing |
|
||||
| Low engagement, session start | WOOP or Pre-Task Priming |
|
||||
| Relaxation < 0.30, stress_index high | Cardiac Coherence or 4-7-8 Breathing |
|
||||
| Cognitive Load > 0.70 sustained | Cognitive Load Offload (Mind Dump) |
|
||||
| Engagement < 0.30 for > 20 min | Novel Stimulation Burst or Environment Change |
|
||||
| Flow State (focus > 0.75, engagement > 0.70) | **Do NOT interrupt — protect the session** |
|
||||
| Drowsiness > 0.60, post-lunch | Ultradian Reset or Power Nap |
|
||||
| FAA < 0, depression_index elevated | FAA Rebalancing |
|
||||
| Low RMSSD (< 25ms) | Vagal Toning |
|
||||
| High stillness + headache signals | Neck Release Sequence |
|
||||
| Pre-sleep, HRV low | Sleep Wind-Down |
|
||||
| Post-social-media, low mood | Envy & Comparison Alchemy |
|
||||
|
||||
---
|
||||
|
||||
## Attention & Focus Protocols
|
||||
|
||||
### Theta-Beta Neurofeedback Anchor
|
||||
**Duration**: ~90 seconds
|
||||
**Trigger**: High TBR (> 1.5) and low focus
|
||||
**Instructions**:
|
||||
1. Close your eyes
|
||||
2. Breathe slowly — 4s inhale, 6s exhale
|
||||
3. Count rhythmically from 1 to 10, matching your breath
|
||||
4. Focus on the counting — if you lose count, restart from 1
|
||||
5. Open your eyes after 4–5 full cycles
|
||||
**Effect**: Suppresses theta dominance and lifts beta activity
|
||||
|
||||
### Focus Reset
|
||||
**Duration**: 90 seconds
|
||||
**Trigger**: Scattered engagement, difficulty settling into task
|
||||
**Instructions**:
|
||||
1. Close your eyes completely
|
||||
2. Take 5 slow, deep breaths
|
||||
3. Mentally state your intention for the next work block
|
||||
4. Open your eyes and begin immediately
|
||||
**Effect**: Resets attentional baseline
|
||||
|
||||
### Working Memory Primer
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: Low PAC θ-γ (theta-gamma coupling), low sample entropy
|
||||
**Instructions**:
|
||||
1. Breathe at theta pace: 4s inhale, 6s exhale, 2s hold
|
||||
2. While breathing, do a verbal 3-back task: listen to or read a sequence
|
||||
of numbers, say which number appeared 3 positions back
|
||||
3. Continue for 3 minutes
|
||||
**Effect**: Lifts theta-gamma coupling and working memory engagement
|
||||
|
||||
### Creativity Unlock
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: High beta, low rel_alpha — system is too analytically locked
|
||||
**Instructions**:
|
||||
1. Stop all structured work
|
||||
2. Let your mind wander without a goal
|
||||
3. Doodle, look out the window, or listen to ambient sound
|
||||
4. Don't force any outcome — just observe what arises
|
||||
5. After 5 minutes, jot down any ideas that surfaced
|
||||
**Effect**: Promotes alpha and theta activity for creative ideation
|
||||
|
||||
### Dual-N-Back Warm-Up
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: Low PAC θ-γ, low sample entropy
|
||||
**Instructions**:
|
||||
1. Read or listen to a sequence of spoken numbers
|
||||
2. Track which number appeared 2 positions back (2-back)
|
||||
3. If comfortable, increase to 3-back
|
||||
**Effect**: Activates prefrontal cortex, lifts executive function
|
||||
|
||||
### Novel Stimulation Burst
|
||||
**Duration**: 2–3 minutes
|
||||
**Trigger**: Low APF (< 9 Hz), dementia_index > 30
|
||||
**Instructions**:
|
||||
1. Pick up an unusual object nearby and describe it in detail
|
||||
2. Name 5 things you can see, 4 you can touch, 3 you can hear
|
||||
3. Try a quick riddle or lateral thinking puzzle
|
||||
**Effect**: Counters cortical slowing, raises alpha peak frequency
|
||||
|
||||
---
|
||||
|
||||
## Autonomic & Stress Regulation Protocols
|
||||
|
||||
### Box Breathing (4-4-4-4)
|
||||
**Duration**: 2–4 minutes
|
||||
**Trigger**: High BAR, high anxiety_index, acute stress
|
||||
**Instructions**:
|
||||
1. Inhale for 4 counts
|
||||
2. Hold for 4 counts
|
||||
3. Exhale for 4 counts
|
||||
4. Hold for 4 counts
|
||||
5. Repeat 4–8 cycles
|
||||
**Effect**: Engages parasympathetic nervous system, reduces beta activity
|
||||
|
||||
### Extended Exhale (4-7-8)
|
||||
**Duration**: 3–5 minutes
|
||||
**Trigger**: Acute stress spikes, racing thoughts, high sympathetic activation
|
||||
**Instructions**:
|
||||
1. Exhale completely through mouth
|
||||
2. Inhale through nose for 4 counts
|
||||
3. Hold for 7 counts
|
||||
4. Exhale through mouth for 8 counts
|
||||
5. Repeat 4 cycles
|
||||
**Effect**: Fastest parasympathetic trigger for acute stress
|
||||
|
||||
### Cardiac Coherence
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: Low RMSSD (< 30 ms), high stress_index
|
||||
**Instructions**:
|
||||
1. Breathe evenly: 5-second inhale, 5-second exhale
|
||||
2. Focus on the area around your heart
|
||||
3. Recall a positive memory or feeling of appreciation
|
||||
4. Maintain for 5 minutes
|
||||
**Effect**: Maximizes HRV, creates coherent heart rhythm pattern
|
||||
|
||||
### Physiological Sigh
|
||||
**Duration**: 30 seconds (1–3 cycles)
|
||||
**Trigger**: Rapid overwhelm, acute panic
|
||||
**Instructions**:
|
||||
1. Take a quick double inhale through the nose (sniff-sniff)
|
||||
2. Follow with a long, slow exhale through the mouth
|
||||
3. Repeat 1–3 times
|
||||
**Effect**: Rapid parasympathetic activation, immediate calming
|
||||
|
||||
### Alpha Induction (Open Focus)
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: High beta, low relaxation — cannot relax
|
||||
**Instructions**:
|
||||
1. Soften your gaze — don't focus on any single object
|
||||
2. Notice the space between and around objects
|
||||
3. Expand your awareness to peripheral vision
|
||||
4. Maintain this "open focus" for 5 minutes
|
||||
**Effect**: Promotes alpha wave production, reduces beta dominance
|
||||
|
||||
### Open Monitoring
|
||||
**Duration**: 5–10 minutes
|
||||
**Trigger**: Low LZC (< 40 on 0-100 scale) — neural complexity too low
|
||||
**Instructions**:
|
||||
1. Sit comfortably with eyes closed or softly focused
|
||||
2. Don't direct attention to anything specific
|
||||
3. Simply notice whatever arises — thoughts, sounds, sensations
|
||||
4. Let each observation pass without engagement
|
||||
**Effect**: Raises neural complexity and consciousness metrics
|
||||
|
||||
### Vagal Toning
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: Low RMSSD (< 25 ms) — weak vagal tone
|
||||
**Instructions**:
|
||||
1. Hum a long, steady note on each exhale for 30 seconds
|
||||
2. Alternatively: gargle cold water for 30 seconds
|
||||
3. Repeat 3–5 times
|
||||
**Effect**: Directly stimulates the vagus nerve, increases parasympathetic tone
|
||||
|
||||
---
|
||||
|
||||
## Emotional Regulation Protocols
|
||||
|
||||
### FAA Rebalancing
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: Negative FAA (right-hemisphere dominant), high depression_index
|
||||
**Instructions**:
|
||||
1. Think of something you're genuinely looking forward to (approach motivation)
|
||||
2. Visualize yourself successfully completing a meaningful goal
|
||||
3. Squeeze your left hand into a fist for 10 seconds, release
|
||||
4. Repeat the visualization + left-hand squeeze 3–4 times
|
||||
**Effect**: Activates left prefrontal cortex, shifts FAA positive
|
||||
|
||||
### Loving-Kindness (Metta)
|
||||
**Duration**: 5–10 minutes
|
||||
**Trigger**: Loneliness signals, shame, low mood
|
||||
**Instructions**:
|
||||
1. Close your eyes and think of someone you care about
|
||||
2. Silently repeat: "May you be happy. May you be healthy. May you be safe."
|
||||
3. Extend the same wishes to yourself
|
||||
4. Extend to a neutral person, then gradually to someone difficult
|
||||
**Effect**: Reduces withdrawal motivation, increases positive affect
|
||||
|
||||
### Emotional Discharge
|
||||
**Duration**: 2 minutes
|
||||
**Trigger**: High bipolar_index or extreme FAA swings
|
||||
**Instructions**:
|
||||
1. Take 30 seconds of vigorous, fast breathing (safely)
|
||||
2. Stop and take 3 slow, deep breaths
|
||||
3. Do a 60-second body scan — notice where tension is held
|
||||
4. Shake out your hands and arms for 15 seconds
|
||||
**Effect**: Releases trapped sympathetic energy, recalibrates
|
||||
|
||||
### Havening Touch
|
||||
**Duration**: 3–5 minutes
|
||||
**Trigger**: Acute distress, trauma activation, overwhelming anxiety
|
||||
**Instructions**:
|
||||
1. Gently stroke your arms from shoulder to elbow, palms down
|
||||
2. Rub your palms together slowly
|
||||
3. Gently touch your forehead, temples
|
||||
4. Continue for 3–5 minutes while breathing slowly
|
||||
**Effect**: Disrupts amygdala-cortex encoding loop, reduces distress
|
||||
|
||||
### Anxiety Surfing
|
||||
**Duration**: ~8 minutes
|
||||
**Trigger**: Rising anxiety without clear cause
|
||||
**Instructions**:
|
||||
1. Notice where anxiety lives in your body — chest? stomach? throat?
|
||||
2. Describe the sensation without judging it (tight? hot? buzzing?)
|
||||
3. Breathe into that area for 3 breaths
|
||||
4. Notice: is it getting bigger, smaller, or changing shape?
|
||||
5. Continue observing for 5–8 minutes — anxiety typically peaks then subsides
|
||||
|
||||
### Anger: Palm-Press Discharge
|
||||
**Duration**: 2 minutes
|
||||
**Trigger**: Anger signals, high BAR + elevated HR
|
||||
**Instructions**:
|
||||
1. Press your palms together firmly for 10 seconds
|
||||
2. Release and take 3 extended exhales (4s in, 8s out)
|
||||
3. Repeat 3–4 times
|
||||
|
||||
### Envy & Comparison Alchemy
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: Post-social-media, envy signals
|
||||
**Instructions**:
|
||||
1. Name the envy: "I feel envious of ___"
|
||||
2. Ask: "What does this envy tell me I actually want?"
|
||||
3. Convert: "My next step toward that is ___"
|
||||
**Effect**: Converts envy into a desire-signal that identifies personal values
|
||||
|
||||
### Awe Induction
|
||||
**Duration**: 3–5 minutes
|
||||
**Trigger**: Existential flatness, low engagement, loss of meaning
|
||||
**Instructions**:
|
||||
1. Imagine standing at the edge of the Grand Canyon, or beneath a starry sky
|
||||
2. Let yourself feel the scale — you are small, and that's beautiful
|
||||
3. Recall a moment of genuine wonder from your past
|
||||
4. Notice what changes in your body
|
||||
**Effect**: Counters hedonic adaptation, restores sense of meaning
|
||||
|
||||
---
|
||||
|
||||
## Sleep & Recovery Protocols
|
||||
|
||||
### Ultradian Reset
|
||||
**Duration**: 20 minutes
|
||||
**Trigger**: End of a 90-minute focus block, drowsiness rising
|
||||
**Instructions**:
|
||||
1. Set a timer for 20 minutes
|
||||
2. No agenda — just rest (don't force sleep)
|
||||
3. Dim lights if possible, close eyes
|
||||
4. Let mind wander without structure
|
||||
**Effect**: Aligns with 90-minute ultradian rhythm, restores cognitive resources
|
||||
|
||||
### Wake Reset
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: narcolepsy_index > 40, severe drowsiness
|
||||
**Instructions**:
|
||||
1. Splash cold water on your face and wrists
|
||||
2. Do 20 seconds of Kapalabhati breath (sharp nasal exhales)
|
||||
3. Expose yourself to bright light for 2–3 minutes
|
||||
**Effect**: Acute arousal response, suppresses drowsiness
|
||||
|
||||
### NSDR (Non-Sleep Deep Rest / Yoga Nidra)
|
||||
**Duration**: 20–30 minutes
|
||||
**Trigger**: Accumulated fatigue, need deep recovery without sleeping
|
||||
**Instructions**:
|
||||
1. Lie on your back, palms up
|
||||
2. Close your eyes and do a slow body scan from toes to crown
|
||||
3. At each body part, notice sensation without changing anything
|
||||
4. If you fall asleep, that's fine — set an alarm
|
||||
**Effect**: Restores dopamine and cognitive resources without sleep inertia
|
||||
|
||||
### Power Nap
|
||||
**Duration**: 10–20 minutes (set alarm!)
|
||||
**Trigger**: Drowsiness > 0.70, post-lunch slump, Theta dominant
|
||||
**Instructions**:
|
||||
1. Set alarm for 20 minutes maximum (avoids N3 sleep inertia)
|
||||
2. Lie down or recline
|
||||
3. Even if you don't fully sleep, rest with eyes closed
|
||||
4. On waking: 30 seconds of stretching before resuming work
|
||||
**Effect**: Restores focus and alertness for 2–3 hours
|
||||
|
||||
### Sleep Wind-Down
|
||||
**Duration**: 60 minutes before bed
|
||||
**Trigger**: Evening session, rising drowsiness, pre-sleep
|
||||
**Instructions**:
|
||||
1. Dim all screens to night mode
|
||||
2. Stop new learning or complex tasks
|
||||
3. Do a mind dump of tomorrow's tasks
|
||||
4. 10 minutes of progressive relaxation or 4-7-8 breathing
|
||||
5. Keep room cool (65–68°F / 18–20°C)
|
||||
|
||||
---
|
||||
|
||||
## Somatic & Physical Protocols
|
||||
|
||||
### Progressive Muscle Relaxation (PMR)
|
||||
**Duration**: 10 minutes
|
||||
**Trigger**: Relaxation < 0.25, HRV declining over session
|
||||
**Instructions**:
|
||||
1. Start with feet — tense for 5 seconds, release for 8–10 seconds
|
||||
2. Move upward: calves → thighs → abdomen → hands → arms → shoulders → face
|
||||
3. Hold each tension 5 seconds, release 8–10 seconds
|
||||
4. End with 3 deep breaths
|
||||
|
||||
### Grounding (5-4-3-2-1)
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: Panic, dissociation, acute anxiety spike
|
||||
**Instructions**:
|
||||
1. Name 5 things you can see
|
||||
2. Name 4 things you can touch
|
||||
3. Name 3 things you can hear
|
||||
4. Name 2 things you can smell
|
||||
5. Name 1 thing you can taste
|
||||
|
||||
### 20-20-20 Vision Reset
|
||||
**Duration**: 20 seconds
|
||||
**Trigger**: Extended screen time, eye strain
|
||||
**Instructions**:
|
||||
1. Every 20 minutes of screen time
|
||||
2. Look at something 20 feet away
|
||||
3. For 20 seconds
|
||||
|
||||
### Neck Release Sequence
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: High stillness (> 0.85) + headache_index elevated
|
||||
**Instructions**:
|
||||
1. Ear-to-shoulder tilt — hold 15 seconds each side
|
||||
2. Chin tucks — 10 reps (pull chin straight back)
|
||||
3. Gentle neck circles — 5 each direction
|
||||
4. Shoulder shrugs — 10 reps (squeeze up, release)
|
||||
|
||||
### Motor Cortex Activation
|
||||
**Duration**: 2 minutes
|
||||
**Trigger**: Very high stillness, prolonged static sitting
|
||||
**Instructions**:
|
||||
1. Cross-body movements: touch right hand to left knee, alternate 10 times
|
||||
2. Shake out hands and feet for 15 seconds
|
||||
3. Roll ankles and wrists 5 times each direction
|
||||
**Effect**: Resets proprioception, activates motor cortex
|
||||
|
||||
### Cognitive Load Offload (Mind Dump)
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: Cognitive load > 0.70 sustained, racing thoughts, high beta
|
||||
**Instructions**:
|
||||
1. Open a blank document or grab paper
|
||||
2. Write everything on your mind without filtering or organizing
|
||||
3. Brain-dump worries, tasks, ideas — anything occupying working memory
|
||||
4. Close the document (review later if needed)
|
||||
**Effect**: Externalizing working memory can reduce cognitive load by 20–40%
|
||||
|
||||
---
|
||||
|
||||
## Digital & Lifestyle Protocols
|
||||
|
||||
### Craving Surf
|
||||
**Duration**: 90 seconds
|
||||
**Trigger**: Phone addiction signals, urge to check social media
|
||||
**Instructions**:
|
||||
1. Notice the urge to check your phone
|
||||
2. Don't act on it — just observe for 90 seconds
|
||||
3. Notice: does the urge peak and then fade?
|
||||
4. Resume what you were doing
|
||||
**Effect**: Breaks automatic dopamine-seeking loop
|
||||
|
||||
### Dopamine Palette Reset
|
||||
**Duration**: Ongoing
|
||||
**Trigger**: Flatness from short-form content spikes
|
||||
**Instructions**:
|
||||
1. Identify activities that provide sustained reward (reading, cooking, walking)
|
||||
2. Replace 15 minutes of scrolling with one sustained-reward activity
|
||||
3. Track mood before/after for 3 days
|
||||
|
||||
### Digital Sunset
|
||||
**Duration**: 60–90 minutes before bed
|
||||
**Trigger**: Evening, pre-sleep routine
|
||||
**Instructions**:
|
||||
1. Hard stop on all screens 60–90 minutes before bed
|
||||
2. Switch to non-screen activities: reading, conversation, stretching
|
||||
3. If screens are necessary, use night mode at minimum brightness
|
||||
|
||||
---
|
||||
|
||||
## Dietary Protocols
|
||||
|
||||
### Caffeine Timing
|
||||
**Trigger**: Morning routine, anxiety_index
|
||||
**Guidelines**:
|
||||
- Consume caffeine 90–120 minutes after waking (cortisol has already peaked)
|
||||
- None after 2 PM (half-life ~6 hours)
|
||||
- If anxiety_index > 50, stack with L-theanine (200mg) to smooth the curve
|
||||
|
||||
### Post-Meal Energy Crash
|
||||
**Trigger**: Post-lunch drowsiness spike
|
||||
**Instructions**:
|
||||
1. 5-minute brisk walk immediately after eating
|
||||
2. 10 minutes of sunlight exposure
|
||||
**Effect**: Counters post-prandial drowsiness
|
||||
|
||||
---
|
||||
|
||||
## Motivation & Planning Protocols
|
||||
|
||||
### WOOP (Wish, Outcome, Obstacle, Plan)
|
||||
**Duration**: 5 minutes
|
||||
**Trigger**: Low engagement before a task
|
||||
**Instructions**:
|
||||
1. **Wish**: What do you want to accomplish in this session?
|
||||
2. **Outcome**: What's the best possible result? Visualize it.
|
||||
3. **Obstacle**: What internal obstacle might get in the way?
|
||||
4. **Plan**: "If [obstacle], then I will [action]."
|
||||
**Effect**: Mental contrasting improves follow-through by 2–3x
|
||||
|
||||
### Pre-Task Priming
|
||||
**Duration**: 3 minutes
|
||||
**Trigger**: Low engagement at session start, drowsiness < 0.50
|
||||
**Instructions**:
|
||||
1. Set a clear intention for the next work block
|
||||
2. Write down the single most important task
|
||||
3. Do 10 jumping jacks or 20 deep breaths
|
||||
4. Start with the easiest sub-task to build momentum
|
||||
|
||||
---
|
||||
|
||||
## Protocol Execution Guidelines
|
||||
|
||||
When guiding the user through a protocol:
|
||||
1. **Match one protocol** to the single most salient metric signal
|
||||
2. **Explain the metric connection** — why this protocol for this state
|
||||
3. **Ask permission** — never start without the user's consent
|
||||
4. **Announce each step** clearly with timing
|
||||
5. **Check in after** — run `npx neuroskill status --json` to see if metrics improved
|
||||
6. **Label the moment** — `npx neuroskill label "post-protocol: [name]"` for tracking
|
||||
|
||||
### Timing Guidelines for Step-by-Step Guidance
|
||||
- Breath inhale: 3–5 seconds
|
||||
- Breath hold: 2–4 seconds
|
||||
- Breath exhale: 4–8 seconds
|
||||
- Muscle tense: 5 seconds
|
||||
- Muscle release: 8–10 seconds
|
||||
- Body-scan region: 10–15 seconds
|
||||
@@ -0,0 +1,3 @@
|
||||
# MCP
|
||||
|
||||
Skills for building, testing, and deploying MCP (Model Context Protocol) servers.
|
||||
@@ -0,0 +1,300 @@
|
||||
---
|
||||
name: fastmcp
|
||||
description: Build, test, inspect, install, and deploy MCP servers with FastMCP in Python. Use when creating a new MCP server, wrapping an API or database as MCP tools, exposing resources or prompts, or preparing a FastMCP server for Claude Code, Cursor, or HTTP deployment.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [MCP, FastMCP, Python, Tools, Resources, Prompts, Deployment]
|
||||
homepage: https://gofastmcp.com
|
||||
related_skills: [native-mcp, mcporter]
|
||||
prerequisites:
|
||||
commands: [python3]
|
||||
---
|
||||
|
||||
# FastMCP
|
||||
|
||||
Build MCP servers in Python with FastMCP, validate them locally, install them into MCP clients, and deploy them as HTTP endpoints.
|
||||
|
||||
## When to Use
|
||||
|
||||
Use this skill when the task is to:
|
||||
|
||||
- create a new MCP server in Python
|
||||
- wrap an API, database, CLI, or file-processing workflow as MCP tools
|
||||
- expose resources or prompts in addition to tools
|
||||
- smoke-test a server with the FastMCP CLI before wiring it into Hermes or another client
|
||||
- install a server into Claude Code, Claude Desktop, Cursor, or a similar MCP client
|
||||
- prepare a FastMCP server repo for HTTP deployment
|
||||
|
||||
Use `native-mcp` when the server already exists and only needs to be connected to Hermes. Use `mcporter` when the goal is ad-hoc CLI access to an existing MCP server instead of building one.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Install FastMCP in the working environment first:
|
||||
|
||||
```bash
|
||||
pip install fastmcp
|
||||
fastmcp version
|
||||
```
|
||||
|
||||
For the API template, install `httpx` if it is not already present:
|
||||
|
||||
```bash
|
||||
pip install httpx
|
||||
```
|
||||
|
||||
## Included Files
|
||||
|
||||
### Templates
|
||||
|
||||
- `templates/api_wrapper.py` - REST API wrapper with auth header support
|
||||
- `templates/database_server.py` - read-only SQLite query server
|
||||
- `templates/file_processor.py` - text-file inspection and search server
|
||||
|
||||
### Scripts
|
||||
|
||||
- `scripts/scaffold_fastmcp.py` - copy a starter template and replace the server name placeholder
|
||||
|
||||
### References
|
||||
|
||||
- `references/fastmcp-cli.md` - FastMCP CLI workflow, installation targets, and deployment checks
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Pick the Smallest Viable Server Shape
|
||||
|
||||
Choose the narrowest useful surface area first:
|
||||
|
||||
- API wrapper: start with 1-3 high-value endpoints, not the whole API
|
||||
- database server: expose read-only introspection and a constrained query path
|
||||
- file processor: expose deterministic operations with explicit path arguments
|
||||
- prompts/resources: add only when the client needs reusable prompt templates or discoverable documents
|
||||
|
||||
Prefer a thin server with good names, docstrings, and schemas over a large server with vague tools.
|
||||
|
||||
### 2. Scaffold from a Template
|
||||
|
||||
Copy a template directly or use the scaffold helper:
|
||||
|
||||
```bash
|
||||
python ~/.hermes/skills/mcp/fastmcp/scripts/scaffold_fastmcp.py \
|
||||
--template api_wrapper \
|
||||
--name "Acme API" \
|
||||
--output ./acme_server.py
|
||||
```
|
||||
|
||||
Available templates:
|
||||
|
||||
```bash
|
||||
python ~/.hermes/skills/mcp/fastmcp/scripts/scaffold_fastmcp.py --list
|
||||
```
|
||||
|
||||
If copying manually, replace `__SERVER_NAME__` with a real server name.
|
||||
|
||||
### 3. Implement Tools First
|
||||
|
||||
Start with `@mcp.tool` functions before adding resources or prompts.
|
||||
|
||||
Rules for tool design:
|
||||
|
||||
- Give every tool a concrete verb-based name
|
||||
- Write docstrings as user-facing tool descriptions
|
||||
- Keep parameters explicit and typed
|
||||
- Return structured JSON-safe data where possible
|
||||
- Validate unsafe inputs early
|
||||
- Prefer read-only behavior by default for first versions
|
||||
|
||||
Good tool examples:
|
||||
|
||||
- `get_customer`
|
||||
- `search_tickets`
|
||||
- `describe_table`
|
||||
- `summarize_text_file`
|
||||
|
||||
Weak tool examples:
|
||||
|
||||
- `run`
|
||||
- `process`
|
||||
- `do_thing`
|
||||
|
||||
### 4. Add Resources and Prompts Only When They Help
|
||||
|
||||
Add `@mcp.resource` when the client benefits from fetching stable read-only content such as schemas, policy docs, or generated reports.
|
||||
|
||||
Add `@mcp.prompt` when the server should provide a reusable prompt template for a known workflow.
|
||||
|
||||
Do not turn every document into a prompt. Prefer:
|
||||
|
||||
- tools for actions
|
||||
- resources for data/document retrieval
|
||||
- prompts for reusable LLM instructions
|
||||
|
||||
### 5. Test the Server Before Integrating It Anywhere
|
||||
|
||||
Use the FastMCP CLI for local validation:
|
||||
|
||||
```bash
|
||||
fastmcp inspect acme_server.py:mcp
|
||||
fastmcp list acme_server.py --json
|
||||
fastmcp call acme_server.py search_resources query=router limit=5 --json
|
||||
```
|
||||
|
||||
For fast iterative debugging, run the server locally:
|
||||
|
||||
```bash
|
||||
fastmcp run acme_server.py:mcp
|
||||
```
|
||||
|
||||
To test HTTP transport locally:
|
||||
|
||||
```bash
|
||||
fastmcp run acme_server.py:mcp --transport http --host 127.0.0.1 --port 8000
|
||||
fastmcp list http://127.0.0.1:8000/mcp --json
|
||||
fastmcp call http://127.0.0.1:8000/mcp search_resources query=router --json
|
||||
```
|
||||
|
||||
Always run at least one real `fastmcp call` against each new tool before claiming the server works.
|
||||
|
||||
### 6. Install into a Client When Local Validation Passes
|
||||
|
||||
FastMCP can register the server with supported MCP clients:
|
||||
|
||||
```bash
|
||||
fastmcp install claude-code acme_server.py
|
||||
fastmcp install claude-desktop acme_server.py
|
||||
fastmcp install cursor acme_server.py -e .
|
||||
```
|
||||
|
||||
Use `fastmcp discover` to inspect named MCP servers already configured on the machine.
|
||||
|
||||
When the goal is Hermes integration, either:
|
||||
|
||||
- configure the server in `~/.hermes/config.yaml` using the `native-mcp` skill, or
|
||||
- keep using FastMCP CLI commands during development until the interface stabilizes
|
||||
|
||||
### 7. Deploy After the Local Contract Is Stable
|
||||
|
||||
For managed hosting, Prefect Horizon is the path FastMCP documents most directly. Before deployment:
|
||||
|
||||
```bash
|
||||
fastmcp inspect acme_server.py:mcp
|
||||
```
|
||||
|
||||
Make sure the repo contains:
|
||||
|
||||
- a Python file with the FastMCP server object
|
||||
- `requirements.txt` or `pyproject.toml`
|
||||
- any environment-variable documentation needed for deployment
|
||||
|
||||
For generic HTTP hosting, validate the HTTP transport locally first, then deploy on any Python-compatible platform that can expose the server port.
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### API Wrapper Pattern
|
||||
|
||||
Use when exposing a REST or HTTP API as MCP tools.
|
||||
|
||||
Recommended first slice:
|
||||
|
||||
- one read path
|
||||
- one list/search path
|
||||
- optional health check
|
||||
|
||||
Implementation notes:
|
||||
|
||||
- keep auth in environment variables, not hardcoded
|
||||
- centralize request logic in one helper
|
||||
- surface API errors with concise context
|
||||
- normalize inconsistent upstream payloads before returning them
|
||||
|
||||
Start from `templates/api_wrapper.py`.
|
||||
|
||||
### Database Pattern
|
||||
|
||||
Use when exposing safe query and inspection capabilities.
|
||||
|
||||
Recommended first slice:
|
||||
|
||||
- `list_tables`
|
||||
- `describe_table`
|
||||
- one constrained read query tool
|
||||
|
||||
Implementation notes:
|
||||
|
||||
- default to read-only DB access
|
||||
- reject non-`SELECT` SQL in early versions
|
||||
- limit row counts
|
||||
- return rows plus column names
|
||||
|
||||
Start from `templates/database_server.py`.
|
||||
|
||||
### File Processor Pattern
|
||||
|
||||
Use when the server needs to inspect or transform files on demand.
|
||||
|
||||
Recommended first slice:
|
||||
|
||||
- summarize file contents
|
||||
- search within files
|
||||
- extract deterministic metadata
|
||||
|
||||
Implementation notes:
|
||||
|
||||
- accept explicit file paths
|
||||
- check for missing files and encoding failures
|
||||
- cap previews and result counts
|
||||
- avoid shelling out unless a specific external tool is required
|
||||
|
||||
Start from `templates/file_processor.py`.
|
||||
|
||||
## Quality Bar
|
||||
|
||||
Before handing off a FastMCP server, verify all of the following:
|
||||
|
||||
- server imports cleanly
|
||||
- `fastmcp inspect <file.py:mcp>` succeeds
|
||||
- `fastmcp list <server spec> --json` succeeds
|
||||
- every new tool has at least one real `fastmcp call`
|
||||
- environment variables are documented
|
||||
- the tool surface is small enough to understand without guesswork
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### FastMCP command missing
|
||||
|
||||
Install the package in the active environment:
|
||||
|
||||
```bash
|
||||
pip install fastmcp
|
||||
fastmcp version
|
||||
```
|
||||
|
||||
### `fastmcp inspect` fails
|
||||
|
||||
Check that:
|
||||
|
||||
- the file imports without side effects that crash
|
||||
- the FastMCP instance is named correctly in `<file.py:object>`
|
||||
- optional dependencies from the template are installed
|
||||
|
||||
### Tool works in Python but not through CLI
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
fastmcp list server.py --json
|
||||
fastmcp call server.py your_tool_name --json
|
||||
```
|
||||
|
||||
This usually exposes naming mismatches, missing required arguments, or non-serializable return values.
|
||||
|
||||
### Hermes cannot see the deployed server
|
||||
|
||||
The server-building part may be correct while the Hermes config is not. Load the `native-mcp` skill and configure the server in `~/.hermes/config.yaml`, then restart Hermes.
|
||||
|
||||
## References
|
||||
|
||||
For CLI details, install targets, and deployment checks, read `references/fastmcp-cli.md`.
|
||||
@@ -0,0 +1,110 @@
|
||||
# FastMCP CLI Reference
|
||||
|
||||
Use this file when the task needs exact FastMCP CLI workflows rather than the higher-level guidance in `SKILL.md`.
|
||||
|
||||
## Install and Verify
|
||||
|
||||
```bash
|
||||
pip install fastmcp
|
||||
fastmcp version
|
||||
```
|
||||
|
||||
FastMCP documents `pip install fastmcp` and `fastmcp version` as the baseline installation and verification path.
|
||||
|
||||
## Run a Server
|
||||
|
||||
Run a server object from a Python file:
|
||||
|
||||
```bash
|
||||
fastmcp run server.py:mcp
|
||||
```
|
||||
|
||||
Run the same server over HTTP:
|
||||
|
||||
```bash
|
||||
fastmcp run server.py:mcp --transport http --host 127.0.0.1 --port 8000
|
||||
```
|
||||
|
||||
## Inspect a Server
|
||||
|
||||
Inspect what FastMCP will expose:
|
||||
|
||||
```bash
|
||||
fastmcp inspect server.py:mcp
|
||||
```
|
||||
|
||||
This is also the check FastMCP recommends before deploying to Prefect Horizon.
|
||||
|
||||
## List and Call Tools
|
||||
|
||||
List tools from a Python file:
|
||||
|
||||
```bash
|
||||
fastmcp list server.py --json
|
||||
```
|
||||
|
||||
List tools from an HTTP endpoint:
|
||||
|
||||
```bash
|
||||
fastmcp list http://127.0.0.1:8000/mcp --json
|
||||
```
|
||||
|
||||
Call a tool with key-value arguments:
|
||||
|
||||
```bash
|
||||
fastmcp call server.py search_resources query=router limit=5 --json
|
||||
```
|
||||
|
||||
Call a tool with a full JSON input payload:
|
||||
|
||||
```bash
|
||||
fastmcp call server.py create_item '{"name": "Widget", "tags": ["sale"]}' --json
|
||||
```
|
||||
|
||||
## Discover Named MCP Servers
|
||||
|
||||
Find named servers already configured in local MCP-aware tools:
|
||||
|
||||
```bash
|
||||
fastmcp discover
|
||||
```
|
||||
|
||||
FastMCP documents name-based resolution for Claude Desktop, Claude Code, Cursor, Gemini, Goose, and `./mcp.json`.
|
||||
|
||||
## Install into MCP Clients
|
||||
|
||||
Register a server with common clients:
|
||||
|
||||
```bash
|
||||
fastmcp install claude-code server.py
|
||||
fastmcp install claude-desktop server.py
|
||||
fastmcp install cursor server.py -e .
|
||||
```
|
||||
|
||||
FastMCP notes that client installs run in isolated environments, so declare dependencies explicitly when needed with flags such as `--with`, `--env-file`, or editable installs.
|
||||
|
||||
## Deployment Checks
|
||||
|
||||
### Prefect Horizon
|
||||
|
||||
Before pushing to Horizon:
|
||||
|
||||
```bash
|
||||
fastmcp inspect server.py:mcp
|
||||
```
|
||||
|
||||
FastMCP’s Horizon docs expect:
|
||||
|
||||
- a GitHub repo
|
||||
- a Python file containing the FastMCP server object
|
||||
- dependencies declared in `requirements.txt` or `pyproject.toml`
|
||||
- an entrypoint like `main.py:mcp`
|
||||
|
||||
### Generic HTTP Hosting
|
||||
|
||||
Before shipping to any other host:
|
||||
|
||||
1. Start the server locally with HTTP transport.
|
||||
2. Verify `fastmcp list` against the local `/mcp` URL.
|
||||
3. Verify at least one `fastmcp call`.
|
||||
4. Document required environment variables.
|
||||
@@ -0,0 +1,56 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Copy a FastMCP starter template into a working file."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
SKILL_DIR = SCRIPT_DIR.parent
|
||||
TEMPLATE_DIR = SKILL_DIR / "templates"
|
||||
PLACEHOLDER = "__SERVER_NAME__"
|
||||
|
||||
|
||||
def list_templates() -> list[str]:
|
||||
return sorted(path.stem for path in TEMPLATE_DIR.glob("*.py"))
|
||||
|
||||
|
||||
def render_template(template_name: str, server_name: str) -> str:
|
||||
template_path = TEMPLATE_DIR / f"{template_name}.py"
|
||||
if not template_path.exists():
|
||||
available = ", ".join(list_templates())
|
||||
raise SystemExit(f"Unknown template '{template_name}'. Available: {available}")
|
||||
return template_path.read_text(encoding="utf-8").replace(PLACEHOLDER, server_name)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--template", help="Template name without .py suffix")
|
||||
parser.add_argument("--name", help="FastMCP server display name")
|
||||
parser.add_argument("--output", help="Destination Python file path")
|
||||
parser.add_argument("--force", action="store_true", help="Overwrite an existing output file")
|
||||
parser.add_argument("--list", action="store_true", help="List available templates and exit")
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.list:
|
||||
for name in list_templates():
|
||||
print(name)
|
||||
return 0
|
||||
|
||||
if not args.template or not args.name or not args.output:
|
||||
parser.error("--template, --name, and --output are required unless --list is used")
|
||||
|
||||
output_path = Path(args.output).expanduser()
|
||||
if output_path.exists() and not args.force:
|
||||
raise SystemExit(f"Refusing to overwrite existing file: {output_path}")
|
||||
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
output_path.write_text(render_template(args.template, args.name), encoding="utf-8")
|
||||
print(f"Wrote {output_path}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,54 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
from fastmcp import FastMCP
|
||||
|
||||
|
||||
mcp = FastMCP("__SERVER_NAME__")
|
||||
|
||||
API_BASE_URL = os.getenv("API_BASE_URL", "https://api.example.com")
|
||||
API_TOKEN = os.getenv("API_TOKEN")
|
||||
REQUEST_TIMEOUT = float(os.getenv("API_TIMEOUT_SECONDS", "20"))
|
||||
|
||||
|
||||
def _headers() -> dict[str, str]:
|
||||
headers = {"Accept": "application/json"}
|
||||
if API_TOKEN:
|
||||
headers["Authorization"] = f"Bearer {API_TOKEN}"
|
||||
return headers
|
||||
|
||||
|
||||
def _request(method: str, path: str, *, params: dict[str, Any] | None = None) -> Any:
|
||||
url = f"{API_BASE_URL.rstrip('/')}/{path.lstrip('/')}"
|
||||
with httpx.Client(timeout=REQUEST_TIMEOUT, headers=_headers()) as client:
|
||||
response = client.request(method, url, params=params)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def health_check() -> dict[str, Any]:
|
||||
"""Check whether the upstream API is reachable."""
|
||||
payload = _request("GET", "/health")
|
||||
return {"base_url": API_BASE_URL, "result": payload}
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def get_resource(resource_id: str) -> dict[str, Any]:
|
||||
"""Fetch one resource by ID from the upstream API."""
|
||||
payload = _request("GET", f"/resources/{resource_id}")
|
||||
return {"resource_id": resource_id, "data": payload}
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def search_resources(query: str, limit: int = 10) -> dict[str, Any]:
|
||||
"""Search upstream resources by query string."""
|
||||
payload = _request("GET", "/resources", params={"q": query, "limit": limit})
|
||||
return {"query": query, "limit": limit, "results": payload}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
mcp.run()
|
||||
@@ -0,0 +1,77 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import sqlite3
|
||||
from typing import Any
|
||||
|
||||
from fastmcp import FastMCP
|
||||
|
||||
|
||||
mcp = FastMCP("__SERVER_NAME__")
|
||||
|
||||
DATABASE_PATH = os.getenv("SQLITE_PATH", "./app.db")
|
||||
MAX_ROWS = int(os.getenv("SQLITE_MAX_ROWS", "200"))
|
||||
TABLE_NAME_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$")
|
||||
|
||||
|
||||
def _connect() -> sqlite3.Connection:
|
||||
return sqlite3.connect(f"file:{DATABASE_PATH}?mode=ro", uri=True)
|
||||
|
||||
|
||||
def _reject_mutation(sql: str) -> None:
|
||||
normalized = sql.strip().lower()
|
||||
if not normalized.startswith("select"):
|
||||
raise ValueError("Only SELECT queries are allowed")
|
||||
|
||||
|
||||
def _validate_table_name(table_name: str) -> str:
|
||||
if not TABLE_NAME_RE.fullmatch(table_name):
|
||||
raise ValueError("Invalid table name")
|
||||
return table_name
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def list_tables() -> list[str]:
|
||||
"""List user-defined SQLite tables."""
|
||||
with _connect() as conn:
|
||||
rows = conn.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%' ORDER BY name"
|
||||
).fetchall()
|
||||
return [row[0] for row in rows]
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def describe_table(table_name: str) -> list[dict[str, Any]]:
|
||||
"""Describe columns for a SQLite table."""
|
||||
safe_table_name = _validate_table_name(table_name)
|
||||
with _connect() as conn:
|
||||
rows = conn.execute(f"PRAGMA table_info({safe_table_name})").fetchall()
|
||||
return [
|
||||
{
|
||||
"cid": row[0],
|
||||
"name": row[1],
|
||||
"type": row[2],
|
||||
"notnull": bool(row[3]),
|
||||
"default": row[4],
|
||||
"pk": bool(row[5]),
|
||||
}
|
||||
for row in rows
|
||||
]
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def query(sql: str, limit: int = 50) -> dict[str, Any]:
|
||||
"""Run a read-only SELECT query and return rows plus column names."""
|
||||
_reject_mutation(sql)
|
||||
safe_limit = max(0, min(limit, MAX_ROWS))
|
||||
wrapped_sql = f"SELECT * FROM ({sql.strip().rstrip(';')}) LIMIT {safe_limit}"
|
||||
with _connect() as conn:
|
||||
cursor = conn.execute(wrapped_sql)
|
||||
columns = [column[0] for column in cursor.description or []]
|
||||
rows = [dict(zip(columns, row)) for row in cursor.fetchall()]
|
||||
return {"limit": safe_limit, "columns": columns, "rows": rows}
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
mcp.run()
|
||||
@@ -0,0 +1,55 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from fastmcp import FastMCP
|
||||
|
||||
|
||||
mcp = FastMCP("__SERVER_NAME__")
|
||||
|
||||
|
||||
def _read_text(path: str) -> str:
|
||||
file_path = Path(path).expanduser()
|
||||
try:
|
||||
return file_path.read_text(encoding="utf-8")
|
||||
except FileNotFoundError as exc:
|
||||
raise ValueError(f"File not found: {file_path}") from exc
|
||||
except UnicodeDecodeError as exc:
|
||||
raise ValueError(f"File is not valid UTF-8 text: {file_path}") from exc
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def summarize_text_file(path: str, preview_chars: int = 1200) -> dict[str, int | str]:
|
||||
"""Return basic metadata and a preview for a UTF-8 text file."""
|
||||
file_path = Path(path).expanduser()
|
||||
text = _read_text(path)
|
||||
return {
|
||||
"path": str(file_path),
|
||||
"characters": len(text),
|
||||
"lines": len(text.splitlines()),
|
||||
"preview": text[:preview_chars],
|
||||
}
|
||||
|
||||
|
||||
@mcp.tool
|
||||
def search_text_file(path: str, needle: str, max_matches: int = 20) -> dict[str, Any]:
|
||||
"""Find matching lines in a UTF-8 text file."""
|
||||
file_path = Path(path).expanduser()
|
||||
matches: list[dict[str, Any]] = []
|
||||
for line_number, line in enumerate(_read_text(path).splitlines(), start=1):
|
||||
if needle.lower() in line.lower():
|
||||
matches.append({"line_number": line_number, "line": line})
|
||||
if len(matches) >= max_matches:
|
||||
break
|
||||
return {"path": str(file_path), "needle": needle, "matches": matches}
|
||||
|
||||
|
||||
@mcp.resource("file://{path}")
|
||||
def read_file_resource(path: str) -> str:
|
||||
"""Expose a text file as a resource."""
|
||||
return _read_text(path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
mcp.run()
|
||||
@@ -0,0 +1,123 @@
|
||||
---
|
||||
name: mcporter
|
||||
description: Use the mcporter CLI to list, configure, auth, and call MCP servers/tools directly (HTTP or stdio), including ad-hoc servers, config edits, and CLI/type generation.
|
||||
version: 1.0.0
|
||||
author: community
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [MCP, Tools, API, Integrations, Interop]
|
||||
homepage: https://mcporter.dev
|
||||
prerequisites:
|
||||
commands: [npx]
|
||||
---
|
||||
|
||||
# mcporter
|
||||
|
||||
Use `mcporter` to discover, call, and manage [MCP (Model Context Protocol)](https://modelcontextprotocol.io/) servers and tools directly from the terminal.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Requires Node.js:
|
||||
```bash
|
||||
# No install needed (runs via npx)
|
||||
npx mcporter list
|
||||
|
||||
# Or install globally
|
||||
npm install -g mcporter
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# List MCP servers already configured on this machine
|
||||
mcporter list
|
||||
|
||||
# List tools for a specific server with schema details
|
||||
mcporter list <server> --schema
|
||||
|
||||
# Call a tool
|
||||
mcporter call <server.tool> key=value
|
||||
```
|
||||
|
||||
## Discovering MCP Servers
|
||||
|
||||
mcporter auto-discovers servers configured by other MCP clients (Claude Desktop, Cursor, etc.) on the machine. To find new servers to use, browse registries like [mcpfinder.dev](https://mcpfinder.dev) or [mcp.so](https://mcp.so), then connect ad-hoc:
|
||||
|
||||
```bash
|
||||
# Connect to any MCP server by URL (no config needed)
|
||||
mcporter list --http-url https://some-mcp-server.com --name my_server
|
||||
|
||||
# Or run a stdio server on the fly
|
||||
mcporter list --stdio "npx -y @modelcontextprotocol/server-filesystem" --name fs
|
||||
```
|
||||
|
||||
## Calling Tools
|
||||
|
||||
```bash
|
||||
# Key=value syntax
|
||||
mcporter call linear.list_issues team=ENG limit:5
|
||||
|
||||
# Function syntax
|
||||
mcporter call "linear.create_issue(title: \"Bug fix needed\")"
|
||||
|
||||
# Ad-hoc HTTP server (no config needed)
|
||||
mcporter call https://api.example.com/mcp.fetch url=https://example.com
|
||||
|
||||
# Ad-hoc stdio server
|
||||
mcporter call --stdio "bun run ./server.ts" scrape url=https://example.com
|
||||
|
||||
# JSON payload
|
||||
mcporter call <server.tool> --args '{"limit": 5}'
|
||||
|
||||
# Machine-readable output (recommended for Hermes)
|
||||
mcporter call <server.tool> key=value --output json
|
||||
```
|
||||
|
||||
## Auth and Config
|
||||
|
||||
```bash
|
||||
# OAuth login for a server
|
||||
mcporter auth <server | url> [--reset]
|
||||
|
||||
# Manage config
|
||||
mcporter config list
|
||||
mcporter config get <key>
|
||||
mcporter config add <server>
|
||||
mcporter config remove <server>
|
||||
mcporter config import <path>
|
||||
```
|
||||
|
||||
Config file location: `./config/mcporter.json` (override with `--config`).
|
||||
|
||||
## Daemon
|
||||
|
||||
For persistent server connections:
|
||||
```bash
|
||||
mcporter daemon start
|
||||
mcporter daemon status
|
||||
mcporter daemon stop
|
||||
mcporter daemon restart
|
||||
```
|
||||
|
||||
## Code Generation
|
||||
|
||||
```bash
|
||||
# Generate a CLI wrapper for an MCP server
|
||||
mcporter generate-cli --server <name>
|
||||
mcporter generate-cli --command <url>
|
||||
|
||||
# Inspect a generated CLI
|
||||
mcporter inspect-cli <path> [--json]
|
||||
|
||||
# Generate TypeScript types/client
|
||||
mcporter emit-ts <server> --mode client
|
||||
mcporter emit-ts <server> --mode types
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- Use `--output json` for structured output that's easier to parse
|
||||
- Ad-hoc servers (HTTP URL or `--stdio` command) work without any config — useful for one-off calls
|
||||
- OAuth auth may require interactive browser flow — use `terminal(command="mcporter auth <server>", pty=true)` if needed
|
||||
@@ -0,0 +1,2 @@
|
||||
Optional migration workflows for importing user state and customizations from
|
||||
other agent systems into Hermes Agent.
|
||||
@@ -0,0 +1,298 @@
|
||||
---
|
||||
name: openclaw-migration
|
||||
description: Migrate a user's OpenClaw customization footprint into Hermes Agent. Imports Hermes-compatible memories, SOUL.md, command allowlists, user skills, and selected workspace assets from ~/.openclaw, then reports exactly what could not be migrated and why.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent (Nous Research)
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Migration, OpenClaw, Hermes, Memory, Persona, Import]
|
||||
related_skills: [hermes-agent]
|
||||
---
|
||||
|
||||
# OpenClaw -> Hermes Migration
|
||||
|
||||
Use this skill when a user wants to move their OpenClaw setup into Hermes Agent with minimal manual cleanup.
|
||||
|
||||
## CLI Command
|
||||
|
||||
For a quick, non-interactive migration, use the built-in CLI command:
|
||||
|
||||
```bash
|
||||
hermes claw migrate # Full interactive migration
|
||||
hermes claw migrate --dry-run # Preview what would be migrated
|
||||
hermes claw migrate --preset user-data # Migrate without secrets
|
||||
hermes claw migrate --overwrite # Overwrite existing conflicts
|
||||
hermes claw migrate --source /custom/path/.openclaw # Custom source
|
||||
```
|
||||
|
||||
The CLI command runs the same migration script described below. Use this skill (via the agent) when you want an interactive, guided migration with dry-run previews and per-item conflict resolution.
|
||||
|
||||
**First-time setup:** The `hermes setup` wizard automatically detects `~/.openclaw` and offers migration before configuration begins.
|
||||
|
||||
## What this skill does
|
||||
|
||||
It uses `scripts/openclaw_to_hermes.py` to:
|
||||
|
||||
- import `SOUL.md` into the Hermes home directory as `SOUL.md`
|
||||
- transform OpenClaw `MEMORY.md` and `USER.md` into Hermes memory entries
|
||||
- merge OpenClaw command approval patterns into Hermes `command_allowlist`
|
||||
- migrate Hermes-compatible messaging settings such as `TELEGRAM_ALLOWED_USERS`, and map OpenClaw workspace settings to Hermes working-directory configuration
|
||||
- copy OpenClaw skills into `~/.hermes/skills/openclaw-imports/`
|
||||
- optionally copy the OpenClaw workspace instructions file into a chosen Hermes workspace
|
||||
- mirror compatible workspace assets such as `workspace/tts/` into `~/.hermes/tts/`
|
||||
- archive non-secret docs that do not have a direct Hermes destination
|
||||
- produce a structured report listing migrated items, conflicts, skipped items, and reasons
|
||||
|
||||
## Path resolution
|
||||
|
||||
The helper script lives in this skill directory at:
|
||||
|
||||
- `scripts/openclaw_to_hermes.py`
|
||||
|
||||
When this skill is installed from the Skills Hub, the normal location is:
|
||||
|
||||
- `~/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py`
|
||||
|
||||
Do not guess a shorter path like `~/.hermes/skills/openclaw-migration/...`.
|
||||
|
||||
Before running the helper:
|
||||
|
||||
1. Prefer the installed path under `~/.hermes/skills/migration/openclaw-migration/`.
|
||||
2. If that path fails, inspect the installed skill directory and resolve the script relative to the installed `SKILL.md`.
|
||||
3. Only use `find` as a fallback if the installed location is missing or the skill was moved manually.
|
||||
4. When calling the terminal tool, do not pass `workdir: "~"`. Use an absolute directory such as the user's home directory, or omit `workdir` entirely.
|
||||
|
||||
With `--migrate-secrets`, it will also import a small allowlisted set of Hermes-compatible secrets, currently:
|
||||
|
||||
- `TELEGRAM_BOT_TOKEN`
|
||||
|
||||
## Default workflow
|
||||
|
||||
1. Inspect first with a dry run.
|
||||
2. Present a simple summary of what can be migrated, what cannot be migrated, and what would be archived.
|
||||
3. If the `clarify` tool is available, use it for user decisions instead of asking for a free-form prose reply.
|
||||
4. If the dry run finds imported skill directory conflicts, ask how those should be handled before executing.
|
||||
5. Ask the user to choose between the two supported migration modes before executing.
|
||||
6. Ask for a target workspace path only if the user wants the workspace instructions file brought over.
|
||||
7. Execute the migration with the matching preset and flags.
|
||||
8. Summarize the results, especially:
|
||||
- what was migrated
|
||||
- what was archived for manual review
|
||||
- what was skipped and why
|
||||
|
||||
## User interaction protocol
|
||||
|
||||
Hermes CLI supports the `clarify` tool for interactive prompts, but it is limited to:
|
||||
|
||||
- one choice at a time
|
||||
- up to 4 predefined choices
|
||||
- an automatic `Other` free-text option
|
||||
|
||||
It does **not** support true multi-select checkboxes in a single prompt.
|
||||
|
||||
For every `clarify` call:
|
||||
|
||||
- always include a non-empty `question`
|
||||
- include `choices` only for real selectable prompts
|
||||
- keep `choices` to 2-4 plain string options
|
||||
- never emit placeholder or truncated options such as `...`
|
||||
- never pad or stylize choices with extra whitespace
|
||||
- never include fake form fields in the question such as `enter directory here`, blank lines to fill in, or underscores like `_____`
|
||||
- for open-ended path questions, ask only the plain sentence; the user types in the normal CLI prompt below the panel
|
||||
|
||||
If a `clarify` call returns an error, inspect the error text, correct the payload, and retry once with a valid `question` and clean choices.
|
||||
|
||||
When `clarify` is available and the dry run reveals any required user decision, your **next action must be a `clarify` tool call**.
|
||||
Do not end the turn with a normal assistant message such as:
|
||||
|
||||
- "Let me present the choices"
|
||||
- "What would you like to do?"
|
||||
- "Here are the options"
|
||||
|
||||
If a user decision is required, collect it via `clarify` before producing more prose.
|
||||
If multiple unresolved decisions remain, do not insert an explanatory assistant message between them. After one `clarify` response is received, your next action should usually be the next required `clarify` call.
|
||||
|
||||
Treat `workspace-agents` as an unresolved decision whenever the dry run reports:
|
||||
|
||||
- `kind="workspace-agents"`
|
||||
- `status="skipped"`
|
||||
- reason containing `No workspace target was provided`
|
||||
|
||||
In that case, you must ask about workspace instructions before execution. Do not silently treat that as a decision to skip.
|
||||
|
||||
Because of that limitation, use this simplified decision flow:
|
||||
|
||||
1. For `SOUL.md` conflicts, use `clarify` with choices such as:
|
||||
- `keep existing`
|
||||
- `overwrite with backup`
|
||||
- `review first`
|
||||
2. If the dry run shows one or more `kind="skill"` items with `status="conflict"`, use `clarify` with choices such as:
|
||||
- `keep existing skills`
|
||||
- `overwrite conflicting skills with backup`
|
||||
- `import conflicting skills under renamed folders`
|
||||
3. For workspace instructions, use `clarify` with choices such as:
|
||||
- `skip workspace instructions`
|
||||
- `copy to a workspace path`
|
||||
- `decide later`
|
||||
4. If the user chooses to copy workspace instructions, ask a follow-up open-ended `clarify` question requesting an **absolute path**.
|
||||
5. If the user chooses `skip workspace instructions` or `decide later`, proceed without `--workspace-target`.
|
||||
5. For migration mode, use `clarify` with these 3 choices:
|
||||
- `user-data only`
|
||||
- `full compatible migration`
|
||||
- `cancel`
|
||||
6. `user-data only` means: migrate user data and compatible config, but do **not** import allowlisted secrets.
|
||||
7. `full compatible migration` means: migrate the same compatible user data plus the allowlisted secrets when present.
|
||||
8. If `clarify` is not available, ask the same question in normal text, but still constrain the answer to `user-data only`, `full compatible migration`, or `cancel`.
|
||||
|
||||
Execution gate:
|
||||
|
||||
- Do not execute while a `workspace-agents` skip caused by `No workspace target was provided` remains unresolved.
|
||||
- The only valid ways to resolve it are:
|
||||
- user explicitly chooses `skip workspace instructions`
|
||||
- user explicitly chooses `decide later`
|
||||
- user provides a workspace path after choosing `copy to a workspace path`
|
||||
- Absence of a workspace target in the dry run is not itself permission to execute.
|
||||
- Do not execute while any required `clarify` decision remains unresolved.
|
||||
|
||||
Use these exact `clarify` payload shapes as the default pattern:
|
||||
|
||||
- `{"question":"Your existing SOUL.md conflicts with the imported one. What should I do?","choices":["keep existing","overwrite with backup","review first"]}`
|
||||
- `{"question":"One or more imported OpenClaw skills already exist in Hermes. How should I handle those skill conflicts?","choices":["keep existing skills","overwrite conflicting skills with backup","import conflicting skills under renamed folders"]}`
|
||||
- `{"question":"Choose migration mode: migrate only user data, or run the full compatible migration including allowlisted secrets?","choices":["user-data only","full compatible migration","cancel"]}`
|
||||
- `{"question":"Do you want to copy the OpenClaw workspace instructions file into a Hermes workspace?","choices":["skip workspace instructions","copy to a workspace path","decide later"]}`
|
||||
- `{"question":"Please provide an absolute path where the workspace instructions should be copied."}`
|
||||
|
||||
## Decision-to-command mapping
|
||||
|
||||
Map user decisions to command flags exactly:
|
||||
|
||||
- If the user chooses `keep existing` for `SOUL.md`, do **not** add `--overwrite`.
|
||||
- If the user chooses `overwrite with backup`, add `--overwrite`.
|
||||
- If the user chooses `review first`, stop before execution and review the relevant files.
|
||||
- If the user chooses `keep existing skills`, add `--skill-conflict skip`.
|
||||
- If the user chooses `overwrite conflicting skills with backup`, add `--skill-conflict overwrite`.
|
||||
- If the user chooses `import conflicting skills under renamed folders`, add `--skill-conflict rename`.
|
||||
- If the user chooses `user-data only`, execute with `--preset user-data` and do **not** add `--migrate-secrets`.
|
||||
- If the user chooses `full compatible migration`, execute with `--preset full --migrate-secrets`.
|
||||
- Only add `--workspace-target` if the user explicitly provided an absolute workspace path.
|
||||
- If the user chooses `skip workspace instructions` or `decide later`, do not add `--workspace-target`.
|
||||
|
||||
Before executing, restate the exact command plan in plain language and make sure it matches the user's choices.
|
||||
|
||||
## Post-run reporting rules
|
||||
|
||||
After execution, treat the script's JSON output as the source of truth.
|
||||
|
||||
1. Base all counts on `report.summary`.
|
||||
2. Only list an item under "Successfully Migrated" if its `status` is exactly `migrated`.
|
||||
3. Do not claim a conflict was resolved unless the report shows that item as `migrated`.
|
||||
4. Do not say `SOUL.md` was overwritten unless the report item for `kind="soul"` has `status="migrated"`.
|
||||
5. If `report.summary.conflict > 0`, include a conflict section instead of silently implying success.
|
||||
6. If counts and listed items disagree, fix the list to match the report before responding.
|
||||
7. Include the `output_dir` path from the report when available so the user can inspect `report.json`, `summary.md`, backups, and archived files.
|
||||
8. For memory or user-profile overflow, do not say the entries were archived unless the report explicitly shows an archive path. If `details.overflow_file` exists, say the full overflow list was exported there.
|
||||
9. If a skill was imported under a renamed folder, report the final destination and mention `details.renamed_from`.
|
||||
10. If `report.skill_conflict_mode` is present, use it as the source of truth for the selected imported-skill conflict policy.
|
||||
11. If an item has `status="skipped"`, do not describe it as overwritten, backed up, migrated, or resolved.
|
||||
12. If `kind="soul"` has `status="skipped"` with reason `Target already matches source`, say it was left unchanged and do not mention a backup.
|
||||
13. If a renamed imported skill has an empty `details.backup`, do not imply the existing Hermes skill was renamed or backed up. Say only that the imported copy was placed in the new destination and reference `details.renamed_from` as the pre-existing folder that remained in place.
|
||||
|
||||
## Migration presets
|
||||
|
||||
Prefer these two presets in normal use:
|
||||
|
||||
- `user-data`
|
||||
- `full`
|
||||
|
||||
`user-data` includes:
|
||||
|
||||
- `soul`
|
||||
- `workspace-agents`
|
||||
- `memory`
|
||||
- `user-profile`
|
||||
- `messaging-settings`
|
||||
- `command-allowlist`
|
||||
- `skills`
|
||||
- `tts-assets`
|
||||
- `archive`
|
||||
|
||||
`full` includes everything in `user-data` plus:
|
||||
|
||||
- `secret-settings`
|
||||
|
||||
The helper script still supports category-level `--include` / `--exclude`, but treat that as an advanced fallback rather than the default UX.
|
||||
|
||||
## Commands
|
||||
|
||||
Dry run with full discovery:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py
|
||||
```
|
||||
|
||||
When using the terminal tool, prefer an absolute invocation pattern such as:
|
||||
|
||||
```json
|
||||
{"command":"python3 /home/USER/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py","workdir":"/home/USER"}
|
||||
```
|
||||
|
||||
Dry run with the user-data preset:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py --preset user-data
|
||||
```
|
||||
|
||||
Execute a user-data migration:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py --execute --preset user-data --skill-conflict skip
|
||||
```
|
||||
|
||||
Execute a full compatible migration:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py --execute --preset full --migrate-secrets --skill-conflict skip
|
||||
```
|
||||
|
||||
Execute with workspace instructions included:
|
||||
|
||||
```bash
|
||||
python3 ~/.hermes/skills/migration/openclaw-migration/scripts/openclaw_to_hermes.py --execute --preset user-data --skill-conflict rename --workspace-target "/absolute/workspace/path"
|
||||
```
|
||||
|
||||
Do not use `$PWD` or the home directory as the workspace target by default. Ask for an explicit workspace path first.
|
||||
|
||||
## Important rules
|
||||
|
||||
1. Run a dry run before writing unless the user explicitly says to proceed immediately.
|
||||
2. Do not migrate secrets by default. Tokens, auth blobs, device credentials, and raw gateway config should stay out of Hermes unless the user explicitly asks for secret migration.
|
||||
3. Do not silently overwrite non-empty Hermes targets unless the user explicitly wants that. The helper script will preserve backups when overwriting is enabled.
|
||||
4. Always give the user the skipped-items report. That report is part of the migration, not an optional extra.
|
||||
5. Prefer the primary OpenClaw workspace (`~/.openclaw/workspace/`) over `workspace.default/`. Only use the default workspace as fallback when the primary files are missing.
|
||||
6. Even in secret-migration mode, only migrate secrets with a clean Hermes destination. Unsupported auth blobs must still be reported as skipped.
|
||||
7. If the dry run shows a large asset copy, a conflicting `SOUL.md`, or overflowed memory entries, call those out separately before execution.
|
||||
8. Default to `user-data only` if the user is unsure.
|
||||
9. Only include `workspace-agents` when the user has explicitly provided a destination workspace path.
|
||||
10. Treat category-level `--include` / `--exclude` as an advanced escape hatch, not the normal flow.
|
||||
11. Do not end the dry-run summary with a vague “What would you like to do?” if `clarify` is available. Use structured follow-up prompts instead.
|
||||
12. Do not use an open-ended `clarify` prompt when a real choice prompt would work. Prefer selectable choices first, then free text only for absolute paths or file review requests.
|
||||
13. After a dry run, never stop after summarizing if there is still an unresolved decision. Use `clarify` immediately for the highest-priority blocking decision.
|
||||
14. Priority order for follow-up questions:
|
||||
- `SOUL.md` conflict
|
||||
- imported skill conflicts
|
||||
- migration mode
|
||||
- workspace instructions destination
|
||||
15. Do not promise to present choices later in the same message. Present them by actually calling `clarify`.
|
||||
16. After the migration-mode answer, explicitly check whether `workspace-agents` is still unresolved. If it is, your next action must be the workspace-instructions `clarify` call.
|
||||
17. After any `clarify` answer, if another required decision remains, do not narrate what was just decided. Ask the next required question immediately.
|
||||
|
||||
## Expected result
|
||||
|
||||
After a successful run, the user should have:
|
||||
|
||||
- Hermes persona state imported
|
||||
- Hermes memory files populated with converted OpenClaw knowledge
|
||||
- OpenClaw skills available under `~/.hermes/skills/openclaw-imports/`
|
||||
- a migration report showing any conflicts, omissions, or unsupported data
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,336 @@
|
||||
---
|
||||
name: huggingface-accelerate
|
||||
description: Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.
|
||||
version: 1.0.0
|
||||
author: Orchestra Research
|
||||
license: MIT
|
||||
dependencies: [accelerate, torch, transformers]
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Distributed Training, HuggingFace, Accelerate, DeepSpeed, FSDP, Mixed Precision, PyTorch, DDP, Unified API, Simple]
|
||||
|
||||
---
|
||||
|
||||
# HuggingFace Accelerate - Unified Distributed Training
|
||||
|
||||
## Quick start
|
||||
|
||||
Accelerate simplifies distributed training to 4 lines of code.
|
||||
|
||||
**Installation**:
|
||||
```bash
|
||||
pip install accelerate
|
||||
```
|
||||
|
||||
**Convert PyTorch script** (4 lines):
|
||||
```python
|
||||
import torch
|
||||
+ from accelerate import Accelerator
|
||||
|
||||
+ accelerator = Accelerator()
|
||||
|
||||
model = torch.nn.Transformer()
|
||||
optimizer = torch.optim.Adam(model.parameters())
|
||||
dataloader = torch.utils.data.DataLoader(dataset)
|
||||
|
||||
+ model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)
|
||||
|
||||
for batch in dataloader:
|
||||
optimizer.zero_grad()
|
||||
loss = model(batch)
|
||||
- loss.backward()
|
||||
+ accelerator.backward(loss)
|
||||
optimizer.step()
|
||||
```
|
||||
|
||||
**Run** (single command):
|
||||
```bash
|
||||
accelerate launch train.py
|
||||
```
|
||||
|
||||
## Common workflows
|
||||
|
||||
### Workflow 1: From single GPU to multi-GPU
|
||||
|
||||
**Original script**:
|
||||
```python
|
||||
# train.py
|
||||
import torch
|
||||
|
||||
model = torch.nn.Linear(10, 2).to('cuda')
|
||||
optimizer = torch.optim.Adam(model.parameters())
|
||||
dataloader = torch.utils.data.DataLoader(dataset, batch_size=32)
|
||||
|
||||
for epoch in range(10):
|
||||
for batch in dataloader:
|
||||
batch = batch.to('cuda')
|
||||
optimizer.zero_grad()
|
||||
loss = model(batch).mean()
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
```
|
||||
|
||||
**With Accelerate** (4 lines added):
|
||||
```python
|
||||
# train.py
|
||||
import torch
|
||||
from accelerate import Accelerator # +1
|
||||
|
||||
accelerator = Accelerator() # +2
|
||||
|
||||
model = torch.nn.Linear(10, 2)
|
||||
optimizer = torch.optim.Adam(model.parameters())
|
||||
dataloader = torch.utils.data.DataLoader(dataset, batch_size=32)
|
||||
|
||||
model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader) # +3
|
||||
|
||||
for epoch in range(10):
|
||||
for batch in dataloader:
|
||||
# No .to('cuda') needed - automatic!
|
||||
optimizer.zero_grad()
|
||||
loss = model(batch).mean()
|
||||
accelerator.backward(loss) # +4
|
||||
optimizer.step()
|
||||
```
|
||||
|
||||
**Configure** (interactive):
|
||||
```bash
|
||||
accelerate config
|
||||
```
|
||||
|
||||
**Questions**:
|
||||
- Which machine? (single/multi GPU/TPU/CPU)
|
||||
- How many machines? (1)
|
||||
- Mixed precision? (no/fp16/bf16/fp8)
|
||||
- DeepSpeed? (no/yes)
|
||||
|
||||
**Launch** (works on any setup):
|
||||
```bash
|
||||
# Single GPU
|
||||
accelerate launch train.py
|
||||
|
||||
# Multi-GPU (8 GPUs)
|
||||
accelerate launch --multi_gpu --num_processes 8 train.py
|
||||
|
||||
# Multi-node
|
||||
accelerate launch --multi_gpu --num_processes 16 \
|
||||
--num_machines 2 --machine_rank 0 \
|
||||
--main_process_ip $MASTER_ADDR \
|
||||
train.py
|
||||
```
|
||||
|
||||
### Workflow 2: Mixed precision training
|
||||
|
||||
**Enable FP16/BF16**:
|
||||
```python
|
||||
from accelerate import Accelerator
|
||||
|
||||
# FP16 (with gradient scaling)
|
||||
accelerator = Accelerator(mixed_precision='fp16')
|
||||
|
||||
# BF16 (no scaling, more stable)
|
||||
accelerator = Accelerator(mixed_precision='bf16')
|
||||
|
||||
# FP8 (H100+)
|
||||
accelerator = Accelerator(mixed_precision='fp8')
|
||||
|
||||
model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)
|
||||
|
||||
# Everything else is automatic!
|
||||
for batch in dataloader:
|
||||
with accelerator.autocast(): # Optional, done automatically
|
||||
loss = model(batch)
|
||||
accelerator.backward(loss)
|
||||
```
|
||||
|
||||
### Workflow 3: DeepSpeed ZeRO integration
|
||||
|
||||
**Enable DeepSpeed ZeRO-2**:
|
||||
```python
|
||||
from accelerate import Accelerator
|
||||
|
||||
accelerator = Accelerator(
|
||||
mixed_precision='bf16',
|
||||
deepspeed_plugin={
|
||||
"zero_stage": 2, # ZeRO-2
|
||||
"offload_optimizer": False,
|
||||
"gradient_accumulation_steps": 4
|
||||
}
|
||||
)
|
||||
|
||||
# Same code as before!
|
||||
model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)
|
||||
```
|
||||
|
||||
**Or via config**:
|
||||
```bash
|
||||
accelerate config
|
||||
# Select: DeepSpeed → ZeRO-2
|
||||
```
|
||||
|
||||
**deepspeed_config.json**:
|
||||
```json
|
||||
{
|
||||
"fp16": {"enabled": false},
|
||||
"bf16": {"enabled": true},
|
||||
"zero_optimization": {
|
||||
"stage": 2,
|
||||
"offload_optimizer": {"device": "cpu"},
|
||||
"allgather_bucket_size": 5e8,
|
||||
"reduce_bucket_size": 5e8
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Launch**:
|
||||
```bash
|
||||
accelerate launch --config_file deepspeed_config.json train.py
|
||||
```
|
||||
|
||||
### Workflow 4: FSDP (Fully Sharded Data Parallel)
|
||||
|
||||
**Enable FSDP**:
|
||||
```python
|
||||
from accelerate import Accelerator, FullyShardedDataParallelPlugin
|
||||
|
||||
fsdp_plugin = FullyShardedDataParallelPlugin(
|
||||
sharding_strategy="FULL_SHARD", # ZeRO-3 equivalent
|
||||
auto_wrap_policy="TRANSFORMER_AUTO_WRAP",
|
||||
cpu_offload=False
|
||||
)
|
||||
|
||||
accelerator = Accelerator(
|
||||
mixed_precision='bf16',
|
||||
fsdp_plugin=fsdp_plugin
|
||||
)
|
||||
|
||||
model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)
|
||||
```
|
||||
|
||||
**Or via config**:
|
||||
```bash
|
||||
accelerate config
|
||||
# Select: FSDP → Full Shard → No CPU Offload
|
||||
```
|
||||
|
||||
### Workflow 5: Gradient accumulation
|
||||
|
||||
**Accumulate gradients**:
|
||||
```python
|
||||
from accelerate import Accelerator
|
||||
|
||||
accelerator = Accelerator(gradient_accumulation_steps=4)
|
||||
|
||||
model, optimizer, dataloader = accelerator.prepare(model, optimizer, dataloader)
|
||||
|
||||
for batch in dataloader:
|
||||
with accelerator.accumulate(model): # Handles accumulation
|
||||
optimizer.zero_grad()
|
||||
loss = model(batch)
|
||||
accelerator.backward(loss)
|
||||
optimizer.step()
|
||||
```
|
||||
|
||||
**Effective batch size**: `batch_size * num_gpus * gradient_accumulation_steps`
|
||||
|
||||
## When to use vs alternatives
|
||||
|
||||
**Use Accelerate when**:
|
||||
- Want simplest distributed training
|
||||
- Need single script for any hardware
|
||||
- Use HuggingFace ecosystem
|
||||
- Want flexibility (DDP/DeepSpeed/FSDP/Megatron)
|
||||
- Need quick prototyping
|
||||
|
||||
**Key advantages**:
|
||||
- **4 lines**: Minimal code changes
|
||||
- **Unified API**: Same code for DDP, DeepSpeed, FSDP, Megatron
|
||||
- **Automatic**: Device placement, mixed precision, sharding
|
||||
- **Interactive config**: No manual launcher setup
|
||||
- **Single launch**: Works everywhere
|
||||
|
||||
**Use alternatives instead**:
|
||||
- **PyTorch Lightning**: Need callbacks, high-level abstractions
|
||||
- **Ray Train**: Multi-node orchestration, hyperparameter tuning
|
||||
- **DeepSpeed**: Direct API control, advanced features
|
||||
- **Raw DDP**: Maximum control, minimal abstraction
|
||||
|
||||
## Common issues
|
||||
|
||||
**Issue: Wrong device placement**
|
||||
|
||||
Don't manually move to device:
|
||||
```python
|
||||
# WRONG
|
||||
batch = batch.to('cuda')
|
||||
|
||||
# CORRECT
|
||||
# Accelerate handles it automatically after prepare()
|
||||
```
|
||||
|
||||
**Issue: Gradient accumulation not working**
|
||||
|
||||
Use context manager:
|
||||
```python
|
||||
# CORRECT
|
||||
with accelerator.accumulate(model):
|
||||
optimizer.zero_grad()
|
||||
accelerator.backward(loss)
|
||||
optimizer.step()
|
||||
```
|
||||
|
||||
**Issue: Checkpointing in distributed**
|
||||
|
||||
Use accelerator methods:
|
||||
```python
|
||||
# Save only on main process
|
||||
if accelerator.is_main_process:
|
||||
accelerator.save_state('checkpoint/')
|
||||
|
||||
# Load on all processes
|
||||
accelerator.load_state('checkpoint/')
|
||||
```
|
||||
|
||||
**Issue: Different results with FSDP**
|
||||
|
||||
Ensure same random seed:
|
||||
```python
|
||||
from accelerate.utils import set_seed
|
||||
set_seed(42)
|
||||
```
|
||||
|
||||
## Advanced topics
|
||||
|
||||
**Megatron integration**: See [references/megatron-integration.md](references/megatron-integration.md) for tensor parallelism, pipeline parallelism, and sequence parallelism setup.
|
||||
|
||||
**Custom plugins**: See [references/custom-plugins.md](references/custom-plugins.md) for creating custom distributed plugins and advanced configuration.
|
||||
|
||||
**Performance tuning**: See [references/performance.md](references/performance.md) for profiling, memory optimization, and best practices.
|
||||
|
||||
## Hardware requirements
|
||||
|
||||
- **CPU**: Works (slow)
|
||||
- **Single GPU**: Works
|
||||
- **Multi-GPU**: DDP (default), DeepSpeed, or FSDP
|
||||
- **Multi-node**: DDP, DeepSpeed, FSDP, Megatron
|
||||
- **TPU**: Supported
|
||||
- **Apple MPS**: Supported
|
||||
|
||||
**Launcher requirements**:
|
||||
- **DDP**: `torch.distributed.run` (built-in)
|
||||
- **DeepSpeed**: `deepspeed` (pip install deepspeed)
|
||||
- **FSDP**: PyTorch 1.12+ (built-in)
|
||||
- **Megatron**: Custom setup
|
||||
|
||||
## Resources
|
||||
|
||||
- Docs: https://huggingface.co/docs/accelerate
|
||||
- GitHub: https://github.com/huggingface/accelerate
|
||||
- Version: 1.11.0+
|
||||
- Tutorial: "Accelerate your scripts"
|
||||
- Examples: https://github.com/huggingface/accelerate/tree/main/examples
|
||||
- Used by: HuggingFace Transformers, TRL, PEFT, all HF libraries
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,453 @@
|
||||
# Custom Plugins for Accelerate
|
||||
|
||||
## Overview
|
||||
|
||||
Accelerate allows creating **custom plugins** to extend distributed training strategies beyond built-in options (DDP, FSDP, DeepSpeed).
|
||||
|
||||
## Plugin Architecture
|
||||
|
||||
### Base Plugin Structure
|
||||
|
||||
```python
|
||||
from accelerate.utils import DistributedDataParallelKwargs
|
||||
from dataclasses import dataclass
|
||||
|
||||
@dataclass
|
||||
class CustomPlugin:
|
||||
"""Custom training plugin."""
|
||||
|
||||
# Plugin configuration
|
||||
param1: int = 1
|
||||
param2: str = "default"
|
||||
|
||||
def __post_init__(self):
|
||||
# Validation logic
|
||||
if self.param1 < 1:
|
||||
raise ValueError("param1 must be >= 1")
|
||||
```
|
||||
|
||||
### Using Custom Plugin
|
||||
|
||||
```python
|
||||
from accelerate import Accelerator
|
||||
|
||||
# Create plugin
|
||||
custom_plugin = CustomPlugin(param1=4, param2="value")
|
||||
|
||||
# Pass to Accelerator
|
||||
accelerator = Accelerator(
|
||||
custom_plugin=custom_plugin # Not a real parameter, example only
|
||||
)
|
||||
```
|
||||
|
||||
## Built-In Plugin Examples
|
||||
|
||||
### 1. GradScalerKwargs (FP16 Configuration)
|
||||
|
||||
```python
|
||||
from accelerate.utils import GradScalerKwargs
|
||||
|
||||
# Configure gradient scaler for FP16
|
||||
scaler_kwargs = GradScalerKwargs(
|
||||
init_scale=2.**16, # Initial loss scale
|
||||
growth_factor=2.0, # Scale growth rate
|
||||
backoff_factor=0.5, # Scale backoff rate
|
||||
growth_interval=2000, # Steps between scale increases
|
||||
enabled=True # Enable scaler
|
||||
)
|
||||
|
||||
accelerator = Accelerator(
|
||||
mixed_precision='fp16',
|
||||
kwargs_handlers=[scaler_kwargs] # Pass as kwargs handler
|
||||
)
|
||||
```
|
||||
|
||||
**Use case**: Fine-tune FP16 gradient scaling behavior
|
||||
|
||||
### 2. DistributedDataParallelKwargs
|
||||
|
||||
```python
|
||||
from accelerate.utils import DistributedDataParallelKwargs
|
||||
|
||||
# Configure DDP behavior
|
||||
ddp_kwargs = DistributedDataParallelKwargs(
|
||||
bucket_cap_mb=25, # Gradient bucketing size
|
||||
find_unused_parameters=False, # Find unused params (slower)
|
||||
check_reduction=False, # Check gradient reduction
|
||||
gradient_as_bucket_view=True, # Memory optimization
|
||||
static_graph=False # Static computation graph
|
||||
)
|
||||
|
||||
accelerator = Accelerator(
|
||||
kwargs_handlers=[ddp_kwargs]
|
||||
)
|
||||
```
|
||||
|
||||
**Use case**: Optimize DDP performance for specific models
|
||||
|
||||
### 3. FP8RecipeKwargs (H100 FP8)
|
||||
|
||||
```python
|
||||
from accelerate.utils import FP8RecipeKwargs
|
||||
|
||||
# Configure FP8 training (H100)
|
||||
fp8_recipe = FP8RecipeKwargs(
|
||||
backend="te", # TransformerEngine backend
|
||||
margin=0, # Scaling margin
|
||||
interval=1, # Scaling interval
|
||||
fp8_format="HYBRID", # E4M3 + E5M2 hybrid
|
||||
amax_history_len=1024, # AMAX history length
|
||||
amax_compute_algo="max" # AMAX computation algorithm
|
||||
)
|
||||
|
||||
accelerator = Accelerator(
|
||||
mixed_precision='fp8',
|
||||
kwargs_handlers=[fp8_recipe]
|
||||
)
|
||||
```
|
||||
|
||||
**Use case**: Ultra-fast training on H100 GPUs
|
||||
|
||||
## Custom DeepSpeed Configuration
|
||||
|
||||
### ZeRO-3 with CPU Offload
|
||||
|
||||
```python
|
||||
from accelerate import Accelerator
|
||||
from accelerate.utils import DeepSpeedPlugin
|
||||
|
||||
# Custom DeepSpeed config
|
||||
ds_plugin = DeepSpeedPlugin(
|
||||
zero_stage=3, # ZeRO-3
|
||||
offload_optimizer_device="cpu", # CPU offload optimizer
|
||||
offload_param_device="cpu", # CPU offload parameters
|
||||
zero3_init_flag=True, # ZeRO-3 initialization
|
||||
zero3_save_16bit_model=True, # Save FP16 weights
|
||||
)
|
||||
|
||||
accelerator = Accelerator(
|
||||
deepspeed_plugin=ds_plugin,
|
||||
mixed_precision='bf16'
|
||||
)
|
||||
```
|
||||
|
||||
### ZeRO-2 with NVMe Offload
|
||||
|
||||
```python
|
||||
ds_plugin = DeepSpeedPlugin(
|
||||
zero_stage=2,
|
||||
offload_optimizer_device="nvme", # NVMe offload
|
||||
offload_param_device="nvme",
|
||||
nvme_path="/local_nvme", # NVMe mount path
|
||||
)
|
||||
```
|
||||
|
||||
### Custom JSON Config
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
# Load custom DeepSpeed config
|
||||
with open('deepspeed_config.json', 'r') as f:
|
||||
ds_config = json.load(f)
|
||||
|
||||
ds_plugin = DeepSpeedPlugin(hf_ds_config=ds_config)
|
||||
|
||||
accelerator = Accelerator(deepspeed_plugin=ds_plugin)
|
||||
```
|
||||
|
||||
**Example config** (`deepspeed_config.json`):
|
||||
```json
|
||||
{
|
||||
"train_batch_size": "auto",
|
||||
"train_micro_batch_size_per_gpu": "auto",
|
||||
"gradient_accumulation_steps": "auto",
|
||||
"gradient_clipping": 1.0,
|
||||
"zero_optimization": {
|
||||
"stage": 3,
|
||||
"offload_optimizer": {
|
||||
"device": "cpu",
|
||||
"pin_memory": true
|
||||
},
|
||||
"offload_param": {
|
||||
"device": "cpu",
|
||||
"pin_memory": true
|
||||
},
|
||||
"overlap_comm": true,
|
||||
"contiguous_gradients": true,
|
||||
"sub_group_size": 1e9,
|
||||
"reduce_bucket_size": 5e8,
|
||||
"stage3_prefetch_bucket_size": 5e8,
|
||||
"stage3_param_persistence_threshold": 1e6,
|
||||
"stage3_max_live_parameters": 1e9,
|
||||
"stage3_max_reuse_distance": 1e9,
|
||||
"stage3_gather_16bit_weights_on_model_save": true
|
||||
},
|
||||
"bf16": {
|
||||
"enabled": true
|
||||
},
|
||||
"steps_per_print": 100,
|
||||
"wall_clock_breakdown": false
|
||||
}
|
||||
```
|
||||
|
||||
## Custom FSDP Configuration
|
||||
|
||||
### FSDP with Custom Auto-Wrap Policy
|
||||
|
||||
```python
|
||||
from accelerate.utils import FullyShardedDataParallelPlugin
|
||||
from torch.distributed.fsdp import BackwardPrefetch, ShardingStrategy
|
||||
from torch.distributed.fsdp.wrap import size_based_auto_wrap_policy
|
||||
import functools
|
||||
|
||||
# Custom wrap policy (size-based)
|
||||
wrap_policy = functools.partial(
|
||||
size_based_auto_wrap_policy,
|
||||
min_num_params=1e6 # Wrap layers with 1M+ params
|
||||
)
|
||||
|
||||
fsdp_plugin = FullyShardedDataParallelPlugin(
|
||||
sharding_strategy=ShardingStrategy.FULL_SHARD, # ZeRO-3 equivalent
|
||||
backward_prefetch=BackwardPrefetch.BACKWARD_PRE, # Prefetch strategy
|
||||
mixed_precision_policy=None, # Use Accelerator's mixed precision
|
||||
auto_wrap_policy=wrap_policy, # Custom wrapping
|
||||
cpu_offload=False,
|
||||
ignored_modules=None, # Modules to not wrap
|
||||
state_dict_type="FULL_STATE_DICT", # Save format
|
||||
optim_state_dict_config=None,
|
||||
limit_all_gathers=False,
|
||||
use_orig_params=True, # Use original param shapes
|
||||
)
|
||||
|
||||
accelerator = Accelerator(
|
||||
fsdp_plugin=fsdp_plugin,
|
||||
mixed_precision='bf16'
|
||||
)
|
||||
```
|
||||
|
||||
### FSDP with Transformer Auto-Wrap
|
||||
|
||||
```python
|
||||
from torch.distributed.fsdp.wrap import transformer_auto_wrap_policy
|
||||
from transformers.models.gpt2.modeling_gpt2 import GPT2Block
|
||||
|
||||
# Wrap at transformer block level
|
||||
wrap_policy = functools.partial(
|
||||
transformer_auto_wrap_policy,
|
||||
transformer_layer_cls={GPT2Block} # Wrap GPT2Block layers
|
||||
)
|
||||
|
||||
fsdp_plugin = FullyShardedDataParallelPlugin(
|
||||
auto_wrap_policy=wrap_policy
|
||||
)
|
||||
```
|
||||
|
||||
## Creating Custom Training Strategy
|
||||
|
||||
### Example: Custom Gradient Accumulation
|
||||
|
||||
```python
|
||||
from accelerate import Accelerator
|
||||
|
||||
class CustomGradientAccumulation:
|
||||
def __init__(self, steps=4, adaptive=False):
|
||||
self.steps = steps
|
||||
self.adaptive = adaptive
|
||||
self.current_step = 0
|
||||
|
||||
def should_sync(self, loss):
|
||||
"""Decide whether to sync gradients."""
|
||||
self.current_step += 1
|
||||
|
||||
# Adaptive: sync on high loss
|
||||
if self.adaptive and loss > threshold:
|
||||
self.current_step = 0
|
||||
return True
|
||||
|
||||
# Regular: sync every N steps
|
||||
if self.current_step >= self.steps:
|
||||
self.current_step = 0
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
# Usage
|
||||
custom_accum = CustomGradientAccumulation(steps=8, adaptive=True)
|
||||
accelerator = Accelerator()
|
||||
|
||||
for batch in dataloader:
|
||||
outputs = model(**batch)
|
||||
loss = outputs.loss
|
||||
|
||||
# Scale loss
|
||||
loss = loss / custom_accum.steps
|
||||
accelerator.backward(loss)
|
||||
|
||||
# Conditional sync
|
||||
if custom_accum.should_sync(loss.item()):
|
||||
optimizer.step()
|
||||
optimizer.zero_grad()
|
||||
```
|
||||
|
||||
### Example: Custom Mixed Precision
|
||||
|
||||
```python
|
||||
import torch
|
||||
|
||||
class CustomMixedPrecision:
|
||||
"""Custom mixed precision with dynamic loss scaling."""
|
||||
|
||||
def __init__(self, init_scale=2**16, scale_window=2000):
|
||||
self.scaler = torch.cuda.amp.GradScaler(
|
||||
init_scale=init_scale,
|
||||
growth_interval=scale_window
|
||||
)
|
||||
self.scale_history = []
|
||||
|
||||
def scale_loss(self, loss):
|
||||
"""Scale loss for backward."""
|
||||
return self.scaler.scale(loss)
|
||||
|
||||
def unscale_and_clip(self, optimizer, max_norm=1.0):
|
||||
"""Unscale gradients and clip."""
|
||||
self.scaler.unscale_(optimizer)
|
||||
torch.nn.utils.clip_grad_norm_(
|
||||
optimizer.param_groups[0]['params'],
|
||||
max_norm
|
||||
)
|
||||
|
||||
def step(self, optimizer):
|
||||
"""Optimizer step with scaler update."""
|
||||
scale_before = self.scaler.get_scale()
|
||||
self.scaler.step(optimizer)
|
||||
self.scaler.update()
|
||||
scale_after = self.scaler.get_scale()
|
||||
|
||||
# Track scale changes
|
||||
if scale_before != scale_after:
|
||||
self.scale_history.append(scale_after)
|
||||
|
||||
# Usage
|
||||
custom_mp = CustomMixedPrecision()
|
||||
|
||||
for batch in dataloader:
|
||||
with torch.cuda.amp.autocast(dtype=torch.float16):
|
||||
loss = model(**batch).loss
|
||||
|
||||
scaled_loss = custom_mp.scale_loss(loss)
|
||||
scaled_loss.backward()
|
||||
|
||||
custom_mp.unscale_and_clip(optimizer, max_norm=1.0)
|
||||
custom_mp.step(optimizer)
|
||||
optimizer.zero_grad()
|
||||
```
|
||||
|
||||
## Advanced: Custom Distributed Backend
|
||||
|
||||
### Custom AllReduce Strategy
|
||||
|
||||
```python
|
||||
import torch.distributed as dist
|
||||
|
||||
class CustomAllReduce:
|
||||
"""Custom all-reduce with compression."""
|
||||
|
||||
def __init__(self, compression_ratio=0.1):
|
||||
self.compression_ratio = compression_ratio
|
||||
|
||||
def compress_gradients(self, tensor):
|
||||
"""Top-k gradient compression."""
|
||||
k = int(tensor.numel() * self.compression_ratio)
|
||||
values, indices = torch.topk(tensor.abs().view(-1), k)
|
||||
return values, indices
|
||||
|
||||
def all_reduce_compressed(self, tensor):
|
||||
"""All-reduce with gradient compression."""
|
||||
# Compress
|
||||
values, indices = self.compress_gradients(tensor)
|
||||
|
||||
# All-reduce compressed gradients
|
||||
dist.all_reduce(values, op=dist.ReduceOp.SUM)
|
||||
|
||||
# Decompress
|
||||
tensor_compressed = torch.zeros_like(tensor).view(-1)
|
||||
tensor_compressed[indices] = values / dist.get_world_size()
|
||||
|
||||
return tensor_compressed.view_as(tensor)
|
||||
|
||||
# Usage in training loop
|
||||
custom_ar = CustomAllReduce(compression_ratio=0.1)
|
||||
|
||||
for batch in dataloader:
|
||||
loss = model(**batch).loss
|
||||
loss.backward()
|
||||
|
||||
# Custom all-reduce
|
||||
for param in model.parameters():
|
||||
if param.grad is not None:
|
||||
param.grad.data = custom_ar.all_reduce_compressed(param.grad.data)
|
||||
|
||||
optimizer.step()
|
||||
optimizer.zero_grad()
|
||||
```
|
||||
|
||||
## Plugin Best Practices
|
||||
|
||||
### 1. Validation in `__post_init__`
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class CustomPlugin:
|
||||
learning_rate: float = 1e-3
|
||||
warmup_steps: int = 1000
|
||||
|
||||
def __post_init__(self):
|
||||
# Validate parameters
|
||||
if self.learning_rate <= 0:
|
||||
raise ValueError("learning_rate must be positive")
|
||||
if self.warmup_steps < 0:
|
||||
raise ValueError("warmup_steps must be non-negative")
|
||||
|
||||
# Compute derived values
|
||||
self.min_lr = self.learning_rate * 0.1
|
||||
```
|
||||
|
||||
### 2. Compatibility Checks
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class CustomPlugin:
|
||||
feature_enabled: bool = True
|
||||
|
||||
def is_compatible(self, accelerator):
|
||||
"""Check if plugin is compatible with accelerator config."""
|
||||
if self.feature_enabled and accelerator.mixed_precision == 'fp8':
|
||||
raise ValueError("Custom plugin not compatible with FP8")
|
||||
return True
|
||||
```
|
||||
|
||||
### 3. State Management
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class CustomPlugin:
|
||||
counter: int = 0
|
||||
history: list = None
|
||||
|
||||
def __post_init__(self):
|
||||
if self.history is None:
|
||||
self.history = []
|
||||
|
||||
def update_state(self, value):
|
||||
"""Update plugin state during training."""
|
||||
self.counter += 1
|
||||
self.history.append(value)
|
||||
```
|
||||
|
||||
## Resources
|
||||
|
||||
- Accelerate Plugins: https://huggingface.co/docs/accelerate/package_reference/kwargs
|
||||
- DeepSpeed Config: https://www.deepspeed.ai/docs/config-json/
|
||||
- FSDP Guide: https://pytorch.org/docs/stable/fsdp.html
|
||||
- Custom Training Loops: https://huggingface.co/docs/accelerate/usage_guides/training_tpu
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user