Live Ghostty test of the prior amendment confirmed RULE A (inline links)
works on first pass, but exposed a new regression: the established output
contract collapsed. First-pass output had "Story 1/2/3" plain-prose
sections instead of bold-headline paragraphs, a plain "Stats" heading
with - bullets instead of the ├─ └─ │ emoji tree, no ✅ All agents
reported back! line, no --- separators, and a flat "I am now an expert"
closer instead of the QUERY_TYPE-specific invitation variant.
Same failure pattern as before (emphasis imbalance between co-equal
rules), opposite direction: the link rules at the top of the output
section became the loudest prose and the existing structural rules
(bold headlines, stats template, emoji tree, QUERY_TYPE invitation,
--- separators) read as style preferences by comparison.
Fix is additive, not reductive. The TWO HARD RULES framing becomes the
OUTPUT CONTRACT with six rules named A-F, each at equal emphasis, stated
in parallel construction:
- RULE A: inline markdown links on every citation.
- RULE B: no trailing list of links.
- RULE C: bold headline per narrative paragraph (**Headline** — body).
- RULE D: stats block with ✅ opening, ├─/└─/│ tree, emoji prefix per
active source. No plain - bullets, no missing emoji.
- RULE E: QUERY_TYPE-specific invitation variant with 2-3 example
follow-ups from this run. No flat "I am now an expert" closer.
- RULE F: --- horizontal rules before and after the stats block.
Closing sentence: "These six rules are a set, not a priority order.
Satisfying some by violating others is a bug, not a compromise."
Corresponding changes:
- PRE-SYNTHESIS COMMIT expanded from 6 link-focused bullets to 7
bullets covering template/headlines/stats/separators/sources/trailing-
list. Structure-first ordering primes the agent to reach for the
template before drafting prose.
- PRE-PRESENT SELF-CHECK reworked. Items 1 and 2 become counted
structural checks (≥3 bold headline openers, ≥3 ├─ tree characters,
one emoji per active source). New items 5 and 6 add separator and
invitation-variant checks. Items 7-10 preserve the existing RULE B
/ RULE A / no-raw-URL / no-broken-link checks.
- Release-notes entry extended to mention the preserved output
contract alongside the clickable links.
No Python changes. This is the third commit on PR #286.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Live test on Ghostty after the prior commit showed a regression: first-pass
/last30days synthesis stripped all inline links. Only after the user asked
"weren't these links supposed to be clickable?" did the agent re-render
with inline links. Agent's own root cause: "I misread LAW 1 — over-applied
'no trailing Sources: block' to mean 'no URLs anywhere'."
The templates and per-citation examples were already correct. The regression
was in EMPHASIS BALANCE — the anti-trailing-Sources prohibition used the
loudest prose (CRITICAL OVERRIDE, MANDATORY, MUST NOT) while the pro-inline-
link rule read as lighter guidance. On a fresh first read, the agent
collapsed both into one prohibition and stripped every citation link.
This commit rebalances emphasis and adds a forcing function:
- New "TWO HARD RULES FOR LINKS" framing at the top of the output section
names RULE A (inline links required on every citation) and RULE B (no
trailing list of links) as a matched pair with equal rhetorical weight.
Explicit: "These rules are complementary, not alternatives."
- New "PRE-SYNTHESIS COMMIT" step forces the agent to identify sources,
pull URLs, and commit to inline-link placement BEFORE drafting — not
audit after the fact.
- Existing CRITICAL OVERRIDE block now opens by pointing back to the
matched-pair framing and drops the "Your citation is the Web: line.
Nothing else is needed." priming sentence that biased the agent toward
link-free drafts.
- New BAD/ALSO-BAD/GOOD triple in the CRITICAL OVERRIDE block shows the
exact regression shape (stripping inline links to "comply" with RULE B)
as a distinct failure mode, not just the trailing-list failure.
- PRE-PRESENT SELF-CHECK reworked: item 6 becomes "RULE B — no trailing
list", item 7 becomes "RULE A — inline links present on first pass"
with a positive count check that regenerates if zero inline links
appear when raw data had URLs (the exact 2026-04-20 regression repro).
- Release notes entry gains one line describing the first-pass guarantee.
No Python changes. This is an amendment on PR #286 folding the fix into
the same review.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Every cited source — X handles, subreddits, publications, YouTube channels,
Polymarket markets — now renders as a blue CMD-clickable markdown link in the
terminal. Raw URL strings remain forbidden everywhere in the output.
The old rule "NEVER paste raw URLs" blocked both raw URLs and links. Claude
Code renders CommonMark [text](url) as hyperlinks (URL hidden), so the new
rule is: "every link must be [text](url), never a raw URL string". Plain text
is the fallback only when the raw research dump has no URL for a source.
Changes:
- SKILL.md: invert URL formatting rule; update narrative / KEY PATTERNS /
RECOMMENDATIONS / stats-block templates; retire the "I have all the links...
Just ask." closer (it's now inline); add two new PRE-PRESENT SELF-CHECK
items (no raw http/https strings, no broken empty links).
- README.md: note the clickable-links capability in the "How it works" list.
- release-notes.md: 2026-04-20 entry crediting @jay_k and @photomatt.
No Python changes — the raw research dump already carries URLs per item;
this is synthesis-side formatting only. No breaking changes to the engine,
the schema, or the CLI flags.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds commands/last30days.md so /last30days registers as a Claude Code
slash command for plugin users. Users type /last30days and autocomplete
prefix-matches to the canonical /last30days:last30days form (same as
/ce:plan resolving to /compound-engineering:ce-plan).
Removes skills/last30days-nux/, a byte-identical duplicate of the root
SKILL.md that created confusing /last30days:last30days-nux autocomplete
entries via Claude Code's plugin namespacing. Root SKILL.md remains
the canonical skill source; natural-language skill-selector invocation
is unchanged.
Recovery for users on v3.0.4: /plugin update last30days then /reload-plugins.
Closes#239 (path-escape error was already fixed in v3.0.4 by dropping
the rogue 'skills' key; v3.0.5 adds the slash command on top).
Supersedes #257 (suggested './' -> '.' workaround is obsolete since
v3.0.4 dropped the 'skills' key entirely, matching ecosystem standard).
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
PR #260 wired YouTube comment enrichment against
`/v1/youtube/video/comments` with `id=<video_id>`, but the endpoint
requires `url=https://www.youtube.com/watch?v=<video_id>`. Every enrich
call was returning 400 "missing_parameter: you must provide a url", so
no YouTube items ever carried `top_comments`.
The SC transcript fallback (`_sc_fetch_transcript`) had the identical
contract mistake. It was latent because `_fetch_transcript` prefers
yt-dlp and the SC path only fires when yt-dlp is missing, but it would
have failed the same way on hosts without yt-dlp installed.
Switching both callers to `url=` surfaces a second issue in the
response parser: SC returns `author` as `{"name": "@handle", ...}` and
nests like counts under `engagement.likes`, not top-level. The parser
was reading `author` as a string and missing the nested likes, so even
after the param fix every comment would land with an object-shaped
author and 0 likes.
- `_fetch_video_comments`: send `url=` on both urllib and requests branches
- `_sc_fetch_transcript`: same
- Response parser: extract `author.name` when author is a dict, read
`engagement.likes` when top-level `likes` is absent, prefer
`publishedTime` / `publishedTimeText` for date. Legacy string-author
and top-level-likes shapes still work, so existing mocks are unchanged.
Verified live against api.scrapecreators.com: `_fetch_video_comments`
now returns fully-populated comments with real @handles and like
counts (e.g. "@JennyNicholson: ... (49000 likes, 2025-04-15)"). All
tests in youtube_yt/normalize/signals/render pass.
Plan: docs/plans/2026-04-15-002-fix-youtube-comments-scrapecreators-param-plan.md
🤖 Generated with Claude Opus 4.6 (1M context) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v2.56.1
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restore the rich synthesis output by closing three prompt-level loopholes
that let the model silently take a degraded path:
1. Research Execution precondition gate. Steps 0.55 (entity resolution)
and 0.75 (query planner) are now non-skippable on WebSearch platforms.
--emit md is banned as a primary user-facing flow; --emit=compact with
--plan is mandatory. OpenClaw --auto-resolve fallback preserved.
2. WebSearch "Sources:" mandate override. The WebSearch tool description
contains a CRITICAL/MUST mandate to append a Sources section. That is
explicitly superseded inside /last30days with matched-register
CRITICAL/MANDATORY override language and a BAD/GOOD example. The
existing web-source line is the citation; nothing appends below the
invitation.
3. Pre-present self-check. Before displaying, the model verifies bold
per-paragraph headlines, per-source emoji stats, quoted highlights,
Polymarket block, coverage footer, and (critically) no trailing
Sources block. One regeneration permitted if checks fail.
Also adds explicit MANDATORY language to the "What I learned" template
requiring bold headline phrases on every narrative paragraph.
Root cause: same-session A/B on 2026-04-15 between /last30days kanye
west (rich output, ran Steps 0.55 + 0.75, --emit=compact --plan) and
/last30days hermes ai (bland output, skipped both, --emit md) showed
the template was fine -- the model was lazily taking a shortcut SKILL.md
tolerated. No engine, render.py, or contributor PR was the cause.
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
plugin.json has declared "skills": ["./"] unchanged since v2.1.0. That
value used to work on older Claude Code but current versions reject it
with: Path escapes plugin directory: ./ (skills). The error surfaces
on fresh /doctor runs even after v3.0.3 restored the archive contents.
Fix: omit the "skills" key entirely. Every other plugin in the Claude
Code marketplace ecosystem (compound-engineering, coding-tutor, codex,
esper, 15+ Anthropic official plugins) omits this key and the loader
auto-discovers skills/*/SKILL.md. Matching that pattern clears the
path-escape error on v2.1.109+ and remains compatible with older
Claude Code versions where the default-discovery path was already the
working code path.
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
v3.0.3's fix (#262) restored skills/ and .claude-plugin/ to the git
archive, which Claude Code needs for /plugin install. But
scripts/build-skill.sh uses the same archive to produce the claude.ai
.skill bundle, which must contain exactly one root SKILL.md and stay
under the 200-file cap.
Fix: after git archive, 'zip -d' strips both directories from the
.skill bundle. git archive output is unchanged (Claude Code still
gets the full tarball on /plugin install).
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
v3.0.1 added .gitattributes rules that excluded both directories from
git archive output, shrinking the claude.ai .skill bundle. But Claude
Code's /plugin install fetches the SAME archive, so users installing
v3.0.1 or v3.0.2 received a tarball with no plugin manifest and no
skill files. Install appeared successful but the plugin was a useless
empty shell.
Proof:
git archive v3.0.0 | grep 'skills/|\.claude-plugin/' | wc -l # 8
git archive v3.0.1 | grep 'skills/|\.claude-plugin/' | wc -l # 0
git archive v3.0.2 | grep 'skills/|\.claude-plugin/' | wc -l # 0
No issue reports yet because:
- Cached pre-v3.0.1 installs keep working (it's the new-install path
that's broken)
- The breakage is under 24 hours old
- Users invoking the skill via natural language go through
skill-selector rather than /last30days slash command
Also reverts v3.0.2's "skills": ["skills"] back to "./", the value
that shipped in every tag from v2.1.0 through v3.0.0. That change was
a misdiagnosis; the manifest wasn't in the tarball anyway so it had
no effect on user-visible installs.
Archive file count after fix: 97 (cap is 200, plenty of room).
Follow-up: move claude.ai-specific bundle exclusions into
scripts/build-skill.sh where they belong, rather than .gitattributes
which cannot distinguish between the two distribution channels.
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Two regressions were silently breaking /last30days for every user:
1. plugin.json declared "skills": ["./"], which newer Claude Code
rejects with "Path escapes plugin directory: ./ (skills)". The
skill loader refused to register the command, so /last30days
returned "Unknown command" even though /plugin list showed the
plugin as installed. Fix: "skills": ["skills"] so the loader
scans the real subdirectory.
2. marketplace.json pinned "version": "3.0.0" while plugin.json
advertised "3.0.1". The /plugin resolver used the marketplace
version and could install a phantom user-scope copy at a stale
SHA alongside the correct project-scope install, creating
duplicate skill-name collisions. Both manifests now agree on
3.0.2.
Prior attempt: commit 93fbed2 fixed (1) before but got reverted.
This lands both fixes together in a tagged release so users can
/plugin update to recover.
Recovery for affected users is in CHANGELOG.md under 3.0.2.
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
* feat(normalize): pass YouTube top_comments through with Reddit-compatible shape
_normalize_youtube silently dropped top_comments after enrich_with_comments
populated them, so the downstream signals/render/entity layers never saw
YouTube comments. Map likes->score and text->excerpt so the existing
Reddit-compatible readers Just Work.
Shared _remap_comments helper will be reused for TikTok in a later commit.
* feat(tiktok): fetch top comments via ScrapeCreators when opted in
Mirrors the youtube_comments pattern: new env.is_tiktok_comments_available
gate (requires SCRAPECREATORS_API_KEY + tiktok_comments in INCLUDE_SOURCES),
tiktok.enrich_with_comments ranks posts and fetches via
GET /v1/tiktok/video/comments. Vote field is digg_count; text and user.nickname
come across verbatim. Pipeline calls the enricher right after TikTok search
when the gate is open.
Comment-fetch errors never crash the pipeline — the enricher returns an
empty list on 4xx/5xx.
* feat(normalize): pass TikTok top_comments through with digg_count->score mapping
Instagram uses the same shortform normalizer and has no comment fetcher
today, so the key is harmlessly absent there — no Instagram regression.
* feat(signals): add YouTube + TikTok top-comment score to engagement formula
Mirrors Reddit's 10% top-comment slot. Without top_comments present, the
formula reduces to views-dominant weighting; with a high-signal comment,
the item gets a meaningful bump (log1p(10k) ~ 9.2, weighted 0.10 = ~0.92
on the engagement score).
Updated the existing dominant-weight and missing-fields tests to the new
weights (0.45/0.32/0.13 for YT, 0.45/0.27/0.18 for TT). Views still dominate.
* feat(render): source-aware thresholds and vote labels for top comments
10 upvotes on Reddit signals community interest; 10 likes on a viral
TikTok is noise. Introduce per-source minimums (reddit 10, youtube 50,
tiktok 500) and native vote labels ('upvotes' for Reddit, 'likes' for
YT/TT). First-pass numbers — tune after live observation.
* docs: generalize top-comment quoting to YouTube + TikTok, add tiktok_comments opt-in
Synthesis instructions previously called out Reddit top comments only.
Now cover Reddit/YouTube/TikTok uniformly with source-appropriate vote
labels (upvotes vs likes), and explicitly frame YT transcript highlights
and comments as complementary signals. README and setup-wizard copy
document the new tiktok_comments INCLUDE_SOURCES token.
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Internal ce:plan output shouldn't ship on the public repo.
Adds docs/plans/ to .gitignore and removes the two already-tracked
plan files from the index. Working copies stay local for reference.
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- install matrix now leads with claude.ai (widest audience, one-click path)
- direct download link to GitHub release's 'latest' asset URL
- 3-step UI walkthrough with link to Settings > Capabilities > Skills
- Claude Code / OpenClaw / Gemini / manual paths still documented, collapsed
- removes the bash scripts/build-skill.sh requirement from end-user flow
- description leads with imperative 'Research' + 'what people actually say' (strong trigger signal for community/social-research prompts)
- argument-hint shows 3 concrete user phrasings instead of marketing copy
- 176 chars, well under Anthropic's 200-char cap
- preserves all source coverage (Reddit, X, YouTube, TikTok, Hacker News, Polymarket, GitHub, web)
Per ecosystem research (April 2026), trigger description quality is the single
biggest lever separating 500-install skills from 350k-install skills.
Atomic bump across all four manifests:
- SKILL.md (root)
- skills/last30days/SKILL.md (internal spec)
- .claude-plugin/plugin.json
- gemini-extension.json
CHANGELOG entry documents the skill-upload packaging fix, vendor/ removal,
legacy plans/ removal, and the new scripts/build-skill.sh builder.
Wraps git archive with --prefix=last30days/ so the zip contains a single
top-level skill folder matching SKILL.md's name: frontmatter. Enforces:
- refuses to build with a dirty working tree (prevents shipping untracked changes)
- fails if zip exceeds 200 files (claude.ai's empirical upload cap)
- fails if zip contains more than one SKILL.md (avoids name: confusion)
Output at dist/last30days.skill (gitignored).
Both plans describe work that was already shipped:
- feat-add-websearch-source.md - websearch is in the v3 pipeline (scripts/lib/perplexity.py etc)
- fix-strict-date-filtering.md - date filtering is enforced in scripts/lib/dates.py
New planning goes in docs/plans/ following the ce:plan convention.
Root vendor/package/ was an accidentally committed extracted npm tarball
(steipete-bird-0.8.0). Zero importers: the real vendored X client lives
at scripts/lib/vendor/bird-search/, referenced by scripts/lib/bird_x.py
and tests/test_bird_x.py.
Removes 215 files + 1 .tgz, dropping repo from 406 to 191 files and
clearing the claude.ai skill-upload 200-file cap.
Adds /vendor/ to .gitignore (leading slash so scripts/lib/vendor/ is unaffected).
Six source modules each defined an identical 8-line _sc_headers(token)
function returning {"x-api-key": token, "Content-Type": "application/json"}.
Moved it to http.scrapecreators_headers() and migrated all 33 call sites.
Affected files: reddit.py, threads.py, tiktok.py, instagram.py, pinterest.py,
youtube_yt.py. Zero per-source variation, zero behavior change.
Net: -40 lines. 1022 tests pass (15 pre-existing failures unchanged).
Live smoke test: reddit search returns 12 threads with full engagement.
When a tweet has no engagement metrics, _first_of() returns None for
every key, producing {"likes": None, "reposts": None, ...}. This
all-None dict propagates to signals.py where it is treated as "data
exists but is zero" rather than "no data available." Return None
instead when every engagement field is missing.
evaluate_search_quality.py and e2e_comparison.py both reference
fixtures/eval_topics.json with hardcoded fallbacks. Supply the
actual fixture: 8 topics spanning all intent types, selected via
MMR dispersion across domains (tech, health, sports, finance,
consumer products).
pyproject.toml declares pytest-cov as a dev dependency and configures
[tool.coverage.run], but the generated .coverage database and htmlcov/
report directory are not gitignored. Also add .venv/ (standard Python
virtualenv) and .memsearch/ (session memory) to keep the working tree
clean for contributors.
The SKILL.md prompt header still said v2.9.5 while pyproject.toml
and the rest of the codebase are on v3.0.0.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Matt Van Horn