DO NOT MERGE until local validation passes on 3+ golden topics.
Problem: the magic footer (✅ All agents reported back!, emoji tree,
Top voices, Raw results saved) was composed by the synthesizer model
following a "Copy this EXACTLY" template buried 1150 lines into SKILL.md.
Under context pressure, Opus 4.7 dropped it. Three recent /last30days
runs (Opus 4.7, programming language for AI agents, Kanye West)
produced clean prose with no footer and used AI-slop section headers
(## The launch, ## Where it disappoints) instead of flowing paragraphs.
Fix:
1. render.py: new _render_emoji_footer() emits the deterministic footer
as the final block of every compact output. Zero-count sources are
omitted. Tree characters (├─ / └─) computed from populated-line
count. The model no longer assembles the tree from text instructions.
2. render.py: new _site_name_for_url() and _format_web_line_sources()
map URLs to clean publication names (Later, Buffer, CNN, etc.) so
the 🌐 Web line is pre-assembled by Python.
3. last30days.py: compute_save_path_display() turns the save path into
a ~/-relative string that the engine puts in the footer. Signature
change: emit_output() and render_compact() both accept save_path.
4. SKILL.md synthesis contract rewritten:
- Footer template DELETED. Replaced with instruction to include the
engine footer block verbatim.
- URL-to-site-name sub-block DELETED. Engine does this.
- "Calculate actual totals" paragraph DELETED. Engine does this.
- All em-dashes in the synthesis section replaced with ` - ` (single
hyphen with spaces). Em-dashes are the most reliable AI-slop tell.
- New rules: no ## markdown section headers in response body, no
invented title line like "{Topic}: last 30 days", no bold section
labels acting as headers. Bold-lead-in paragraph shape stays.
- SELF-CHECK updated to verify footer presence, no em-dashes, no
body-level headers.
Tests: 15 new tests covering footer emission, zero-source omission,
tree character placement, save-path threading, URL-to-name helper,
Web line formatting, Top voices combination, Polymarket line.
All 127 tests pass across render, rerank, cluster, briefing, CLI,
internals, fun-scoring.
Plan: docs/plans/2026-04-17-003-feat-deterministic-footer-plan.md
Local validation protocol (blocks merge):
- Run /last30days in a fresh Claude Code window on 5 golden topics
- Verify each output contains the footer block verbatim
- Verify zero ## body headers, zero em-dashes/en-dashes, zero invented
title lines
- Report 5x8 pass/fail matrix; all 40 cells must be green before merge
🤖 Generated with Claude Opus 4.7 (1M context) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v2.63.1
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Most users never touch FUN_LEVEL. Default medium was shipping a stats
block but rarely a Best Takes block, and when it did it was below the
cluster fold where a synthesizing model had already stopped reading.
A 2,304-upvote Reddit comment ("WHAT?! I reached my monthly limit
just reading this post") on the 2026-04-17 Opus 4.7 run sat inside
cluster 11 and never made it into synthesis. Four coordinated changes:
1. render: promote Best Takes above the cluster list so the synthesizer
sees comedy before it anchors on cluster 1.
2. render: lower medium threshold from 70 to 55 (heuristic maxes at 80),
drop the two-gem floor to one-gem. Default now reliably emits the
block on typical runs.
3. rerank: score individual top_comments by upvote ratio to their parent
thread. A 2,304-upvote comment on a 300-upvote thread now outranks a
400-upvote comment on a 3,400-upvote thread, which is the viral-wit
signal. Handles both the LLM scoring path and the heuristic fallback.
4. render: merge scored comment gems into Best Takes alongside candidate
gems, sorted together. Comment lines show body + parent title +
r/subreddit or @handle + absolute upvotes.
5. SKILL: tell the synthesizer to quote at least two Best Takes entries
verbatim, with an example of the new comment format.
Plan: docs/plans/2026-04-17-001-feat-default-fun-surfacing-plan.md
🤖 Generated with Claude Opus 4.7 (1M context) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v2.56.1
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Restore the rich synthesis output by closing three prompt-level loopholes
that let the model silently take a degraded path:
1. Research Execution precondition gate. Steps 0.55 (entity resolution)
and 0.75 (query planner) are now non-skippable on WebSearch platforms.
--emit md is banned as a primary user-facing flow; --emit=compact with
--plan is mandatory. OpenClaw --auto-resolve fallback preserved.
2. WebSearch "Sources:" mandate override. The WebSearch tool description
contains a CRITICAL/MUST mandate to append a Sources section. That is
explicitly superseded inside /last30days with matched-register
CRITICAL/MANDATORY override language and a BAD/GOOD example. The
existing web-source line is the citation; nothing appends below the
invitation.
3. Pre-present self-check. Before displaying, the model verifies bold
per-paragraph headlines, per-source emoji stats, quoted highlights,
Polymarket block, coverage footer, and (critically) no trailing
Sources block. One regeneration permitted if checks fail.
Also adds explicit MANDATORY language to the "What I learned" template
requiring bold headline phrases on every narrative paragraph.
Root cause: same-session A/B on 2026-04-15 between /last30days kanye
west (rich output, ran Steps 0.55 + 0.75, --emit=compact --plan) and
/last30days hermes ai (bland output, skipped both, --emit md) showed
the template was fine -- the model was lazily taking a shortcut SKILL.md
tolerated. No engine, render.py, or contributor PR was the cause.
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(normalize): pass YouTube top_comments through with Reddit-compatible shape
_normalize_youtube silently dropped top_comments after enrich_with_comments
populated them, so the downstream signals/render/entity layers never saw
YouTube comments. Map likes->score and text->excerpt so the existing
Reddit-compatible readers Just Work.
Shared _remap_comments helper will be reused for TikTok in a later commit.
* feat(tiktok): fetch top comments via ScrapeCreators when opted in
Mirrors the youtube_comments pattern: new env.is_tiktok_comments_available
gate (requires SCRAPECREATORS_API_KEY + tiktok_comments in INCLUDE_SOURCES),
tiktok.enrich_with_comments ranks posts and fetches via
GET /v1/tiktok/video/comments. Vote field is digg_count; text and user.nickname
come across verbatim. Pipeline calls the enricher right after TikTok search
when the gate is open.
Comment-fetch errors never crash the pipeline — the enricher returns an
empty list on 4xx/5xx.
* feat(normalize): pass TikTok top_comments through with digg_count->score mapping
Instagram uses the same shortform normalizer and has no comment fetcher
today, so the key is harmlessly absent there — no Instagram regression.
* feat(signals): add YouTube + TikTok top-comment score to engagement formula
Mirrors Reddit's 10% top-comment slot. Without top_comments present, the
formula reduces to views-dominant weighting; with a high-signal comment,
the item gets a meaningful bump (log1p(10k) ~ 9.2, weighted 0.10 = ~0.92
on the engagement score).
Updated the existing dominant-weight and missing-fields tests to the new
weights (0.45/0.32/0.13 for YT, 0.45/0.27/0.18 for TT). Views still dominate.
* feat(render): source-aware thresholds and vote labels for top comments
10 upvotes on Reddit signals community interest; 10 likes on a viral
TikTok is noise. Introduce per-source minimums (reddit 10, youtube 50,
tiktok 500) and native vote labels ('upvotes' for Reddit, 'likes' for
YT/TT). First-pass numbers — tune after live observation.
* docs: generalize top-comment quoting to YouTube + TikTok, add tiktok_comments opt-in
Synthesis instructions previously called out Reddit top comments only.
Now cover Reddit/YouTube/TikTok uniformly with source-appropriate vote
labels (upvotes vs likes), and explicitly frame YT transcript highlights
and comments as complementary signals. README and setup-wizard copy
document the new tiktok_comments INCLUDE_SOURCES token.
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
- description leads with imperative 'Research' + 'what people actually say' (strong trigger signal for community/social-research prompts)
- argument-hint shows 3 concrete user phrasings instead of marketing copy
- 176 chars, well under Anthropic's 200-char cap
- preserves all source coverage (Reddit, X, YouTube, TikTok, Hacker News, Polymarket, GitHub, web)
Per ecosystem research (April 2026), trigger description quality is the single
biggest lever separating 500-install skills from 350k-install skills.
Atomic bump across all four manifests:
- SKILL.md (root)
- skills/last30days/SKILL.md (internal spec)
- .claude-plugin/plugin.json
- gemini-extension.json
CHANGELOG entry documents the skill-upload packaging fix, vendor/ removal,
legacy plans/ removal, and the new scripts/build-skill.sh builder.
The SKILL.md prompt header still said v2.9.5 while pyproject.toml
and the rest of the codebase are on v3.0.0.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: INCLUDE_SOURCES config + TikTok/Instagram opt-in in NUX
- INCLUDE_SOURCES=tiktok,instagram in .env forces sources on for all
query types, bypassing the tier system
- NUX shows opt-in modal after ScrapeCreators key is saved: "Also
search TikTok and Instagram?" with honest call-usage warning
- Tier system preserved as default — override only when INCLUDE_SOURCES set
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: neutral call-usage copy — works for free and paid tiers
---------
Co-authored-by: Matt Van Horn <mvanhorn@MacBook-Pro.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add extract_transcript_highlights() that scores sentences by specificity
(numbers, proper nouns, topic relevance) and filters YouTube filler
(subscribe, welcome back, etc). Top 5 highlights shown as structured
bullets in compact output. Full transcript moved to collapsible <details>
block so the LLM reads highlights first, full text on demand.
SKILL.md updated to instruct the judge agent to quote highlights
directly in synthesis, same as Reddit top comments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Updated frontmatter description to be more search-friendly for ClawHub.
Added 12 new tags: deep-research, twitter, bluesky, recency, news,
citations, multi-source, social-media, analysis, web-search, ai-skill,
clawhub. Also added 11 GitHub repo topics.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The short alias /last30 only works on some platforms. Claude Code requires
the full /last30days name, so the follow-up suggestions after comparison
research were producing "Unknown skill: last30" errors.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Mastodon-compatible API at truthsocial.com/api/v2/search.
Opt-in via TRUTHSOCIAL_TOKEN env var (bearer token from browser).
Silent when unconfigured. Full pipeline: search, parse, normalize,
score, dedupe, render across all 10 pipeline files.
27 new tests, 440 total passing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
searchPosts endpoint now returns 403 for unauthenticated requests.
Add session auth via createSession, gate on BSKY_HANDLE + BSKY_APP_PASSWORD
env vars. When unconfigured, Bluesky is completely invisible (no error).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add COMPARISON query type for "X vs Y" research with 3 parallel passes.
Add Bluesky stats line and update all source list references.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove double quotes around $ARGUMENTS so argparse can parse flags
like --deep, --store separately instead of as part of the topic string.
Fixes#61.
- Add ~/.claude/plugins/marketplaces/last30days-skill to the path
discovery loop so marketplace installs can find scripts/.
Fixes#54.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add gemini-extension.json manifest with correct array-format settings,
symlink skills/last30days/SKILL.md to root SKILL.md for Gemini skill
discovery, add Gemini install paths to bash for-loop in both main and
open variant, and add Gemini CLI install instructions to README.
Incorporates the good parts of PR #53 (manifest, paths, README) while
avoiding duplicate SKILL.md, tool name scattering, and allowed-tools
pollution that would have created maintenance issues.
Closes#45
Co-Authored-By: Alex Ferrari <alex@thealexferrari.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PR merges on March 7 (PR #48 Xiaohongshu, upstream merge) regressed
SKILL.md by re-introducing the "Save Research to Documents" section
that v2.9.4 removed. Those branches were forked before v2.9.4 and
brought the old content back via merge resolution.
Fixes: remove save section, restore --save-dir flag on bash command,
update agent mode line, add tool-call guard to STOP instruction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove prompt-injection false positive ("you are now" → "treat yourself as")
- Declare AUTH_TOKEN and CT0 in frontmatter optionalEnv
- Clarify X token access language (no browser session access)
- Add permissions overview block near top of file
Zero functionality changes — metadata and prose only.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add --save-dir flag to last30days.py that saves raw research output
during the existing script run. Remove entire "Save Research to
Documents" section from SKILL.md (~45 lines). No more extra tool
calls, no (No output), no multi-minute cogitation after invitation.
Tested: --mock confirms file creation and duplicate date suffixing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CRITICAL: run_in_background callbacks caused model to re-engage after
save, hallucinate fake "Human:" messages, and generate unsolicited
multi-paragraph responses. Switch to foreground cat > heredoc which
executes sub-second with no callback.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Background Bash heredoc instead of Write tool
- Suppress response text on save completion
- 📎 footer line replaces verbose confirmation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Write tool displays "Wrote N lines..." after the invitation,
ruining the end-of-run experience. Now saves via background Bash
with a subtle 📎 footer line in the invitation text.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Write tool displays "Wrote N lines..." after the invitation,
ruining the end-of-run experience. Now saves via background Bash
with a subtle 📎 footer line in the invitation text.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sync from public repo. Every run now saves the complete briefing as a
topic-named .md file to ~/Documents/Last30Days/. Credit @devin_explores.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bump version to 2.9.1, update changelog and release notes.
Credit @devin_explores for inspiring the feature.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Every run now automatically saves the complete briefing (synthesis,
stats, follow-up suggestions) as a topic-named .md file in the user's
Documents folder. Agent mode also saves. No Python script changes -
this is purely a SKILL.md instruction addition.
Inspired by @devin_explores manually saving results to build a
personal research library.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- New scripts/lib/reddit.py: multi-query expansion, global search,
subreddit discovery, targeted subreddit search, comment enrichment
- 68 results in 17s vs ~15 results in 60-90s (OpenAI)
- Cost: ~$0.02/search vs $0.03-0.10 (15-50x cheaper)
- Real engagement data (score, comments, dates) from API
- No more 429 rate limits on comment enrichment
- Falls back to OpenAI if SCRAPECREATORS_API_KEY missing
- Registered as last30daysbeta for parallel local testing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Instagram Reels as the 8th research source via ScrapeCreators API.
One API key (SCRAPECREATORS_API_KEY) now covers both TikTok and Instagram.
- Add scripts/lib/instagram.py: keyword search, transcript extraction,
relevance scoring, engagement metrics (views, likes, comments)
- Add InstagramItem to schema, normalization, scoring, dedup, rendering
- Add Instagram to orchestrator pipeline, watchlist, and UI spinners
- Update SKILL.md: stats template, citation priority, item format,
URL-to-name extraction rules, anti-Sources instruction
- Update README and CHANGELOG for v2.8
- Fix: Instagram/TikTok not running in --search= web-only path
- Fix: web stats line showing full URLs instead of domain names
- Replace APIFY_API_TOKEN with SCRAPECREATORS_API_KEY throughout
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all Apify references with ScrapeCreators. Key points:
- No subscription required (was $5/mo with Apify)
- 100 free credits, pay-as-you-go after
- SCRAPECREATORS_API_KEY replaces APIFY_API_TOKEN
- Backwards compatible: APIFY_API_TOKEN still works as fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove double quotes around $ARGUMENTS in SKILL.md so bash word-splits
the expansion, and change argparse topic from nargs="?" to nargs="*"
so multi-word topics still work. Also document --store, --include-web,
--diagnose, and --timeout flags in the Options section.
Closes#36
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When running in Claude Code, the assistant has a built-in WebSearch tool
that's free and higher quality than Parallel AI/Brave/OpenRouter. Adding
--no-native-web to the SKILL.md invocation defers web search to the
assistant, saving API credits. OpenClaw invocations don't pass this flag,
so they continue using native web backends.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add TikTok search, scoring, and rendering using the Apify platform
(clockworks/tiktok-scraper actor). Users bring their own APIFY_API_TOKEN
($5/month free credits, no CC required). The shared apify_client_wrapper
module is designed for reuse by future Facebook/Instagram sources.
- New modules: tiktok.py (search + caption extraction), apify_client_wrapper.py
- Schema: TikTokItem dataclass, shares field on Engagement, Report.tiktok
- Pipeline: normalize → filter → score → sort → dedupe → cross-link → render
- Scoring: 0.50*log1p(views) + 0.30*log1p(likes) + 0.20*log1p(comments)
- SKILL.md bumped to v2.7 with TikTok stats, citations, and security docs
- 26 unit tests covering relevance, normalize, score, dedupe, render, round-trip
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
URLs in markdown links wrap badly in terminals (discovered after first
fix attempt). Change to plain names like "Newsweek, Sportskeeda, Medium"
on the Web: stats line. Update citation note to explain the reason.
Tested on Dor Brothers, Kanye West, Logan Paul - no trailing Sources:
block appeared in any of the three test runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Matt Van Horn