Add --save-dir flag to last30days.py that saves raw research output
during the existing script run. Remove entire "Save Research to
Documents" section from SKILL.md (~45 lines). No more extra tool
calls, no (No output), no multi-minute cogitation after invitation.
Tested: --mock confirms file creation and duplicate date suffixing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sync from public repo. Every run now saves the complete briefing as a
topic-named .md file to ~/Documents/Last30Days/. Credit @devin_explores.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three focused improvements based on 5 full-pipeline beta tests:
1. Elevate top Reddit comments in scoring and rendering
2. Improve subreddit discovery heuristic for ambiguous queries
3. Make ScrapeCreators the default recommended Reddit method
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Instagram Reels as the 8th research source via ScrapeCreators API.
One API key (SCRAPECREATORS_API_KEY) now covers both TikTok and Instagram.
- Add scripts/lib/instagram.py: keyword search, transcript extraction,
relevance scoring, engagement metrics (views, likes, comments)
- Add InstagramItem to schema, normalization, scoring, dedup, rendering
- Add Instagram to orchestrator pipeline, watchlist, and UI spinners
- Update SKILL.md: stats template, citation priority, item format,
URL-to-name extraction rules, anti-Sources instruction
- Update README and CHANGELOG for v2.8
- Fix: Instagram/TikTok not running in --search= web-only path
- Fix: web stats line showing full URLs instead of domain names
- Replace APIFY_API_TOKEN with SCRAPECREATORS_API_KEY throughout
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove double quotes around $ARGUMENTS in SKILL.md so bash word-splits
the expansion, and change argparse topic from nargs="?" to nargs="*"
so multi-word topics still work. Also document --store, --include-web,
--diagnose, and --timeout flags in the Options section.
Closes#36
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When running in Claude Code, the assistant has a built-in WebSearch tool
that's free and higher quality than Parallel AI/Brave/OpenRouter. Adding
--no-native-web to the SKILL.md invocation defers web search to the
assistant, saving API credits. OpenClaw invocations don't pass this flag,
so they continue using native web backends.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add TikTok search, scoring, and rendering using the Apify platform
(clockworks/tiktok-scraper actor). Users bring their own APIFY_API_TOKEN
($5/month free credits, no CC required). The shared apify_client_wrapper
module is designed for reuse by future Facebook/Instagram sources.
- New modules: tiktok.py (search + caption extraction), apify_client_wrapper.py
- Schema: TikTokItem dataclass, shares field on Engagement, Report.tiktok
- Pipeline: normalize → filter → score → sort → dedupe → cross-link → render
- Scoring: 0.50*log1p(views) + 0.30*log1p(likes) + 0.20*log1p(comments)
- SKILL.md bumped to v2.7 with TikTok stats, citations, and security docs
- 26 unit tests covering relevance, normalize, score, dedupe, render, round-trip
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
URLs in markdown links wrap badly in terminals (discovered after first
fix attempt). Change to plain names like "Newsweek, Sportskeeda, Medium"
on the Web: stats line. Update citation note to explain the reason.
Tested on Dor Brothers, Kanye West, Logan Paul - no trailing Sources:
block appeared in any of the three test runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The WebSearch tool has a system-level mandate to append a Sources:
section at the end of every response. SKILL.md's old "DO NOT output
Sources: list" instruction was too weak to override it.
Fix: redirect citations into the stats block's Web: line as inline
links. The WebSearch citation requirement is satisfied there; an
explicit note after the stats block tells the model not to append
a separate trailing section.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- .gitignore: protect docs/comparison-results/ and other private
benchmark artifacts from accidental upstream push
- docs/plans: add plan for PR #37 Codex auth finalization
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removes disable-model-invocation restriction so the skill can be called
by other agents via the Skill tool. Adds --agent flag for non-interactive
report output (skips intro, AskUserQuestion, wait pause, and invitation).
Fixes false security doc claiming autonomous invocation was blocked.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reorder v2.5 features: Polymarket prediction markets and HN lead as #1,
multi-signal quality-ranked relevance scoring as #2. Add Anthropic Odds
example showcasing 11 live markets from a two-word query. Add Anthropic
and OpenAI Polymarket transcripts to launch tweets.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Gamma API only searches event titles/slugs, missing markets where the
topic is an outcome (e.g., "Arizona" in NCAA Tournament Winner). This adds:
- All-word query expansion (not just first word): "Arizona Basketball" now
searches "Arizona", "Basketball" independently
- Tag-based domain expansion: extracts category tags (e.g., "NCAA") from
first-pass results and searches those as a second pass
- Neg-risk binary market synthesis: shows team names from market questions
instead of generic Yes/No outcomes
- Question shortening: extracts "Arizona" from "Will Arizona win the NCAA
Tournament?" for clean display
- Increased depth (3 pages) and result caps (15) for more coverage
Live results: "Arizona Basketball" now finds NCAA Tournament Winner (12%),
#1 Seed (88%), Big 12 Champion (69%). "Iran War" returns 15 markets (up
from 9) with no regression.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- _compute_text_similarity() now checks outcome names with bidirectional
substring matching (0.85) and token overlap (0.7), not just event titles
- Collect outcomes from ALL active markets per event, filter to >1% price
- Reorder outcome_prices to surface topic-matching outcome before top-3 truncation
- Add SKILL.md "Prediction Markets" synthesis section with structural/long-term
market preference, domain examples, citation format, and narrative weaving
- Add Polymarket to citation priority list between HN and Web
- Update stats box template to show up to 5 market odds
- Fix render.py "vol24h" label to "volume"
- Add NCAA seed fixture event for outcome-only matching tests
- 82 polymarket tests pass (14 new)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Polymarket results now rank by text similarity, volume, liquidity, price
movement, and competitive score instead of API return position. Also fixes
pagination (DEPTH_CONFIG now controls page count, not a no-op limit param)
and caps results after re-ranking.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Search Polymarket's free Gamma API for relevant prediction markets on any
topic. Uses smart multi-query expansion to cast a wider net (e.g., "Arizona
Basketball" also searches "Arizona"), merges and dedupes by event ID, and
shows price movement context ("up 22.5% this week"). No API key required.
Also hides sources with zero results from the stats output (all sources).
54 new tests, all passing. Full pipeline integration with scoring, dedupe,
cross-source linking, and rendering.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a topic is a person/brand (e.g. "Dor Brothers", "Jason Calacanis"),
the agent now resolves their X handle via WebSearch before running the
script, then passes --x-handle to search their posts unfiltered (no
topic keywords required). This finds posts the entity made without
mentioning their own name.
- SKILL.md + OpenClaw variant: Step 0.5 handle resolution instructions
- last30days.py: --x-handle CLI arg, passed through to _run_supplemental()
- bird_x.search_handles(): topic is now Optional[str] for unfiltered mode
- schema.py: resolved_x_handle field on Report
- render.py: show resolved handle in stats output
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ran 15-way blinded comparison (5 topics x 3 versions). CROSS won all 5 topics
(4.74/5.0 avg vs HN 4.10, Base 3.73). Then improved CROSS further:
- dedupe.py: hybrid similarity (token+trigram Jaccard) at 0.40 threshold,
cross-source links went from 3 to 13 items across 5 topics
- render.py: [xref: HN5, HN4] -> [also on: HN, Reddit] for human-readable tags
- youtube_yt.py: SYNONYMS dict so "hip hop" matches "rap" (0.33 -> 0.71 score)
- SKILL.md: instruction #7 tells Claude to lead with cross-platform signals
Validation: improved CROSS scores 4.38/5.0 vs original 3.98 (+0.40), wins 4/5
topics. Biggest gains in specificity (+0.8) and format compliance (+1.0).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
YouTube videos now get real relevance scores based on token overlap
between the search query and video title (was hardcoded at 0.7).
Uses ratio overlap with stopword removal, floored at 0.1.
Cross-source linking annotates items that discuss the same story
across different platforms (e.g., Reddit + HN + X). Items get
bidirectional cross_refs displayed as [xref: R3, HN5] in compact
output so Claude can triangulate multi-platform coverage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
HN was appearing before YouTube in the stats block, sort tiebreaker,
and source status. Now consistently: Reddit > X > YouTube > HN > Web.
Also restored emoji + box-drawing chars in test skill SKILL.md.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add HN search via free Algolia API (no key needed). Two-phase approach:
search for stories, then enrich top ones with comments. Integrated into
the full pipeline (normalize, score, dedupe, render) running in parallel
with Reddit/X/YouTube. Source priority: Reddit > X > HN > YouTube > Web.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add CHANGELOG.md (Keep a Changelog format) and release-notes.md
for the v2.1.0 GitHub Release. Also includes queued copy updates:
plugin.json version bump to 2.1.0, YouTube added to descriptions,
SKILL.md security/permissions section and ClawdBot metadata,
updated watchlist examples, and v2.1 launch post refinements.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace all em dashes with hyphens
- "Claude Code skill" -> "skill" in intro
- Ralph Wiggum -> Seedance 2.0 / Nano Banana Pro
- tweet/Tweet -> post/Post throughout
- "signal of a tweet" -> "signal of a single post"
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add agents/openai.yaml for Codex skill discovery
- Make SKILL.md script path portable (repo, Claude, Codex, agents dirs)
- Platform-neutral output text ("assistant" instead of "Claude")
- Sandbox-friendly cache/output dirs with env var overrides and tempdir fallback
- Add Codex installation docs to README
Inspired by PR #24 (el-analista) and PR #5 (jblwilliams).
Zero impact on existing Claude Code behavior.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Split V2.1 blurb into two clear bullets (YouTube + X search)
- Better X auth section with browser-specific notes and verify command
- Updated intro/tagline to mention YouTube as 4th source
- Added social post drafts and updated issue #19 response with YouTube
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
YouTube search and transcript extraction runs automatically when yt-dlp
is installed. Searches for topic videos from the last N days, fetches
auto-generated transcripts for top results, and feeds them through the
same scoring pipeline (relevance + recency + engagement) as Reddit/X.
New files:
- youtube_yt.py: search, transcript extraction, VTT cleanup
Modified files:
- schema.py: YouTubeItem dataclass, updated Report
- normalize.py: normalize_youtube_items()
- score.py: YouTube engagement scoring (views-dominated)
- dedupe.py: YouTube deduplication
- render.py: YouTube section in compact output
- env.py: is_ytdlp_available() check
- ui.py: YouTube progress messages
- last30days.py: _search_youtube(), parallel execution with Reddit/X
- SKILL.md: YouTube in stats box, citation priority
- README.md: YouTube docs, yt-dlp requirement, Peter shoutout
Inspired by Peter Steinberger's yt-dlp + summarize toolchain approach.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Vendor Bird's Twitter GraphQL search client directly into /last30days,
eliminating the dependency on `npm install -g @steipete/bird`. X search
now works out of the box with just Node.js 22+ and browser cookies.
- Add vendored bird-search.mjs wrapper (search-only subset of Bird v0.8.0)
- Vendor @steipete/sweet-cookie for browser cookie extraction
- Update bird_x.py to call vendored Node.js module instead of `bird` binary
- Update README.md and SKILL.md for v2.1 (remove Bird CLI install steps)
- Include Bird's MIT LICENSE in vendor directory
The fallback chain is: vendored search -> xAI API key -> web-only mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three bugs in _extract_core_subject():
1. Multi-word noise phrases ("what are", "how to") never matched
because code compared individual words against multi-word strings.
"what are people saying about DeepSeek R1" became "what are people
saying" — losing the entire topic.
2. Missing meta words — "prompt", "techniques", "tips" weren't
filtered (only "prompting" was). "vibe motion best prompt
techniques" kept 4 keywords instead of 2.
3. No retry on 0 results — Reddit retries with simplified queries
but X accepted 0 and moved on.
Fix: Two-phase extraction (strip multi-word prefixes/suffixes first,
then individual noise words), expanded noise set, max 3 words (was 4),
and automatic retry with first 2 words when Bird returns 0 results.
Before → After:
- "vibe motion best prompt techniques" → "vibe motion" (was 4 words, 0 results)
- "what are people saying about DeepSeek R1" → "deepseek r1" (was "what are people saying")
- "nano banana pro prompts for gemini" → "nano banana pro" (was 4 words)
Tested: vibe motion (12 X posts, was 0), DeepSeek R1 (12 posts),
kanye west (12 posts, no regression).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After the initial broad search (Phase 1), extract key entities from results
and run targeted secondary searches to surface content the broad pass missed:
- New entity_extract.py: parses @handles, #hashtags, subreddits from results
- bird_x.py: search_handles() does targeted from:handle searches via Bird CLI
- openai_reddit.py: search_subreddits() uses Reddit's free .json search endpoint
- last30days.py: Phase 2 orchestration runs after enrichment, merges + dedupes
Tested with "kanye west" (+9 Reddit, +1 X) and "claude code skills" (+6 Reddit, +1 X).
Phase 2 is skipped on --quick mode. Default caps at 3 handles/subs, deep at 5.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add citation priority rule: @handles > r/subreddits > web sources
- Fix examples: "per @honest30bgfan_" instead of "per Rolling Stone"
- Add "lead with people, not publications" instruction
- Stats box: web now shows "(supplementary)" instead of domain list
- No changes to actual research — same quality, better attribution framing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Feature audit found 11 things v2 dropped from v1. Restored the 7 that affect
output quality: quality checklist for prompts, anti-pattern examples, self-check
instruction, "don't re-search" context memory, prompt format anti-pattern,
multi-prompt guidance, and emoji footer with engagement counts.
Also adds visible query parsing display (🔍 **{TOPIC}** · {QUERY_TYPE}) and
the full v1-vs-v2 comparison test plan with all 17 test queries from README.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The old v1 skill worked because context: fork was SILENTLY IGNORED
due to Claude Code bug #17283. The skill ran inline in the main
conversation. Claude Code 2.1+ fixed the bug and now properly
honors fork mode, creating an isolated subagent that ignores all
instruction ordering (text output, bash-first, etc.).
Fix: remove context: fork so the skill runs inline again, matching
the behavior the user knows works. Also restored v1-style
instruction flow: parse intent first, then run script, then WebSearch.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Stats: replace BAD/GOOD examples with strict fill-in-the-blank template
- Reddit: add subreddit-targeted fallback search, soften scoring penalties
(engagement -10→-3, date confidence -10→-5), add minimum result guarantee
- Citations: limit to 1 per insight, short format, no engagement metrics
- Summary: add bold topic headers template for structured paragraphs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Explore agent was treating SKILL.md as documentation to summarize
instead of instructions to follow. Removed agent:Explore from frontmatter
and restructured opening to immediately command script execution.
Also adds release plan for Bird CLI v2.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
12-task implementation plan for integrating Bird CLI as free X/Twitter
search alternative. Includes detection, interactive install, search
dispatch, and main flow integration.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Adds design doc for integrating Bird CLI as an alternative X/Twitter
search source. Bird uses browser cookies (free, no API key) and
provides direct GraphQL access to X.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Matt Van Horn