Commit Graph

373 Commits

Author SHA1 Message Date
Matt Van Horn 23b7906d63 docs: add open variant section and new CLI flags to README
- Document open variant installation (watchlist, briefings, history)
- Add --store, --diagnose, --include-web to options table
- Note optional web search API keys (Parallel AI, Brave, OpenRouter)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 23:47:54 -08:00
Matt Van Horn 9ff00ce38b feat(open): create open variant SKILL.md and reference files
Router SKILL.md dispatches to mode-specific references:
- research.md: one-shot research with --store for persistence
- watchlist.md: add/remove/list/run topics
- briefing.md: daily/weekly briefing generation
- history.md: query/search accumulated findings
- context.md: agent memory template

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 23:46:35 -08:00
Matt Van Horn 08e2010554 feat(engine): add native web search, --store, and --diagnose to main engine
- _search_web() dispatches to Parallel AI / Brave / OpenRouter based on config
- Web results flow through full pipeline: normalize → score → dedupe
- --diagnose shows all source availability (API keys, Bird, YouTube, web backends)
- --store persists findings to SQLite via store.py for watchlist/briefing system
- run_research() now returns web_items alongside reddit/x/youtube
- web_needed flag only set when no native web backend is available

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 23:43:34 -08:00
Matt Van Horn ba330e9a0c feat(open): Port web search backends, persistence layer, and env merge from openclaw 2026-02-14 23:35:44 -08:00
Matt Van Horn a09413608d feat: Add Codex CLI compatibility
- Add agents/openai.yaml for Codex skill discovery
- Make SKILL.md script path portable (repo, Claude, Codex, agents dirs)
- Platform-neutral output text ("assistant" instead of "Claude")
- Sandbox-friendly cache/output dirs with env var overrides and tempdir fallback
- Add Codex installation docs to README

Inspired by PR #24 (el-analista) and PR #5 (jblwilliams).
Zero impact on existing Claude Code behavior.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 23:18:53 -08:00
Matt Van Horn 9397fcc937 docs: Polish README for v2.1 launch + add launch copy with social posts
- Split V2.1 blurb into two clear bullets (YouTube + X search)
- Better X auth section with browser-specific notes and verify command
- Updated intro/tagline to mention YouTube as 4th source
- Added social post drafts and updated issue #19 response with YouTube

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 22:50:47 -08:00
Matt Van Horn c66ca7f43d feat: Add YouTube as 4th research source via yt-dlp
YouTube search and transcript extraction runs automatically when yt-dlp
is installed. Searches for topic videos from the last N days, fetches
auto-generated transcripts for top results, and feeds them through the
same scoring pipeline (relevance + recency + engagement) as Reddit/X.

New files:
- youtube_yt.py: search, transcript extraction, VTT cleanup

Modified files:
- schema.py: YouTubeItem dataclass, updated Report
- normalize.py: normalize_youtube_items()
- score.py: YouTube engagement scoring (views-dominated)
- dedupe.py: YouTube deduplication
- render.py: YouTube section in compact output
- env.py: is_ytdlp_available() check
- ui.py: YouTube progress messages
- last30days.py: _search_youtube(), parallel execution with Reddit/X
- SKILL.md: YouTube in stats box, citation priority
- README.md: YouTube docs, yt-dlp requirement, Peter shoutout

Inspired by Peter Steinberger's yt-dlp + summarize toolchain approach.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 21:38:04 -08:00
Matt Van Horn 31313c69ac v2.1: Bundle Bird X search - no external CLI needed
Vendor Bird's Twitter GraphQL search client directly into /last30days,
eliminating the dependency on `npm install -g @steipete/bird`. X search
now works out of the box with just Node.js 22+ and browser cookies.

- Add vendored bird-search.mjs wrapper (search-only subset of Bird v0.8.0)
- Vendor @steipete/sweet-cookie for browser cookie extraction
- Update bird_x.py to call vendored Node.js module instead of `bird` binary
- Update README.md and SKILL.md for v2.1 (remove Bird CLI install steps)
- Include Bird's MIT LICENSE in vendor directory

The fallback chain is: vendored search -> xAI API key -> web-only mode.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 17:16:04 -08:00
Matt Van Horn 0ec338c630 Add speed tradeoff note to README intro
Honest about 2-8 min research time vs V1's speed. Points users
to @mvanhorn / @slashlast30days for quick mode feedback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 11:01:54 -08:00
Matt Van Horn 205f8fa7ae Add timing disclaimer to parsed intent output
"Research typically takes 2-8 minutes (niche topics take longer)."
Shown right after parsed intent, before research starts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 11:00:06 -08:00
Matt Van Horn fdfe3c7e9e Add Vibe Motion as third README example showing brand-new tool research
Demonstrates /last30days researching a tool that launched 2 days ago —
before blog roundups existed. Real-time X/Reddit reactions from
@Hartdrawss, @rezkhere, @Totinhiiio plus honest Chase Jarvis review.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 10:57:35 -08:00
Matt Van Horn ff5cc1445c Add Kanye West as second README example showing current events mode
Real V2 output: 33 X posts (9.3K likes), @dekreationz viral post,
WSJ apology arc, BULLY album, Bianca Vanity Fair — demonstrates
/last30days as a news/culture research tool, not just prompting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 10:56:25 -08:00
Matt Van Horn 04b532582c Add Nano Banana Pro as first README example, remove old duplicate
New example uses fresh V2 research output showing JSON prompting
dominance, @handle citations, and the full research-to-prompt flow.
Removed the old iOS App Mockup example which used the same SwimMom
prompt/image but with stale V1 data.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 10:54:01 -08:00
Matt Van Horn 45702efd63 Update argument-hint with real example queries
Replace generic "[topic]" and --days=7 with concrete examples that
show the range: prompting, news, recommendations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 10:49:52 -08:00
Matt Van Horn 232a948576 Reframe README V2 around better search results as primary feature
Lead with dramatically better X/Reddit search quality (smarter query
construction + Phase 2 supplemental), Bird CLI as secondary headline,
community contributions in their own section.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 10:46:08 -08:00
Matt Van Horn 41779b81c0 Fix X search returning 0 results on popular topics
Three bugs in _extract_core_subject():

1. Multi-word noise phrases ("what are", "how to") never matched
   because code compared individual words against multi-word strings.
   "what are people saying about DeepSeek R1" became "what are people
   saying" — losing the entire topic.

2. Missing meta words — "prompt", "techniques", "tips" weren't
   filtered (only "prompting" was). "vibe motion best prompt
   techniques" kept 4 keywords instead of 2.

3. No retry on 0 results — Reddit retries with simplified queries
   but X accepted 0 and moved on.

Fix: Two-phase extraction (strip multi-word prefixes/suffixes first,
then individual noise words), expanded noise set, max 3 words (was 4),
and automatic retry with first 2 words when Bird returns 0 results.

Before → After:
- "vibe motion best prompt techniques" → "vibe motion" (was 4 words, 0 results)
- "what are people saying about DeepSeek R1" → "deepseek r1" (was "what are people saying")
- "nano banana pro prompts for gemini" → "nano banana pro" (was 4 words)

Tested: vibe motion (12 X posts, was 0), DeepSeek R1 (12 posts),
kanye west (12 posts, no regression).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 10:43:45 -08:00
Matt Van Horn 83f8d7ece5 Add V2 highlights to README intro, credit @galligan for plugin inspiration
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 10:28:54 -08:00
Matt Van Horn 52456e5820 Update README for V2 and bump USER_AGENT to 2.0
- Add V2 to title
- Document Bird CLI installation (free X search)
- Add --days=N to options table
- Update Requirements to show Bird CLI as recommended free option
- Replace How It Works with two-phase search architecture
- Add What's New in V2 section with all new features
- Credit community contributors (@JosephOIbrahim, @levineam, @jonthebeef)
- Bump http.py USER_AGENT from 1.0 to 2.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 10:26:53 -08:00
Matt Van Horn 6c620cbbf4 Smart invitations with research-specific example suggestions
Every invitation now includes 2-3 specific examples based on what the
research actually found. PROMPTING mode explicitly offers to write prompts
with creative examples. NEWS suggests follow-up questions about the
biggest stories. GENERAL suggests specific deep-dives and applications.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 10:11:30 -08:00
Matt Van Horn 88bca6e55c Fix invitation wording: add 'I'm now an expert' to all modes, remove prompt language from GENERAL
GENERAL mode said "write a prompt for it" which sounds weird for people/topics
("write a prompt for Kanye West"). Changed to "describe what you're working on
and I'll help." Added "I'm now an expert on {TOPIC}" prefix to all 4 modes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 10:04:10 -08:00
Matt Van Horn 1ae7a16c75 feat: Smart supplemental search — Phase 2 entity-aware drill-down
After the initial broad search (Phase 1), extract key entities from results
and run targeted secondary searches to surface content the broad pass missed:

- New entity_extract.py: parses @handles, #hashtags, subreddits from results
- bird_x.py: search_handles() does targeted from:handle searches via Bird CLI
- openai_reddit.py: search_subreddits() uses Reddit's free .json search endpoint
- last30days.py: Phase 2 orchestration runs after enrichment, merges + dedupes

Tested with "kanye west" (+9 Reddit, +1 X) and "claude code skills" (+6 Reddit, +1 X).
Phase 2 is skipped on --quick mode. Default caps at 3 handles/subs, deep at 5.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 10:00:28 -08:00
Matt Van Horn 650aa1100b Context-aware invitation, smart follow-ups, no raw URLs
- Invitation adapts to QUERY_TYPE: NEWS gets "ask me anything",
  PROMPTING gets "share your vision", RECOMMENDATIONS gets "compare options"
- Follow-up handler reads intent: questions get answers, not forced prompts
- Restored V1's "if they ask a question, answer it" context memory line
- URL formatting rule: "per Rolling Stone" not raw URLs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 09:34:30 -08:00
Matt Van Horn 6da4b2f5c7 Remove citation priority plan file
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 09:22:42 -08:00
Matt Van Horn 9da95019e6 Elevate Reddit/X citations over web sources in output
- Add citation priority rule: @handles > r/subreddits > web sources
- Fix examples: "per @honest30bgfan_" instead of "per Rolling Stone"
- Add "lead with people, not publications" instruction
- Stats box: web now shows "(supplementary)" instead of domain list
- No changes to actual research — same quality, better attribution framing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 09:22:07 -08:00
Matt Van Horn a953d57730 Add PR credits tracking for V2 launch
Track community PRs to thank contributors when V2 goes public.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 09:16:23 -08:00
Matt Van Horn 6d80af3807 Add Claude Code marketplace plugin support
- Add .claude-plugin/marketplace.json and plugin.json manifests
- Use ${CLAUDE_PLUGIN_ROOT:-$HOME/.claude/skills/last30days} for script path
- Plugin install: uses CLAUDE_PLUGIN_ROOT when set
- Manual install: falls back to ~/.claude/skills/last30days (no breakage)
- SKILL.md stays at root, plugin.json points skills to "./"

Users can now install via:
  claude plugin marketplace add mvanhorn/last30days-skill
  claude plugin install last30days@last30days

Inspired by PR #1 (galligan) — lighter approach without repo restructure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 09:11:28 -08:00
Matt Van Horn 60f06ea86e Merge community PRs: Windows Unicode fix, 403 fallback, --days flag
- PR #17 (JosephOIbrahim): Fix UnicodeEncodeError on Windows cp1252
- PR #16 (levineam): Handle HTTP 403 model access errors, add gpt-4.1 fallback
- PR #18 (jonthebeef): Add --days=N flag for configurable lookback (1-30)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 09:05:46 -08:00
Matt Van Horn c8ddb60397 Revert query parsing display to V1 format
User preferred V1's "I'll research X across Reddit, X, and the web..."
with explicit Parsed intent block over V2's emoji format.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 08:18:37 -08:00
Matt Van Horn 10f61c503d Restore 7 missing v1 features + add query parsing display + v1-vs-v2 test plan
Feature audit found 11 things v2 dropped from v1. Restored the 7 that affect
output quality: quality checklist for prompts, anti-pattern examples, self-check
instruction, "don't re-search" context memory, prompt format anti-pattern,
multi-prompt guidance, and emoji footer with engagement counts.

Also adds visible query parsing display (🔍 **{TOPIC}** · {QUERY_TYPE}) and
the full v1-vs-v2 comparison test plan with all 17 test queries from README.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 15:27:46 -08:00
Matt Van Horn 4b0098b754 Remove context: fork — root cause of broken skill execution
The old v1 skill worked because context: fork was SILENTLY IGNORED
due to Claude Code bug #17283. The skill ran inline in the main
conversation. Claude Code 2.1+ fixed the bug and now properly
honors fork mode, creating an isolated subagent that ignores all
instruction ordering (text output, bash-first, etc.).

Fix: remove context: fork so the skill runs inline again, matching
the behavior the user knows works. Also restored v1-style
instruction flow: parse intent first, then run script, then WebSearch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 10:28:58 -08:00
Matt Van Horn df29bbb53d Revert to bash-first execution with echo progress display
The agent in fork mode ignores text output instructions and jumps
straight to WebSearch. Reverted to the original "YOUR FIRST ACTION:
Run this command. EXECUTE." pattern that actually worked. Embedded
progress display as echo statements inside the bash command so they
can't be skipped. The agent was not running the Python script at all
when the first instruction was about text output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 10:18:53 -08:00
Matt Van Horn a6dd0a8b0e Move progress block to very first instruction in SKILL.md
The agent was skipping the progress text because it was buried after
parsing logic at line 41. Restructured so the FIRST section is
"YOUR FIRST OUTPUT" with the progress block, BEFORE any tool call
instructions. Added explicit "DO NOT skip this" enforcement.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 10:16:49 -08:00
Matt Van Horn c3120d29a3 Show visible progress block before research starts
The ui.py spinner/color output goes to stderr which Claude Code
buffers invisibly. Instead, have the agent output a visible progress
block with emoji showing Reddit/X/Web agents deploying.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 10:13:59 -08:00
Matt Van Horn 11f9ab8519 Restore acknowledgment step before research execution
Add back the brief topic acknowledgment ("Researching X — searching
Reddit for upvotes...") and descriptive intro that the v1 skill had.
The v2 version was jumping straight into script execution with no
user-facing feedback about what was being researched.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 10:10:49 -08:00
Matt Van Horn 7c36866524 Fix v2 output quality: stats format, Reddit results, citations, summary structure
- Stats: replace BAD/GOOD examples with strict fill-in-the-blank template
- Reddit: add subreddit-targeted fallback search, soften scoring penalties
  (engagement -10→-3, date confidence -10→-5), add minimum result guarantee
- Citations: limit to 1 per insight, short format, no engagement metrics
- Summary: add bold topic headers template for structured paragraphs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 10:04:46 -08:00
Matt Van Horn 38451d44c9 Restore original description, enforce emoji stats format, require source attribution
- Reverted description to original public repo copy
- Restored H1 heading, moved imperative instruction to bold paragraph
- Added BAD/GOOD examples enforcing emoji tree stats format
- Required @handle and r/subreddit citation in ALL summary types
- Removed agent:Explore and disable-model-invocation from frontmatter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 09:48:21 -08:00
Matt Van Horn 63de326df0 Fix skill not executing: remove agent:Explore, front-load run command
The Explore agent was treating SKILL.md as documentation to summarize
instead of instructions to follow. Removed agent:Explore from frontmatter
and restructured opening to immediately command script execution.

Also adds release plan for Bird CLI v2.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-06 09:38:32 -08:00
Matt Van Horn 4230fa2b69 Fix YAML parsing error in argument-hint field
Wrap value in single quotes to properly escape nested double quotes.
This fixes skills CLI installation.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-05 07:37:17 -08:00
Matt Van Horn 4e972d009c Rename skill from last30daystest to last30days
- Update skill name and description (remove TEST markers)
- Update script path to ~/.claude/skills/last30days/
- Clean up header, keep Bird CLI features prominent

Ready for local testing as the main skill.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 16:11:21 -08:00
Matt Van Horn 17485ec797 Make RECOMMENDATIONS format flexible (table or stacked)
- Remove over-prescriptive format requirements
- Let Claude choose table vs stacked naturally based on content
- Keep critical requirement: Sources line with @handles
- Claude Code's renderer handles the responsive wrapping

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 15:55:41 -08:00
Matt Van Horn 56f9eceb9c Switch RECOMMENDATIONS to stacked card format (responsive)
- Replace ASCII table with stacked card format
- Each card: Tool/Mentions/Use Case/Sources on separate lines
- Cards separated by horizontal rule (────)
- Doesn't break when terminal window is narrow
- Still includes @handles in Sources line

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 15:12:56 -08:00
Matt Van Horn c534c3b9a5 Update RECOMMENDATIONS format to use ASCII tables with @handles
- Changed from numbered list to ASCII box-drawing table format
- Added "Key Sources" column that MUST include @handles from X posts
- Added explicit instructions to parse and display highest-engagement handles
- Matches the output quality of the original xAI version

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 15:04:41 -08:00
Matt Van Horn 3afc1213b8 Add explicit instructions to calculate engagement totals
Claude was summarizing qualitatively ('mostly casual conversation')
instead of summing actual engagement numbers from the output.

Added CRITICAL note to:
- Parse [Xlikes, Yrt] from each X post
- Sum totals for the stats display
- Show top voices with engagement (e.g., @handle (15K likes))

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 14:47:41 -08:00
Matt Van Horn e7610ddcac Fix Bird search: strip noise words + fix date parsing
1. Add _extract_core_subject() to strip noise words before X search
   - X search is literal keyword matching, not semantic
   - "best open claw usecases" → "open claw" (27 results vs 0)

2. Fix ISO date detection bug
   - Was: if "T" in created_at (matched "Tue" in "Tue Feb 03...")
   - Now: if created_at[10] == "T" (proper ISO separator check)
   - Dates now parse correctly: (2026-02-03) instead of (date unknown)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 14:38:38 -08:00
Matt Van Horn f800b5eb1b Fix Bird CLI response parsing
1. last30days.py: Check isinstance(dict) before .get() - Bird returns
   list on success, dict on error

2. bird_x.py: Update field mappings for Bird's actual response format:
   - author.username not user.screen_name
   - createdAt not created_at (camelCase)
   - likeCount, retweetCount, etc. (camelCase)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 14:00:40 -08:00
Matt Van Horn 2c398ef6cb Enable API keys: remove clean mode
Now reads from ~/.config/last30days/.env like the regular skill.
Bird CLI + API keys both work together.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:47:10 -08:00
Matt Van Horn bcf5c2cc3f Fix: embed date filter in query instead of --since flag
Bird CLI doesn't support --since as a CLI flag.
Use X's search syntax: 'topic since:YYYY-MM-DD'

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:26:51 -08:00
Matt Van Horn 89bdbae0fe Fix: restore pretty emojis and box chars in stats summary
Original format:
 All agents reported back!
├─ 🟠 Reddit: {n} threads │ {sum} upvotes │ {sum} comments
├─ 🔵 X: {n} posts │ {sum} likes │ {sum} reposts
├─ 🌐 Web: {n} pages │ {domains}
└─ Top voices: r/{sub1}, r/{sub2} │ @{handle1}, @{handle2}

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 13:09:14 -08:00
Matt Van Horn 06061884f1 Update SKILL.md: remove interactive prompt references
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 12:39:16 -08:00
Matt Van Horn 976a6e2867 Match original NUX: non-blocking Bird promotion
- Remove interactive "Install Bird CLI?" prompt
- Add Bird as FREE option in promo messages
- Auto-detect Bird silently (use if authenticated)
- Keep original flow: show promo → continue with available sources

Bird now appears in the promo alongside API keys:
  🔵 X (Twitter)
     └─ FREE: npm install -g @steipete/bird (uses browser session)
     └─ Or: Add XAI_API_KEY (paid API)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 12:38:54 -08:00