After the initial broad search (Phase 1), extract key entities from results
and run targeted secondary searches to surface content the broad pass missed:
- New entity_extract.py: parses @handles, #hashtags, subreddits from results
- bird_x.py: search_handles() does targeted from:handle searches via Bird CLI
- openai_reddit.py: search_subreddits() uses Reddit's free .json search endpoint
- last30days.py: Phase 2 orchestration runs after enrichment, merges + dedupes
Tested with "kanye west" (+9 Reddit, +1 X) and "claude code skills" (+6 Reddit, +1 X).
Phase 2 is skipped on --quick mode. Default caps at 3 handles/subs, deep at 5.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Stats: replace BAD/GOOD examples with strict fill-in-the-blank template
- Reddit: add subreddit-targeted fallback search, soften scoring penalties
(engagement -10→-3, date confidence -10→-5), add minimum result guarantee
- Citations: limit to 1 per insight, short format, no engagement metrics
- Summary: add bold topic headers template for structured paragraphs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. last30days.py: Check isinstance(dict) before .get() - Bird returns
list on success, dict on error
2. bird_x.py: Update field mappings for Bird's actual response format:
- author.username not user.screen_name
- createdAt not created_at (camelCase)
- likeCount, retweetCount, etc. (camelCase)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove interactive "Install Bird CLI?" prompt
- Add Bird as FREE option in promo messages
- Auto-detect Bird silently (use if authenticated)
- Keep original flow: show promo → continue with available sources
Bird now appears in the promo alongside API keys:
🔵 X (Twitter)
└─ FREE: npm install -g @steipete/bird (uses browser session)
└─ Or: Add XAI_API_KEY (paid API)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move Bird CLI check to run before topic validation so users see
the install prompt even when running without a topic argument.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Move progress display initialization early to support Bird prompts
- Check Bird availability and offer install if no X source available
- Override available sources when Bird is ready
- Pass x_source parameter to run_research function
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add bird_x import and setup_bird_if_needed function to main script.
The function checks Bird status and offers installation if needed,
returning 'bird' if ready, 'declined' if user declined, or None if
unavailable.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Refactor run_research() to use ThreadPoolExecutor for parallel execution
- Reddit timeout/crash no longer blocks X search from running
- Add catch for ConnectionResetError/OSError in http.py
- Per-item error handling in Reddit enrichment (one failure doesn't crash all)
- Increase API timeouts from 60/90/120 to 90/120/180 seconds
- Add ClawdBot setup example to README
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
If initial search returns <5 threads, extract core subject and retry:
- "best nano banana prompting practices" → retry with "nano banana"
- Combines results from both searches, deduped by URL
Note: OpenAI's web_search still tends to find old content. This retry
helps cast a wider net but doesn't fully solve the recency issue.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Skill now works without any API keys using WebSearch fallback
- Shows promo banner marketing Reddit/X data when keys are missing
- Partial mode (one key) shows shorter tip for the missing source
- Updated SKILL.md to document three modes: Full, Partial, Web-Only
- Added get_missing_keys() to env.py for promo logic
- Added show_promo(), start_web_only(), show_web_only_complete() to ui.py
- Updated render_compact() to include inline promo for web-only mode
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously Reddit was returning ~60% old content (some from 2022).
This commit adds multiple layers of date enforcement:
- Reddit prompt: Explicit from_date/to_date with "fewer results > older results"
- Hard filter: filter_by_date_range() in normalize.py excludes old content
- WebSearch Date Detective: Extracts dates from URLs (/2026/01/24/) and
snippets ("January 24, 2026", "3 days ago")
- WebSearch scoring: +10 bonus for verified dates, -20 penalty for unknown
The skill now guarantees only content from the last 30 days.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add Claude's built-in WebSearch tool as a third research source for
/last30days. This enables the skill to work out of the box with zero
API keys while preserving Reddit/X as the primary sources.
Key changes:
- Add WebSearchItem schema for web results (no engagement metrics)
- Add score_websearch_items() with 55/45 relevance/recency weighting
- Apply -15pt source penalty so WebSearch ranks below Reddit/X
- Add --include-web CLI flag to opt-in to WebSearch
- Return 'web' mode when no API keys configured (zero-config)
- Update render.py with [WEB] source label formatting
When WebSearch is enabled, the script outputs instructions for Claude
to use its built-in WebSearch tool, then synthesize results together.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add LAST30DAYS_DEBUG env var / --debug flag
- Log HTTP requests, responses, and errors
- Show API error details when debug enabled
- Helps diagnose API failures
Usage: python3 last30days.py "topic" --debug
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Wrap xAI search in try/except like Reddit
- Show error message but continue with Reddit results
- Parse function checks for API errors before processing
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Show "⚡ CACHED RESULTS (Xh old)" in compact output header
- Add "use --refresh for fresh data" hint
- Track from_cache and cache_age_hours in Report schema
- Update UI to show cache age in stderr message
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- New ui.py module with colored output and animations
- Animated spinner during Reddit/X searches
- Progress tracking for enrichment phase [1/N]
- Fun random status messages per phase
- Mini ASCII banner at start
- Completion summary with timing
Makes the research feel more alive while waiting!
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The Report.to_dict() serializes range as {from, to} but constructor
expects range_from/range_to. Added from_dict() classmethod to properly
deserialize cached data, reconstructing all nested objects (Engagement,
Comment, SubScores, RedditItem, XItem).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- SKILL.md: Move "What I learned" BEFORE "Research Complete" stats
- Add error tracking to Report schema (reddit_error, x_error fields)
- Wrap OpenAI API calls in try/catch with clear error messages
- Show explicit error or "no results" messages in compact output
- Fix false positive error detection for null error fields
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- quick: 8-12 sources each, faster response
- default: 20-30 sources each (unchanged behavior)
- deep: 50-70 Reddit, 40-60 X for comprehensive research
Adjusts API timeouts based on depth. Cache keys include depth
so different depths are cached separately.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Research topics across Reddit + X from the last 30 days using
OpenAI and xAI APIs. Features:
- Auto model selection (GPT-5.x, Grok-3)
- Popularity-aware scoring (relevance + recency + engagement)
- Reddit thread enrichment with real metrics
- Near-duplicate detection
- Multiple emit modes (compact, json, context, path)
- 24h caching with --refresh bypass
- NUX for API key setup
- 87 passing unit tests
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Matt Van Horn