Commit Graph

9 Commits

Author SHA1 Message Date
Matt Van Horn 0a9ff16dfc feat: v3.0.0 - intelligent search, GitHub person/project mode, ELI5, 13+ sources
v3 rewrites the search engine from the ground up:

- Intelligent pre-research: resolves X handles, GitHub repos, subreddits,
  TikTok hashtags, and YouTube channels before searching
- GitHub person-mode: PR velocity, top repos by stars, release notes
- GitHub project-mode: live star counts, README, releases, top issues
- ELI5 mode: plain language synthesis, no jargon
- 13+ sources: Reddit, X, YouTube, TikTok, Instagram, HN, Polymarket,
  GitHub, Threads, Pinterest, Perplexity, Bluesky, Web
- Free Reddit comments via public JSON (no API key needed)
- Fun judge v2: humor scoring baked into narrative
- Cookie consent before browser scanning
- 10,000 free ScrapeCreators calls
- 1,012 tests

Thank you to the community contributors whose issues and PRs shaped v3:
@uppinote20 (#143), @zerone0x (#134, #136), @thinkun (#116),
@thomasmktong (#124), @fanispoulinakisai-boop (#100), @pejmanjohn (#78),
@zl190 (#115), @hnshah (#84, #85, #86)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:52:23 -07:00
Ilia Alshanetsky d7bff81757 fix(bird_x): pass .env credentials to Node subprocesses for WSL2/headless auth
* chore: fix YAML error in argument-hint

* add codex auth support to responses API

* Use gpt-5.1-codex-mini as default model for Codex auth

Add CODEX_FALLBACK_MODELS chain (gpt-5.1-codex-mini → gpt-5.2) for
Codex endpoint which doesn't support standard OpenAI models like
gpt-4o-mini. Adds model fallback retry on 400 errors in the Codex
search path. Also adds test_codex_auth.py with 22 unit tests covering
JWT decoding, auth resolution, SSE parsing, and payload building.

* Pass .env credentials to Bird Node subprocesses for X auth

On platforms without browser cookie access (e.g. WSL2), Bird's
vendored Node.js module cannot read AUTH_TOKEN/CT0 from Firefox
or Chrome cookie stores. The .env config file already supports
these values, but they were only loaded into the Python config
dict — never exported to the environment of Node subprocesses.

- Add AUTH_TOKEN/CT0 to env.py config key loading
- Add set_credentials()/\_subprocess_env() to bird_x.py to inject
  credentials into the env dict passed to subprocess.run/Popen
- Call set_credentials() in main() before Bird auth detection

---------

Co-authored-by: Justin Williams <jblwilliams@gmail.com>
2026-03-02 23:24:59 -08:00
Tjarko Leifer 451ebb3e22 Fix OpenAI 429 rate limiting with exponential backoff
The Reddit search uses OpenAI's Responses API with web_search, which
frequently returns 429 rate limit errors. The previous retry logic used
linear backoff (1s, 2s, 3s) which is too aggressive for OpenAI's rate
limiter (often needs 10-60s waits).

Changes:
- Increase max retries from 3 to 5
- Switch from linear to exponential backoff (2s, 5s, 9s, 17s, 33s)
- Parse and respect Retry-After header from OpenAI 429 responses
- Fall back to cheaper models (gpt-4.1 → gpt-4o) on 429s, not just
  on 400/403 access errors
- Remove gpt-4o-mini from fallback chain — it doesn't support
  web_search with the filters parameter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-20 12:40:36 +01:00
Matt Van Horn 20a859ecec fix: YouTube timeout bump to 90s + Reddit 429 fail-fast
YouTube: Add youtube_future timeout key (60/90/120s for quick/default/deep)
separate from the shared future timeout. YouTube needs more time because
it does search + parallel transcript fetching. Previously, 20 videos +
5 transcripts exceeded the 60s budget and all results were discarded.

Reddit 429: Propagate rate-limit errors instead of swallowing them.
Enrichment now uses 10s timeout / 1 retry (was 30s / 3 retries).
On first 429, cancel remaining enrichment and skip Phase 2 Reddit.
Total time wasted on 429 drops from ~75s to ~12s.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 00:48:26 -08:00
Matt Van Horn a09413608d feat: Add Codex CLI compatibility
- Add agents/openai.yaml for Codex skill discovery
- Make SKILL.md script path portable (repo, Claude, Codex, agents dirs)
- Platform-neutral output text ("assistant" instead of "Claude")
- Sandbox-friendly cache/output dirs with env var overrides and tempdir fallback
- Add Codex installation docs to README

Inspired by PR #24 (el-analista) and PR #5 (jblwilliams).
Zero impact on existing Claude Code behavior.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 23:18:53 -08:00
Matt Van Horn 52456e5820 Update README for V2 and bump USER_AGENT to 2.0
- Add V2 to title
- Document Bird CLI installation (free X search)
- Add --days=N to options table
- Update Requirements to show Bird CLI as recommended free option
- Replace How It Works with two-phase search architecture
- Add What's New in V2 section with all new features
- Credit community contributors (@JosephOIbrahim, @levineam, @jonthebeef)
- Bump http.py USER_AGENT from 1.0 to 2.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 10:26:53 -08:00
Matt Van Horn 3a4a727f4b Run Reddit and X searches in parallel, fix timeout handling
- Refactor run_research() to use ThreadPoolExecutor for parallel execution
- Reddit timeout/crash no longer blocks X search from running
- Add catch for ConnectionResetError/OSError in http.py
- Per-item error handling in Reddit enrichment (one failure doesn't crash all)
- Increase API timeouts from 60/90/120 to 90/120/180 seconds
- Add ClawdBot setup example to README

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 13:33:41 -08:00
Matt Van Horn 7b3150a69d Add --debug flag for verbose logging
- Add LAST30DAYS_DEBUG env var / --debug flag
- Log HTTP requests, responses, and errors
- Show API error details when debug enabled
- Helps diagnose API failures

Usage: python3 last30days.py "topic" --debug

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 16:54:31 -08:00
Matt Van Horn 5ca4829be4 Initial commit: last30days skill
Research topics across Reddit + X from the last 30 days using
OpenAI and xAI APIs. Features:
- Auto model selection (GPT-5.x, Grok-3)
- Popularity-aware scoring (relevance + recency + engagement)
- Reddit thread enrichment with real metrics
- Near-duplicate detection
- Multiple emit modes (compact, json, context, path)
- 24h caching with --refresh bypass
- NUX for API key setup
- 87 passing unit tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-23 12:37:31 -08:00