Commit Graph
9 Commits
Author SHA1 Message Date
Matt Van HornandClaude Opus 4.6 49d45c2b42 feat(podcasts): add YouTube podcast source with transcript-first discovery
New "podcasts" source that discovers podcast content by scanning transcripts
from LLM-resolved YouTube channels. Finds content invisible to title-based
search — Acquired's "The NFL" episode mentions Taylor Swift 18x, ESPN 117x,
Netflix 102x, none in the title.

Architecture:
- LLM resolves 6-12 podcast channel @handles per topic
- Engine fetches recent episodes via yt-dlp (no video download)
- Downloads auto-captions and greps for topic keywords
- Episodes with 5+ mentions become podcast results with highlights
- Runs in parallel, ~15-20s latency, invisible in 3-min research run

Pipeline integration:
- New source module: scripts/lib/podcast_yt.py
- Registered in pipeline, normalizer, signals, planner, render
- CLI flag: --podcast-channels=AcquiredFM,lexfridman,...
- SOURCE_QUALITY: 0.88 (above YouTube's 0.85)
- Opt-in via INCLUDE_SOURCES=podcasts or --search=podcasts

Zero new API keys. Zero new dependencies. Reuses yt-dlp + transcript pipeline.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 00:28:40 -04:00
Matt Van Horn 565deb443e fix: rename plugin from last30days-3 to last30days 2026-04-08 11:12:06 -07:00
Matt Van HornandClaude Opus 4.6 0a9ff16dfc feat: v3.0.0 - intelligent search, GitHub person/project mode, ELI5, 13+ sources
v3 rewrites the search engine from the ground up:

- Intelligent pre-research: resolves X handles, GitHub repos, subreddits,
  TikTok hashtags, and YouTube channels before searching
- GitHub person-mode: PR velocity, top repos by stars, release notes
- GitHub project-mode: live star counts, README, releases, top issues
- ELI5 mode: plain language synthesis, no jargon
- 13+ sources: Reddit, X, YouTube, TikTok, Instagram, HN, Polymarket,
  GitHub, Threads, Pinterest, Perplexity, Bluesky, Web
- Free Reddit comments via public JSON (no API key needed)
- Fun judge v2: humor scoring baked into narrative
- Cookie consent before browser scanning
- 10,000 free ScrapeCreators calls
- 1,012 tests

Thank you to the community contributors whose issues and PRs shaped v3:
@uppinote20 (#143), @zerone0x (#134, #136), @thinkun (#116),
@thomasmktong (#124), @fanispoulinakisai-boop (#100), @pejmanjohn (#78),
@zl190 (#115), @hnshah (#84, #85, #86)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:52:23 -07:00
Matt Van HornandClaude Opus 4.6 4d6224f79a feat(youtube): extract transcript highlights like Reddit comment gems
Add extract_transcript_highlights() that scores sentences by specificity
(numbers, proper nouns, topic relevance) and filters YouTube filler
(subscribe, welcome back, etc). Top 5 highlights shown as structured
bullets in compact output. Full transcript moved to collapsible <details>
block so the LLM reads highlights first, full text on demand.

SKILL.md updated to instruct the judge agent to quote highlights
directly in synthesis, same as Reddit top comments.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 17:49:38 -07:00
Matt Van HornandClaude Opus 4.6 499074b564 fix(youtube): pass full transcripts to LLM instead of truncating to 200 chars
TRANSCRIPT_MAX_WORDS raised from 500 to 5000 so the LLM gets the full
content of most videos (up to ~25 minutes). Removed the second 200-char
truncation in render.py that was reducing transcripts to a single sentence
before the judge agent ever saw them.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-23 17:28:01 -07:00
Matt Van HornandClaude Opus 4.6 0c6358ab98 fix(bluesky): surface real error instead of misleading "auth failed"
When Cloudflare blocks requests to bsky.social or public.api.bsky.app
with a 403, the error was swallowed by a generic except clause and
reported as "Bluesky auth failed" - misleading users into thinking
their credentials were wrong.

Now _create_session() preserves the specific error in _session_error,
and search_bluesky() surfaces it. Cloudflare 403s get a clear message
about network-level blocks. Actual 401s say "Invalid credentials".

Closes #69

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 22:13:28 -07:00
Matt Van HornandClaude Opus 4.6 7ffa508bb4 feat(last30days): improve ClawHub discoverability - description, tags
Updated frontmatter description to be more search-friendly for ClawHub.
Added 12 new tags: deep-research, twitter, bluesky, recency, news,
citations, multi-source, social-media, analysis, web-search, ai-skill,
clawhub. Also added 11 GitHub repo topics.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 17:27:53 -07:00
Matt Van HornandClaude Opus 4.6 c2e18dbbeb fix(skill): use full /last30days name in comparison follow-up suggestions
The short alias /last30 only works on some platforms. Claude Code requires
the full /last30days name, so the follow-up suggestions after comparison
research were producing "Unknown skill: last30" errors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 07:55:17 -07:00
Matt Van Horn 52a22f5cb1 Merge pull request #65 from j-sperling/feat/search-quality-consolidation
Consolidate query/relevance modules and improve search quality
2026-03-14 07:31:24 -07:00