Commit Graph

5 Commits

Author SHA1 Message Date
Brad Ferguson edea402b7c fix(sources): unblock SC YouTube + multi-token HN searches
Two related fixes that surface when running last30days with multi-keyword
themed queries (e.g. "claude, personal agents, agentic infra"). Both bugs
caused entire sources to silently return zero items.

YouTube (ScrapeCreators)
  SC's /v1/youtube/search rejects ?keyword= with HTTP 400:
    {"error":"missing_parameter","message":"You must provide a query"}
  The canonical SC parameter for that endpoint is `query`. Other SC
  endpoints we use (Reddit, TikTok, Instagram) happened to work because
  they use their own per-endpoint parameter names — YouTube was the lone
  outlier.

Hacker News (Algolia)
  Multi-keyword theme queries returned zero hits across every theme.
  Algolia treats query= as strict AND across tokens, so a 4-5 word query
  like "claude, personal agents, agentic infra" matches no stories.

  Three changes in hackernews.py:

  1. Hoist comma/hyphen flattening into _flatten_query_for_algolia() so
     search_hackernews and _title_matches_query normalize the query the
     same way — addresses Greptile P2 #2 about the two callsites needing
     to stay in sync.
  2. Pass `optionalWords` for all-but-the-first token so Algolia ranks
     by token-overlap instead of requiring every token.
  3. Relax _title_matches_query from all-words to any-word, *but match
     on word boundaries (\b<word>\b) rather than naive substring* —
     addresses Greptile P2 #1, which flagged that the previous any-word
     relaxation would let "ai" falsely match "email" or "rail".

  Token-overlap relevance scoring at parse time already demotes weak
  matches, so word-boundary any-word matching is safe.

Tests: added coverage for no-token-in-title rejection, word-boundary vs
substring, and hyphen/comma flattening alignment between the search
parameter and the post-filter.

Co-authored-by: Trevin Chow <trevin@trevinchow.com>
2026-05-16 19:37:16 -07:00
Claire Novotny 72495c1c14 Restructure as Codex plugin 2026-04-23 20:15:02 -04:00
Matt Van Horn 0a9ff16dfc feat: v3.0.0 - intelligent search, GitHub person/project mode, ELI5, 13+ sources
v3 rewrites the search engine from the ground up:

- Intelligent pre-research: resolves X handles, GitHub repos, subreddits,
  TikTok hashtags, and YouTube channels before searching
- GitHub person-mode: PR velocity, top repos by stars, release notes
- GitHub project-mode: live star counts, README, releases, top issues
- ELI5 mode: plain language synthesis, no jargon
- 13+ sources: Reddit, X, YouTube, TikTok, Instagram, HN, Polymarket,
  GitHub, Threads, Pinterest, Perplexity, Bluesky, Web
- Free Reddit comments via public JSON (no API key needed)
- Fun judge v2: humor scoring baked into narrative
- Cookie consent before browser scanning
- 10,000 free ScrapeCreators calls
- 1,012 tests

Thank you to the community contributors whose issues and PRs shaped v3:
@uppinote20 (#143), @zerone0x (#134, #136), @thinkun (#116),
@thomasmktong (#124), @fanispoulinakisai-boop (#100), @pejmanjohn (#78),
@zl190 (#115), @hnshah (#84, #85, #86)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 10:52:23 -07:00
Matt Van Horn 7a9f447231 fix(ordering): move HN after YouTube in stats, sort priority, and SKILL.md
HN was appearing before YouTube in the stats block, sort tiebreaker,
and source status. Now consistently: Reddit > X > YouTube > HN > Web.
Also restored emoji + box-drawing chars in test skill SKILL.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 19:41:14 -08:00
Matt Van Horn 38a7ea253e feat(hackernews): add Hacker News as 5th research source
Add HN search via free Algolia API (no key needed). Two-phase approach:
search for stories, then enrich top ones with comments. Integrated into
the full pipeline (normalize, score, dedupe, render) running in parallel
with Reddit/X/YouTube. Source priority: Reddit > X > HN > YouTube > Web.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 18:33:31 -08:00