Add platform-specific query optimizations

- hackernews: use extract_core_subject instead of raw topic, add
  points>5 filter and restrictSearchableAttributes=title to reduce
  noise from URL-match and low-signal posts
- youtube: add --dateafter parameter to yt-dlp for server-side date
  filtering (Python soft filter still handles fallback)
- reddit: skip opinion/review query variant for how_to/comparison
  queries where it adds noise
- bird_x: add OR-group retry with compound terms before falling back
  to word-dropping (uses X OR operator for multi-concept queries)
- query.py: add detect_query_type() and extract_compound_terms()
This commit is contained in:
Jeffrey Sperling
2026-03-11 15:24:44 -07:00
parent c5be117701
commit 1002f1f020
6 changed files with 140 additions and 11 deletions
+4 -3
View File
@@ -100,15 +100,16 @@ def search_youtube(
_log(f"Searching YouTube for '{core_topic}' (since {from_date}, count={count})")
# yt-dlp search with full metadata (no --flat-playlist so dates are real).
# No --dateafter — we filter by date in Python with a soft fallback,
# because YouTube search returns relevance-sorted results and strict date
# filtering returns 0 for evergreen topics like "thumbnail tips".
# --dateafter helps yt-dlp filter server-side, but Python soft filter
# (below) handles the fallback for evergreen topics with 0 recent results.
dateafter = from_date.replace("-", "") # YYYYMMDD format for yt-dlp
cmd = [
"yt-dlp",
f"ytsearch{count}:{core_topic}",
"--dump-json",
"--no-warnings",
"--no-download",
"--dateafter", dateafter,
]
preexec = os.setsid if hasattr(os, 'setsid') else None