Address review feedback: deduplicate query_type, clean unused imports, fix defaults

- Remove duplicate detect_query_type from query.py (divergent 5-type version);
  canonical 7-type version lives in query_type.py
- Fix reddit.py import to use query_type.detect_query_type
- Clean unused STOPWORDS/SYNONYMS/tokenize imports from youtube_yt, instagram,
  tiktok, scrapecreators_x, bird_x after relevance consolidation
- Fix _relevance_filter default from 0.7 to 0.0 (items without relevance
  should not silently pass the filter)
- Remove --dateafter from yt-dlp (returns 0 results for evergreen topics)
- Remove restrictSearchableAttributes from HN search (misses Ask/Show HN)
- Lower HN points filter from >5 to >2 (avoids filtering niche posts)
- Add error logging to select_openai_model HTTP failures
- Remove mise.toml and internal planning doc from repo
- Update module docstrings to describe current purpose, not migration history
- Update tests to import from canonical relevance module
This commit is contained in:
Jeffrey Sperling
2026-03-11 18:40:07 -07:00
parent 6c402f66b7
commit 036bcd2ae3
18 changed files with 44 additions and 207 deletions
+2 -2
View File
@@ -1835,11 +1835,11 @@ def main():
"""Filter items below relevance threshold with minimum-result guarantee."""
if len(items) <= 3:
return items
passed = [i for i in items if getattr(i, 'relevance', 0.7) >= threshold]
passed = [i for i in items if getattr(i, 'relevance', 0.0) >= threshold]
if not passed:
# Keep top 3 by relevance if all filtered
print(f"[{source_name} WARNING] All results below relevance {threshold}, keeping top 3", file=sys.stderr)
by_rel = sorted(items, key=lambda x: getattr(x, 'relevance', 0.7), reverse=True)
by_rel = sorted(items, key=lambda x: getattr(x, 'relevance', 0.0), reverse=True)
return by_rel[:3]
return passed