feat: surface YouTube + TikTok top comments alongside Reddit (#260)

* feat(normalize): pass YouTube top_comments through with Reddit-compatible shape

_normalize_youtube silently dropped top_comments after enrich_with_comments
populated them, so the downstream signals/render/entity layers never saw
YouTube comments. Map likes->score and text->excerpt so the existing
Reddit-compatible readers Just Work.

Shared _remap_comments helper will be reused for TikTok in a later commit.

* feat(tiktok): fetch top comments via ScrapeCreators when opted in

Mirrors the youtube_comments pattern: new env.is_tiktok_comments_available
gate (requires SCRAPECREATORS_API_KEY + tiktok_comments in INCLUDE_SOURCES),
tiktok.enrich_with_comments ranks posts and fetches via
GET /v1/tiktok/video/comments. Vote field is digg_count; text and user.nickname
come across verbatim. Pipeline calls the enricher right after TikTok search
when the gate is open.

Comment-fetch errors never crash the pipeline — the enricher returns an
empty list on 4xx/5xx.

* feat(normalize): pass TikTok top_comments through with digg_count->score mapping

Instagram uses the same shortform normalizer and has no comment fetcher
today, so the key is harmlessly absent there — no Instagram regression.

* feat(signals): add YouTube + TikTok top-comment score to engagement formula

Mirrors Reddit's 10% top-comment slot. Without top_comments present, the
formula reduces to views-dominant weighting; with a high-signal comment,
the item gets a meaningful bump (log1p(10k) ~ 9.2, weighted 0.10 = ~0.92
on the engagement score).

Updated the existing dominant-weight and missing-fields tests to the new
weights (0.45/0.32/0.13 for YT, 0.45/0.27/0.18 for TT). Views still dominate.

* feat(render): source-aware thresholds and vote labels for top comments

10 upvotes on Reddit signals community interest; 10 likes on a viral
TikTok is noise. Introduce per-source minimums (reddit 10, youtube 50,
tiktok 500) and native vote labels ('upvotes' for Reddit, 'likes' for
YT/TT). First-pass numbers — tune after live observation.

* docs: generalize top-comment quoting to YouTube + TikTok, add tiktok_comments opt-in

Synthesis instructions previously called out Reddit top comments only.
Now cover Reddit/YouTube/TikTok uniformly with source-appropriate vote
labels (upvotes vs likes), and explicitly frame YT transcript highlights
and comments as complementary signals. README and setup-wizard copy
document the new tiktok_comments INCLUDE_SOURCES token.

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>

scripts/lib/normalize.py

@@ -69,6 +69,47 @@ def normalize_source_items(
     return filtered
 
 
+def _remap_comments(
+    raw: list[Any],
+    score_keys: tuple[str, ...],
+    excerpt_keys: tuple[str, ...],
+) -> list[dict[str, Any]]:
+    """Normalize comments from any source into the shared Reddit-compatible shape.
+
+    Downstream code (signals._top_comment_score, render._top_comments_list,
+    entity_extract, rerank) all expect `score` and `excerpt`. This helper maps
+    per-source field names (YT: likes/text, TikTok: digg_count/text) onto that
+    shape while preserving author/date/url passthrough.
+    """
+    out: list[dict[str, Any]] = []
+    for raw_c in raw:
+        if not isinstance(raw_c, dict):
+            continue
+        score = _first_present(raw_c, score_keys, default=0)
+        excerpt = _first_present(raw_c, excerpt_keys, default="")
+        try:
+            score_int = int(score or 0)
+        except (TypeError, ValueError):
+            score_int = 0
+        entry: dict[str, Any] = {
+            "score": score_int,
+            "excerpt": str(excerpt or "")[:400],
+            "author": str(raw_c.get("author") or ""),
+            "date": str(raw_c.get("date") or ""),
+        }
+        if raw_c.get("url"):
+            entry["url"] = str(raw_c["url"])
+        out.append(entry)
+    return out
+
+
+def _first_present(d: dict[str, Any], keys: tuple[str, ...], default: Any) -> Any:
+    for key in keys:
+        if key in d and d[key] not in (None, ""):
+            return d[key]
+    return default
+
+
 def _domain_from_url(url: str) -> str | None:
     if not url:
         return None
@@ -200,6 +241,11 @@ def _normalize_youtube(
     metadata: dict[str, Any] = {}
     if highlights:
         metadata["transcript_highlights"] = highlights
+    metadata["top_comments"] = _remap_comments(
+        item.get("top_comments") or [],
+        score_keys=("score", "likes"),
+        excerpt_keys=("excerpt", "text"),
+    )
     return _source_item(
         item_id=str(item.get("video_id") or item.get("id") or f"YT{index + 1}"),
         source=source,
@@ -242,7 +288,16 @@ def _normalize_shortform_video(
         relevance_hint=item.get("relevance", 0.5),
         why_relevant=str(item.get("why_relevant") or ""),
         snippet=caption,
-        metadata={"hashtags": item.get("hashtags") or []},
+        metadata={
+            "hashtags": item.get("hashtags") or [],
+            "top_comments": _remap_comments(
+                item.get("top_comments") or [],
+                # TikTok uses digg_count as the vote field; Instagram has no
+                # comment fetcher today so the key is harmlessly absent.
+                score_keys=("score", "digg_count", "likes"),
+                excerpt_keys=("excerpt", "text"),
+            ),
+        },
     )