fix(youtube): surface transcript-fetch ratio in footer + add degraded nudge

When yt-dlp is installed but stale (or otherwise unable to fetch transcripts for any returned videos), runs previously reported YouTube as fully successful in two user-facing surfaces: 1. Footer (render.py): showed "N videos | M views" with no indication that zero transcripts were captured. The "with transcripts" segment was conditionally suppressed when the count was zero - converting the canonical stale-binary failure mode into a silent absence at the very surface users read for "did this work?". 2. Quality nudge (quality_nudge.py): classified YouTube as "active" based purely on yt-dlp installation + absence of a top-level error. Per-video transcript-fetch ratio was never inspected. A run that returned N videos with 0 transcripts (canonical stale-binary failure) was reported as fully active. The engine itself logs the failure correctly at default stderr level (`[YouTube] Got transcripts for 0/N videos (N failed)`), but that line gets buried in 100+ lines of parallel-source progress output and is contradicted by the success-shaped footer and nudge that follow. This change makes both conclusion surfaces honest: * render.py footer always renders "M/N with transcripts" so the ratio is visible regardless of value. Zero is no longer hidden. Format is M/N (not bare M) so the denominator is in the message and the user does not have to cross-reference the "videos" count. * quality_nudge.py adds a third tier between "active" and "missing": "degraded". Triggered when yt-dlp is installed AND videos were returned AND transcript-fetch ratio is below threshold (default 50%, tunable via DEGRADED_TRANSCRIPT_THRESHOLD env var). Emits an actionable nudge: "YouTube returned N videos but only M transcripts captured. The most common cause is a stale yt-dlp binary - YouTube's caption format changes frequently and old binaries silently fail every transcript. Update via your package manager: scoop update yt-dlp (Windows), brew upgrade yt-dlp (macOS), or pip install -U yt-dlp." * last30days.py populates youtube_videos_count and youtube_transcripts_count in the research_results dict it passes to compute_quality_score, enabling the new degraded check at the call site. Threshold rationale: 50% accommodates a few legitimate caption-disabled videos in a multi-video result, but a stale-binary run that fails every transcript trips the nudge cleanly. Score impact: degradation is informational, not score-affecting. YouTube still counts as "active" in score_pct so users do not see their score drop for a fixable client-side issue. The nudge directs them to their own package manager. Tests: * tests/test_quality_nudge.py: 6 new TestYouTubeDegraded cases cover zero-transcripts-flags-degraded, partial-above-threshold-does-not-flag, zero-videos-does-not-flag (no false positives on absence), one-of-three-flags-degraded, threshold-tunable-via-config, and degraded-does-not-affect-score. * tests/test_render_v3.py: 4 new YoutubeFooterTranscriptRatioTests cases cover zero-transcripts-with-videos-renders-zero-over-total (the regression repro), partial-renders-ratio, full-renders-ratio, and no-videos-suppresses-entire-segment. All 29 new test cases verified GREEN with the fix and RED without it (temp-reverted both files separately to confirm each test catches the specific regression it asserts). Integration validation: ran the engine against an intentionally stale yt-dlp 2025.03.31 binary placed first on PATH. Pre-fix the footer showed `YouTube: 3 videos | 386,815 views` (no transcript signal). Post-fix the footer shows `YouTube: 3 videos | 386,815 views | 0/3 with transcripts` and stderr emits "Degraded: YouTube" plus the actionable update-yt-dlp nudge. Out of scope (deserves its own PR): exposing transcripts_captured in the EVIDENCE FOR SYNTHESIS block so the synthesizing model can flag degradation in prose. Larger schema-touching change.
2026-05-02 22:02:45 -04:00
parent 7214dd6051
commit 0a5102e193
5 changed files with 266 additions and 11 deletions
@@ -552,5 +552,90 @@ class DegradedRunBannerTests(unittest.TestCase):
        self.assertIn("--plan", text)


+class YoutubeFooterTranscriptRatioTests(unittest.TestCase):
+    """The YouTube footer line must surface the transcript-fetch ratio in all
+    cases where videos were returned. Pre-fix the segment was suppressed when
+    transcripts == 0, which converted the canonical stale-yt-dlp failure mode
+    into a silent absence at the footer (the very surface users read for
+    'did this work?'). Always-render the ratio so zero is loud.
+    """
+
+    def _build_youtube_report(self, transcript_flags: list[bool]) -> schema.Report:
+        """Build a Report with one YouTube item per entry in transcript_flags.
+        True means the item has transcript data; False means it does not.
+        """
+        items = []
+        for idx, has_transcript in enumerate(transcript_flags):
+            metadata = {"views": 1000}
+            if has_transcript:
+                metadata["transcript_highlights"] = ["Some pre-extracted quote."]
+            items.append(schema.SourceItem(
+                item_id=f"yt{idx}",
+                source="youtube",
+                title=f"Video {idx}",
+                body=f"Description for video {idx}.",
+                url=f"https://youtube.com/watch?v=v{idx}",
+                container="some-channel",
+                published_at="2026-04-15",
+                date_confidence="high",
+                engagement={"views": 1000, "likes": 100},
+                metadata=metadata,
+            ))
+        return schema.Report(
+            topic="test topic",
+            range_from="2026-04-01",
+            range_to="2026-05-01",
+            generated_at="2026-05-01T00:00:00+00:00",
+            provider_runtime=schema.ProviderRuntime(
+                reasoning_provider="gemini",
+                planner_model="gemini",
+                rerank_model="gemini",
+            ),
+            query_plan=schema.QueryPlan(
+                intent="general",
+                freshness_mode="balanced_recent",
+                cluster_mode="none",
+                raw_topic="test topic",
+                subqueries=[schema.SubQuery(
+                    label="primary", search_query="test topic",
+                    ranking_query="What about test topic?", sources=["youtube"],
+                )],
+                source_weights={"youtube": 1.0},
+            ),
+            clusters=[],
+            ranked_candidates=[],
+            items_by_source={"youtube": items},
+            errors_by_source={},
+        )
+
+    def test_zero_transcripts_with_videos_present_renders_zero_over_total(self):
+        # The canonical stale-yt-dlp case: 6 videos found, 0 transcripts captured.
+        # Pre-fix the footer hid this entirely; post-fix it must say "0/6 with transcripts".
+        report = self._build_youtube_report([False] * 6)
+        text = render.render_compact(report)
+        self.assertIn("0/6 with transcripts", text)
+
+    def test_partial_transcripts_renders_ratio(self):
+        # 5 of 6 transcripts captured - shows ratio so user knows one was missed.
+        report = self._build_youtube_report([True] * 5 + [False])
+        text = render.render_compact(report)
+        self.assertIn("5/6 with transcripts", text)
+
+    def test_full_transcripts_renders_ratio(self):
+        # All 3 transcripts captured - still shows ratio for consistency.
+        report = self._build_youtube_report([True] * 3)
+        text = render.render_compact(report)
+        self.assertIn("3/3 with transcripts", text)
+
+    def test_no_videos_no_transcript_segment(self):
+        # When YouTube has no items at all, the YouTube footer line is
+        # suppressed entirely (existing behavior) - the transcript segment
+        # should not appear without a parent line.
+        report = self._build_youtube_report([])
+        text = render.render_compact(report)
+        # No YouTube footer line at all - so no transcript segment either
+        self.assertNotIn("with transcripts", text)
+
+
 if __name__ == "__main__":
    unittest.main()