fix(youtube): surface transcript-fetch ratio in footer + add degraded nudge

When yt-dlp is installed but stale (or otherwise unable to fetch transcripts for any returned videos), runs previously reported YouTube as fully successful in two user-facing surfaces: 1. Footer (render.py): showed "N videos | M views" with no indication that zero transcripts were captured. The "with transcripts" segment was conditionally suppressed when the count was zero - converting the canonical stale-binary failure mode into a silent absence at the very surface users read for "did this work?". 2. Quality nudge (quality_nudge.py): classified YouTube as "active" based purely on yt-dlp installation + absence of a top-level error. Per-video transcript-fetch ratio was never inspected. A run that returned N videos with 0 transcripts (canonical stale-binary failure) was reported as fully active. The engine itself logs the failure correctly at default stderr level (`[YouTube] Got transcripts for 0/N videos (N failed)`), but that line gets buried in 100+ lines of parallel-source progress output and is contradicted by the success-shaped footer and nudge that follow. This change makes both conclusion surfaces honest: * render.py footer always renders "M/N with transcripts" so the ratio is visible regardless of value. Zero is no longer hidden. Format is M/N (not bare M) so the denominator is in the message and the user does not have to cross-reference the "videos" count. * quality_nudge.py adds a third tier between "active" and "missing": "degraded". Triggered when yt-dlp is installed AND videos were returned AND transcript-fetch ratio is below threshold (default 50%, tunable via DEGRADED_TRANSCRIPT_THRESHOLD env var). Emits an actionable nudge: "YouTube returned N videos but only M transcripts captured. The most common cause is a stale yt-dlp binary - YouTube's caption format changes frequently and old binaries silently fail every transcript. Update via your package manager: scoop update yt-dlp (Windows), brew upgrade yt-dlp (macOS), or pip install -U yt-dlp." * last30days.py populates youtube_videos_count and youtube_transcripts_count in the research_results dict it passes to compute_quality_score, enabling the new degraded check at the call site. Threshold rationale: 50% accommodates a few legitimate caption-disabled videos in a multi-video result, but a stale-binary run that fails every transcript trips the nudge cleanly. Score impact: degradation is informational, not score-affecting. YouTube still counts as "active" in score_pct so users do not see their score drop for a fixable client-side issue. The nudge directs them to their own package manager. Tests: * tests/test_quality_nudge.py: 6 new TestYouTubeDegraded cases cover zero-transcripts-flags-degraded, partial-above-threshold-does-not-flag, zero-videos-does-not-flag (no false positives on absence), one-of-three-flags-degraded, threshold-tunable-via-config, and degraded-does-not-affect-score. * tests/test_render_v3.py: 4 new YoutubeFooterTranscriptRatioTests cases cover zero-transcripts-with-videos-renders-zero-over-total (the regression repro), partial-renders-ratio, full-renders-ratio, and no-videos-suppresses-entire-segment. All 29 new test cases verified GREEN with the fix and RED without it (temp-reverted both files separately to confirm each test catches the specific regression it asserts). Integration validation: ran the engine against an intentionally stale yt-dlp 2025.03.31 binary placed first on PATH. Pre-fix the footer showed `YouTube: 3 videos | 386,815 views` (no transcript signal). Post-fix the footer shows `YouTube: 3 videos | 386,815 views | 0/3 with transcripts` and stderr emits "Degraded: YouTube" plus the actionable update-yt-dlp nudge. Out of scope (deserves its own PR): exposing transcripts_captured in the EVIDENCE FOR SYNTHESIS block so the synthesizing model can flag degradation in prose. Larger schema-touching change.
2026-05-02 22:02:45 -04:00
parent 7214dd6051
commit 0a5102e193
5 changed files with 266 additions and 11 deletions
@@ -199,3 +199,97 @@ class TestRedditNeverInCoreErrored:
        # Reddit is always-active in core (public path), error doesn't demote it
        assert "reddit" in q["core_active"]
        assert q["score_pct"] == 100
+
+
+class TestYouTubeDegraded:
+    """YouTube is `degraded` when videos returned but transcripts below threshold.
+
+    Canonical failure mode: a stale yt-dlp binary still finds videos via search
+    but silently fails every transcript fetch because YouTube's caption format
+    has moved on. Pre-fix the user got no signal of this; the footer hid zero,
+    and quality_nudge only checked top-level errors.
+    """
+
+    def test_zero_of_six_transcripts_flags_degraded(self):
+        q = _compute(
+            ytdlp_installed=True,
+            result_overrides={
+                "youtube_videos_count": 6,
+                "youtube_transcripts_count": 0,
+            },
+        )
+        assert "youtube" in q["core_degraded"]
+        assert q["nudge_text"] is not None
+        # Counts surface in the message so the user sees the actual ratio
+        assert "6 videos" in q["nudge_text"]
+        assert "0 transcripts" in q["nudge_text"]
+        assert "stale yt-dlp" in q["nudge_text"].lower()
+        # Updates path mentions all three common package managers
+        assert "scoop" in q["nudge_text"].lower()
+        assert "brew" in q["nudge_text"].lower()
+        assert "pip install" in q["nudge_text"].lower()
+
+    def test_five_of_six_transcripts_does_not_flag_degraded(self):
+        # 83% transcript success - well above the 50% threshold
+        # X is also enabled so all 5 cores are active and no nudge should fire
+        q = _compute(
+            config_overrides={"AUTH_TOKEN": "tok123"},
+            ytdlp_installed=True,
+            result_overrides={
+                "youtube_videos_count": 6,
+                "youtube_transcripts_count": 5,
+            },
+        )
+        assert "youtube" not in q["core_degraded"]
+        assert q["nudge_text"] is None  # All 5 core sources active, no degradation
+
+    def test_zero_videos_does_not_flag_degraded(self):
+        # No videos returned -> degraded check is meaningless and must not fire
+        q = _compute(
+            ytdlp_installed=True,
+            result_overrides={
+                "youtube_videos_count": 0,
+                "youtube_transcripts_count": 0,
+            },
+        )
+        assert "youtube" not in q["core_degraded"]
+
+    def test_one_of_three_transcripts_flags_degraded(self):
+        # 33% - below 50% threshold; the canonical "yt-dlp partially working" case
+        q = _compute(
+            ytdlp_installed=True,
+            result_overrides={
+                "youtube_videos_count": 3,
+                "youtube_transcripts_count": 1,
+            },
+        )
+        assert "youtube" in q["core_degraded"]
+        assert "Degraded: YouTube" in q["nudge_text"]
+
+    def test_threshold_tunable_via_config(self):
+        # Operator overrides threshold via env-style config to be more permissive
+        q = _compute(
+            config_overrides={"DEGRADED_TRANSCRIPT_THRESHOLD": "0.1"},
+            ytdlp_installed=True,
+            result_overrides={
+                "youtube_videos_count": 10,
+                "youtube_transcripts_count": 2,  # 20%, below default 50% but above override 10%
+            },
+        )
+        assert "youtube" not in q["core_degraded"]
+
+    def test_degraded_does_not_affect_score(self):
+        # Degradation is informational, not score-affecting; YouTube still counts as active
+        q = _compute(
+            config_overrides={"AUTH_TOKEN": "tok123"},
+            ytdlp_installed=True,
+            result_overrides={
+                "youtube_videos_count": 6,
+                "youtube_transcripts_count": 0,
+            },
+        )
+        assert "youtube" in q["core_active"]
+        assert q["score_pct"] == 100  # Full active count regardless of degradation
+        # But nudge still fires
+        assert q["nudge_text"] is not None
+        assert "Degraded: YouTube" in q["nudge_text"]