feat: configuration enablement — env-var defaults + source resilience
Six small additive changes that make the skill correctly understand its configured sources, plus tests + docs. User-visible benefits - LAST30DAYS_STORE=1 in .env turns persistence default-on without remembering --store on every invocation. Mirrors LAST30DAYS_DEBUG / LAST30DAYS_SKIP_PREFLIGHT convention. - SCRAPE_CREATORS_API_KEY (with underscore) accepted as alias for the canonical name. Matches the spelling used in the vendor's own example code (Adrian Horning's repo); saves the next user the same diagnostic rabbit hole. - Bluesky search now hits api.bsky.app (canonical AppView) instead of public.api.bsky.app (BunnyCDN-blocked public mirror as of 2026-05-04). BSKY_SEARCH_HOST env var lets users self-rescue future host migrations without a code release. Pre-fix: silent 0 Bluesky posts on every run. - App-password format validator emits a one-shot stderr warning when BSKY_APP_PASSWORD doesn't match xxxx-xxxx-xxxx-xxxx form. Detect-don't- gate: createSession still accepts main passwords; the warning helps users identify a hygiene issue without breaking existing setups. - Instagram retry on multi-token 500. SC's v2 reels endpoint wraps Google Search and 500's frequently on multi-word queries; a hashtag- form retry runs once before bubbling up. Documented vendor instability. - LAST30DAYS_TRANSCRIPT_TIMEOUT env var (default 30s, was hardcoded 15s). SC's transcript endpoint regularly takes >15s; the old default was clipping legitimate responses. - Silent-failure visibility: new bonus_errored field in the quality nudge fires when SC is configured but Instagram returned 0 items. Users see "Bonus source silent: Instagram" instead of unexplained absence. - YouTube degraded-ratio false-positive fixed. Captions-disabled videos can never produce a transcript regardless of yt-dlp version; they're now subtracted from the denominator so a single uploader-disabled video doesn't false-trigger the "stale yt-dlp" nudge. - urllib retry path: status_code attribute typo fix. The Instagram 500-retry was dead code on the urllib branch (getattr(e, 'status', ...) while http.HTTPError exposes status_code). Docs - README.md: added /plugin install last30days step after marketplace add in three places (the install was previously omitted in the docs). - CONFIGURATION.md: documented LAST30DAYS_STORE env var, added BSKY_SEARCH_HOST + app-password format section, mentioned LAST30DAYS_TRANSCRIPT_TIMEOUT in the Instagram source row. Test plan - 43 new unit tests across test_bluesky.py, test_instagram_sc.py, test_quality_nudge.py, test_youtube_yt.py - 141 total tests passing in target suite - Verified end-to-end: /last30days "Toronto resale condo market" with all 11+ sources active stored 35 new + 5 updated findings, all builder- PR-style accounts absent (organic agent voice in Instagram + TikTok results) Backward compatibility All changes are strictly additive. Optional kwargs default to None. New env vars are opt-in. Existing CLI flags untouched. Existing callers of public functions unaffected. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Daniel Zivkovic
committed by
Trevin Chow
Trevin Chow
parent
a8e462c978
commit
44971a6aae
@@ -293,3 +293,174 @@ class TestYouTubeDegraded:
|
||||
# But nudge still fires
|
||||
assert q["nudge_text"] is not None
|
||||
assert "Degraded: YouTube" in q["nudge_text"]
|
||||
|
||||
|
||||
class TestYouTubeCaptionsDisabledDoesNotFalseFlag:
|
||||
"""Captions-disabled videos must not lower the transcript-fetch ratio.
|
||||
|
||||
A video where the uploader disabled captions can never produce a transcript,
|
||||
no matter how fresh yt-dlp is. Counting it in the denominator of the
|
||||
degraded-ratio check produces false positives - one captions-disabled video
|
||||
in a small result set was triggering a "stale yt-dlp binary" nudge that was
|
||||
wrong. Fix: subtract captions_disabled from the denominator.
|
||||
"""
|
||||
|
||||
def test_zero_captions_disabled_preserves_existing_behavior(self):
|
||||
# Pre-existing case: 0 of 6 transcripts is still degraded (no captions
|
||||
# disabled to discount). Behavior is unchanged from TestYouTubeDegraded.
|
||||
q = _compute(
|
||||
ytdlp_installed=True,
|
||||
result_overrides={
|
||||
"youtube_videos_count": 6,
|
||||
"youtube_transcripts_count": 0,
|
||||
"youtube_captions_disabled_count": 0,
|
||||
},
|
||||
)
|
||||
assert "youtube" in q["core_degraded"]
|
||||
|
||||
def test_all_videos_captions_disabled_does_not_flag(self):
|
||||
# Every returned video had captions disabled by the uploader.
|
||||
# That's not a yt-dlp problem - it's an upstream content fact. Must not
|
||||
# flag degraded.
|
||||
q = _compute(
|
||||
ytdlp_installed=True,
|
||||
result_overrides={
|
||||
"youtube_videos_count": 3,
|
||||
"youtube_transcripts_count": 0,
|
||||
"youtube_captions_disabled_count": 3,
|
||||
},
|
||||
)
|
||||
assert "youtube" not in q["core_degraded"]
|
||||
|
||||
def test_mixed_uses_corrected_denominator(self):
|
||||
# 6 videos, 3 captions_disabled, 2 transcripts.
|
||||
# Naive (buggy) ratio: 2/6 = 33% (would flag).
|
||||
# Corrected ratio: 2/(6-3) = 67% (does NOT flag).
|
||||
# This case demonstrates the fix changes the verdict.
|
||||
q = _compute(
|
||||
ytdlp_installed=True,
|
||||
result_overrides={
|
||||
"youtube_videos_count": 6,
|
||||
"youtube_transcripts_count": 2,
|
||||
"youtube_captions_disabled_count": 3,
|
||||
},
|
||||
)
|
||||
assert "youtube" not in q["core_degraded"]
|
||||
|
||||
def test_mixed_still_flags_when_truly_degraded(self):
|
||||
# Even after discounting captions-disabled, the ratio is still bad.
|
||||
# 8 videos, 1 captions_disabled, 1 transcript -> 1/(8-1) = 14% (flags).
|
||||
q = _compute(
|
||||
ytdlp_installed=True,
|
||||
result_overrides={
|
||||
"youtube_videos_count": 8,
|
||||
"youtube_transcripts_count": 1,
|
||||
"youtube_captions_disabled_count": 1,
|
||||
},
|
||||
)
|
||||
assert "youtube" in q["core_degraded"]
|
||||
# Nudge should still mention the stale yt-dlp possibility but also
|
||||
# acknowledge that captions-disabled is a separate cause.
|
||||
assert q["nudge_text"] is not None
|
||||
assert "captions disabled" in q["nudge_text"].lower()
|
||||
|
||||
def test_missing_count_defaults_to_zero(self):
|
||||
# Older callers that don't pass the new key still work (default 0).
|
||||
q = _compute(
|
||||
ytdlp_installed=True,
|
||||
result_overrides={
|
||||
"youtube_videos_count": 6,
|
||||
"youtube_transcripts_count": 0,
|
||||
# youtube_captions_disabled_count intentionally omitted
|
||||
},
|
||||
)
|
||||
assert "youtube" in q["core_degraded"]
|
||||
|
||||
|
||||
class TestInstagramSilentFailure:
|
||||
"""Instagram is a `bonus` source via SC. Silent-failure detection: if SC
|
||||
is configured but the source returned zero items, surface a nudge so the
|
||||
user understands why the brief lacks an Instagram section.
|
||||
|
||||
Pre-fix the user got no signal - SC's /v2/instagram/reels/search 500s
|
||||
frequently on multi-token queries and the pipeline silently returned
|
||||
empty without any indication.
|
||||
"""
|
||||
|
||||
def test_zero_items_with_sc_flags_bonus_errored(self):
|
||||
q = _compute(
|
||||
config_overrides={
|
||||
"AUTH_TOKEN": "tok123",
|
||||
"SCRAPECREATORS_API_KEY": "sc_key",
|
||||
},
|
||||
ytdlp_installed=True,
|
||||
result_overrides={"instagram_items_count": 0},
|
||||
)
|
||||
assert "instagram" in q["bonus_errored"]
|
||||
assert q["nudge_text"] is not None
|
||||
assert "Instagram" in q["nudge_text"]
|
||||
|
||||
def test_zero_items_without_sc_does_not_flag(self):
|
||||
q = _compute(
|
||||
config_overrides={"AUTH_TOKEN": "tok123"},
|
||||
ytdlp_installed=True,
|
||||
result_overrides={"instagram_items_count": 0},
|
||||
)
|
||||
assert "instagram" not in q.get("bonus_errored", [])
|
||||
|
||||
def test_nonzero_items_does_not_flag(self):
|
||||
q = _compute(
|
||||
config_overrides={
|
||||
"AUTH_TOKEN": "tok123",
|
||||
"SCRAPECREATORS_API_KEY": "sc_key",
|
||||
},
|
||||
ytdlp_installed=True,
|
||||
result_overrides={"instagram_items_count": 5},
|
||||
)
|
||||
assert "instagram" not in q["bonus_errored"]
|
||||
assert q["nudge_text"] is None
|
||||
|
||||
def test_missing_key_means_source_did_not_run(self):
|
||||
q = _compute(
|
||||
config_overrides={
|
||||
"AUTH_TOKEN": "tok123",
|
||||
"SCRAPECREATORS_API_KEY": "sc_key",
|
||||
},
|
||||
ytdlp_installed=True,
|
||||
)
|
||||
assert "instagram" not in q["bonus_errored"]
|
||||
assert q["nudge_text"] is None
|
||||
|
||||
def test_nudge_text_explains_workaround(self):
|
||||
q = _compute(
|
||||
config_overrides={
|
||||
"AUTH_TOKEN": "tok123",
|
||||
"SCRAPECREATORS_API_KEY": "sc_key",
|
||||
},
|
||||
ytdlp_installed=True,
|
||||
result_overrides={"instagram_items_count": 0},
|
||||
)
|
||||
assert q["nudge_text"] is not None
|
||||
text_lower = q["nudge_text"].lower()
|
||||
assert "instagram" in text_lower
|
||||
assert ("0 reels" in text_lower or "silent" in text_lower
|
||||
or "hashtag" in text_lower)
|
||||
|
||||
def test_bonus_errored_does_not_affect_core_score(self):
|
||||
q = _compute(
|
||||
config_overrides={
|
||||
"AUTH_TOKEN": "tok123",
|
||||
"SCRAPECREATORS_API_KEY": "sc_key",
|
||||
},
|
||||
ytdlp_installed=True,
|
||||
result_overrides={"instagram_items_count": 0},
|
||||
)
|
||||
assert q["score_pct"] == 100
|
||||
assert "instagram" in q["bonus_errored"]
|
||||
assert q["nudge_text"] is not None
|
||||
assert "Bonus source silent" in q["nudge_text"]
|
||||
|
||||
def test_bonus_errored_field_always_present(self):
|
||||
q = _compute()
|
||||
assert q.get("bonus_errored") == []
|
||||
|
||||
|
||||
Reference in New Issue
Block a user