Trevin Chow
9f08bb68b5
Two related drifts surfaced while reviewing PR #399 (EXCLUDE_SOURCES) — docs claimed several SC-backed sources required INCLUDE_SOURCES opt-in that the code didn't actually enforce, and threads was inconsistently gated relative to its same-key siblings. This commit picks the "code as source of truth + EXCLUDE_SOURCES as suppression knob" model and aligns docs to match. It also promotes threads to the same auto-on tier as tiktok and instagram, since all three share the SC key and per-call cost shape — there was no real product reason for threads being opt-in while the other two weren't. The resulting source-gating model is three-tier and intentional: • **Auto-on if backing infra present** (suppress via EXCLUDE_SOURCES): reddit, HN, polymarket, X, youtube, github, bluesky, truthsocial, grounding, **tiktok, instagram, threads** • **INCLUDE_SOURCES persistent opt-in** (cost/billing reasons): perplexity (different paid API — OpenRouter), tiktok_comments / youtube_comments (N× extra SC calls per video) • **--search per-query opt-in** (relevance reasons): pinterest (visual pins, narrow utility), xiaohongshu (Chinese-market specific) Changes: - env.py: `is_threads_available()` drops the INCLUDE_SOURCES check, now mirrors tiktok/instagram (SC key → True). Docstring updated. - tests/test_env_v3.py: new `ThreadsAvailabilityTests` class locks in the new contract and includes a regression guard ("INCLUDE_SOURCES should not be needed"). - SKILL.md: lines 333-338 rewritten so the model's "Build ACTIVE_SOURCES_LIST" checklist reflects what the engine actually runs. Drops false INCLUDE_SOURCES requirement for tiktok/instagram/threads; corrects pinterest to mention --search; adds missing INCLUDE_SOURCES=perplexity requirement. - README: same alignment for the user-facing "Everything else in v3" section. Note on EXCLUDE_SOURCES references in the new docs: the suppression flag is wired up in PR #399. SKILL.md and README mention EXCLUDE_SOURCES as the opt-out path; that prose is forward-looking until #399 lands. The behavior changes in this PR (threads auto-on) are self-contained and don't require #399 to function — but for users who want to suppress the newly-auto-on threads source, #399 needs to land first.
67 lines
2.7 KiB
Python
67 lines
2.7 KiB
Python
import os
|
|
import sys
|
|
import unittest
|
|
from pathlib import Path
|
|
from unittest import mock
|
|
|
|
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "skills" / "last30days" / "scripts"))
|
|
|
|
from lib import bird_x, env
|
|
|
|
|
|
class EnvV3Tests(unittest.TestCase):
|
|
def setUp(self):
|
|
self._saved_credentials = dict(bird_x._credentials)
|
|
|
|
def tearDown(self):
|
|
bird_x._credentials.clear()
|
|
bird_x._credentials.update(self._saved_credentials)
|
|
|
|
def test_x_source_prefers_xai_without_bird_probe(self):
|
|
with mock.patch("lib.bird_x.is_bird_authenticated", side_effect=AssertionError("should not probe bird auth")):
|
|
source = env.get_x_source({"XAI_API_KEY": "test"})
|
|
self.assertEqual("xai", source)
|
|
|
|
def test_x_source_uses_bird_with_explicit_cookies(self):
|
|
with mock.patch("lib.bird_x.is_bird_installed", return_value=True):
|
|
source = env.get_x_source({"AUTH_TOKEN": "a", "CT0": "b"})
|
|
self.assertEqual("bird", source)
|
|
self.assertEqual("a", bird_x._credentials["AUTH_TOKEN"])
|
|
self.assertEqual("b", bird_x._credentials["CT0"])
|
|
|
|
def test_bird_auth_never_checks_browser_cookies(self):
|
|
# The guarantee: is_bird_authenticated() must not spawn any child
|
|
# process to probe for cookies. All subprocess paths in bird_x go
|
|
# through subproc.run_with_timeout, so patching that covers it.
|
|
with mock.patch("lib.bird_x.is_bird_installed", return_value=True), mock.patch(
|
|
"lib.bird_x.subproc.run_with_timeout",
|
|
side_effect=AssertionError("browser-cookie whoami should not run"),
|
|
):
|
|
bird_x._credentials.clear()
|
|
with mock.patch.dict(os.environ, {}, clear=False):
|
|
self.assertIsNone(bird_x.is_bird_authenticated())
|
|
|
|
|
|
class ThreadsAvailabilityTests(unittest.TestCase):
|
|
"""Threads is in the SC default-on family: same key, same per-call cost
|
|
shape as TikTok / Instagram, so the same default-on rule applies.
|
|
Suppression goes through EXCLUDE_SOURCES, not gated opt-in."""
|
|
|
|
def test_threads_available_with_sc_key_only(self):
|
|
self.assertTrue(env.is_threads_available({"SCRAPECREATORS_API_KEY": "k"}))
|
|
|
|
def test_threads_unavailable_without_sc_key(self):
|
|
self.assertFalse(env.is_threads_available({}))
|
|
self.assertFalse(env.is_threads_available({"INCLUDE_SOURCES": "threads"}))
|
|
|
|
def test_threads_does_not_require_include_sources(self):
|
|
"""Regression guard: INCLUDE_SOURCES should not be needed."""
|
|
self.assertTrue(env.is_threads_available({
|
|
"SCRAPECREATORS_API_KEY": "k",
|
|
"INCLUDE_SOURCES": "",
|
|
}))
|
|
|
|
|
|
if __name__ == "__main__":
|
|
unittest.main()
|