Adds a per-run denylist via the existing-but-unused EXCLUDE_SOURCES
config key. Two coupled changes:
1. pipeline.available_sources() filters out any source listed in
config["EXCLUDE_SOURCES"] (comma-separated, case-insensitive,
whitespace-tolerant) before returning.
2. hooks/scripts/check-config.sh "Ready — N sources active" banner
subtracts excluded sources from the ScrapeCreators +3 (Reddit
comments + TikTok + Instagram) so the count matches what the
pipeline actually runs.
Use case: skip TikTok/Instagram on runs where you only want
text-substantive sources, without unsetting SCRAPECREATORS_API_KEY
(which would also kill Reddit comments). The existing INCLUDE_SOURCES
allowlist covers Perplexity opt-in but doesn't cover this denylist case
— tiktok and instagram are added unconditionally when
SCRAPECREATORS_API_KEY is set, with no opt-out short of removing the key.
Tests (tests/test_pipeline_v3.py::TestExcludeSources):
- excludes tiktok+instagram when listed
- no exclusion when env unset or empty string
- case-insensitive + whitespace-tolerant parsing
- works for any source (e.g. EXCLUDE_SOURCES=hackernews), not just SC-backed
The prior pipeline.py only logged the planner outcome when an external
--plan was passed ("[Planner] Using external plan (N subqueries)").
The internal LLM planner and the deterministic fallback ran silently,
so retrieval-breadth failures were invisible without --debug.
After plan finalization, emit a unified trace:
[Planner] Plan: intent=X, freshness=Y, cluster_mode=Z, subqueries=N, source=external|llm|deterministic
[Planner] sq1 label=... search="..." sources=[...]
[Planner] sq2 ...
Stderr only; does not touch the user-facing stdout synthesis. The
source= annotation distinguishes --plan (external), provider-backed
(llm), and deterministic paths — so when the 2026-04-19 Hermes Agent
Use Cases failure mode recurs, the trace tells the user which path ran
and what subqueries it produced.
Tests: added test_planner_trace_always_fires_on_mock_run which captures
stderr on a mock pipeline run and asserts the summary + per-subquery
lines appear.
Tobi