fix: per-entity Step 0.55, LAW 7 sub-run quiet, default 2, canonical SKILL.md (#311)
Four fixes based on 2026-04-22 test-window feedback on v3.0.11 --competitors:
- Each competitor sub-run now runs Step 0.55 (X handle / subreddits /
GitHub) via resolve.auto_resolve inside the fanout closure. Deep-copied
config per entity prevents _auto_resolve_context leak across sub-runs.
Resolved data stored on report.artifacts["resolved"] for the renderer.
- New internal_subrun keyword on planner.plan_query and pipeline.run
suppresses the LAW 7 "No --plan passed" stderr for engine-internal
fan-out only. Default path unchanged.
- Default --competitors count is now 2 (3-way total). --competitors=N
still customizes; range 1..6.
- SKILL.md STEP 0 canonical-path self-check forces readers who loaded
from marketplaces/ (auto-restored to origin/main, stale) to re-read
from plugins/cache/last30days-skill/last30days/{VERSION}/SKILL.md.
Two of three 2026-04-22 test windows hit this stale-path trap.
- New ## Resolved Entities block in render_comparison_multi shows
per-entity handles/subs/github for debug visibility.
Bumps plugin.json to 3.0.12. 12 new tests; 1,175 total passing.
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matt Van Horn
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "last30days",
|
||||
"version": "3.0.11",
|
||||
"version": "3.0.12",
|
||||
"description": "Research any topic across Reddit, X, YouTube, TikTok, Instagram, Hacker News, Polymarket, GitHub, and 5+ more sources. AI agent scores by upvotes, likes, and real money - not editors.",
|
||||
"author": {
|
||||
"name": "Matt Van Horn",
|
||||
|
||||
@@ -5,6 +5,22 @@ All notable changes to this project will be documented in this file.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [3.0.12] - 2026-04-22
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Per-entity Step 0.55 resolution for competitor sub-runs.** In 3.0.11, only the main topic got X handle / subreddit / GitHub resolution; competitor sub-runs ran with planner defaults and produced visibly thinner evidence (Reddit 403 fallbacks, single-word queries). Each competitor sub-run now calls `resolve.auto_resolve()` inside `fanout.run_competitor_fanout` when a web backend is available, mirroring the main topic's pre-flight resolution. Per-entity X handle, subreddit list, GitHub user/repos, and news context are threaded into each sub-run's `pipeline.run()` call. Deep-copied config per sub-run prevents `_auto_resolve_context` cross-leak. Surfaces in a new `## Resolved Entities` output block so the resolution coverage is visible without reading stderr.
|
||||
- **LAW 7 false-positive on internal fan-out sub-runs.** Each competitor sub-run was emitting the `[Planner] No --plan passed... YOU ARE the planner` stderr warning. LAW 7 targets the hosting-reasoning-model path, not engine-internal fan-out. New `internal_subrun=True` keyword on `planner.plan_query` and `pipeline.run` suppresses the warning for sub-runs only; the default path is unchanged.
|
||||
- **Marketplace-stale SKILL.md trap.** Added a STEP 0 canonical-path self-check at the top of SKILL.md. Two of three 2026-04-22 test runs loaded SKILL.md from `plugins/marketplaces/last30days-skill/` (Claude-Code-managed git clone pinned to origin/main, lagging the versioned cache), then ran `--help` against the same stale path, did not see `--competitors`, and fell back to a manual comparison plan. The STEP 0 block forces any reader to verify they loaded from `plugins/cache/last30days-skill/last30days/{VERSION}/SKILL.md` and re-read from the versioned cache if not.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Default `--competitors` count is now 2 (3-way total: original + 2 peers).** Previously 3. `--competitors=N` still customizes (range 1..6). Matches the feature description's canonical example (`Kanye vs Drake vs Kendrick`).
|
||||
|
||||
### Added
|
||||
|
||||
- **`## Resolved Entities` block** in `render_comparison_multi` output. Shows per-entity X handle, subreddits, GitHub user/repos, and truncated context for every entity in the comparison. Block is omitted entirely when no entity has a resolved payload (mock mode, no backend).
|
||||
|
||||
## [3.0.11] - 2026-04-22
|
||||
|
||||
### Added
|
||||
|
||||
@@ -116,7 +116,7 @@ When the same story appears on Reddit, X, and YouTube, v3 merges them into one c
|
||||
|
||||
### Auto-discovered competitor comparisons
|
||||
|
||||
`/last30days OpenAI --competitors` discovers the top 3 peers via web search (Anthropic, xAI, Google Gemini), runs the full pipeline on each in parallel, and returns one N-way comparison report. Override with `--competitors=N` or `--competitors-list="A,B,C"`.
|
||||
`/last30days OpenAI --competitors` discovers the top 2 peers via web search (Anthropic, xAI), runs the full pipeline on each in parallel, and returns one 3-way comparison report. Override with `--competitors=N` (range 1..6) or `--competitors-list="A,B,C"`.
|
||||
|
||||
### GitHub person-mode
|
||||
|
||||
|
||||
@@ -59,6 +59,26 @@ metadata:
|
||||
- clawhub
|
||||
---
|
||||
|
||||
# STEP 0: CANONICAL PATH SELF-CHECK — RUN BEFORE READING BELOW
|
||||
|
||||
Before reading anything else in this file, verify that the SKILL.md you just loaded is the versioned cache copy, not a stale marketplace clone:
|
||||
|
||||
```bash
|
||||
CANONICAL_ROOT="$HOME/.claude/plugins/cache/last30days-skill/last30days"
|
||||
CANONICAL_LATEST=$(ls -d "$CANONICAL_ROOT"/*/ 2>/dev/null | sort -V | tail -1)
|
||||
CANONICAL_LATEST="${CANONICAL_LATEST%/}"
|
||||
```
|
||||
|
||||
If the SKILL.md file you just read is NOT under `$CANONICAL_LATEST/`, STOP and re-read `$CANONICAL_LATEST/SKILL.md` before proceeding. Do not run `--help`, do not plan, do not invoke the engine against any other path.
|
||||
|
||||
**Why:** `~/.claude/plugins/marketplaces/last30days-skill/` is a git clone Claude Code auto-restores to `origin/main` on session start. It can lag the versioned cache by one or more releases. Three 2026-04-22 test runs (Linear, Coinbase) loaded SKILL.md from `marketplaces/`, ran `--help` from the same stale path, did not see the `--competitors` flag that existed in the cache, and fell back to a manual comparison plan. Result: 2 of 3 windows never invoked the feature they were asked to test.
|
||||
|
||||
**How to self-check:** the file path you used in your last Read tool call should match `$CANONICAL_LATEST/SKILL.md`. If it contains `marketplaces/` or any other prefix, that is the stale-path failure mode. Re-read from `$CANONICAL_LATEST/SKILL.md` and restart this contract from the top.
|
||||
|
||||
The same pinned resolver appears later in Step 1 for the engine Bash invocation. That guard is necessary but insufficient — by the time you reach Step 1, you may have already internalized an out-of-date flag list from the stale SKILL.md above it. This STEP 0 runs first so the CONTRACT itself is read from the right file.
|
||||
|
||||
---
|
||||
|
||||
# SKILL CONTRACT — READ BEFORE ANY TOOL CALL
|
||||
|
||||
You are inside the `/last30days` SKILL. This is a specific research tool with a 1400+ line instruction contract (the rest of this file) that defines EXACTLY how to produce the research output. It is not a generic "last 30 days of X" research prompt. Do NOT treat `/last30days` as a search keyword you can improvise against.
|
||||
@@ -575,10 +595,10 @@ Then do WebSearch for: `{TOPIC_A} vs {TOPIC_B} comparison {YEAR}` and `{TOPIC_A}
|
||||
|
||||
### Competitor mode (`--competitors`)
|
||||
|
||||
When the user passes `--competitors` on a single-entity topic, the engine auto-discovers 2-6 peer entities and fans out the full pipeline over the topic plus each competitor in parallel. Example: `last30days Kanye West --competitors` resolves to a 4-way comparison against Drake, Kendrick Lamar, and one other peer; `last30days OpenAI --competitors` resolves against Anthropic, xAI, and Google Gemini.
|
||||
When the user passes `--competitors` on a single-entity topic, the engine auto-discovers 1-6 peer entities and fans out the full pipeline over the topic plus each competitor in parallel. Example: `last30days Kanye West --competitors` resolves to a 3-way comparison against Drake and Kendrick Lamar; `last30days OpenAI --competitors=3` resolves against Anthropic, xAI, and Google Gemini.
|
||||
|
||||
**Flag surface:**
|
||||
- `--competitors` (bare) - discover and compare against 3 peers.
|
||||
- `--competitors` (bare) - discover and compare against 2 peers (3-way comparison: original + 2).
|
||||
- `--competitors=N` - discover N peers (range 1..6; out-of-range clamps with a stderr warning).
|
||||
- `--competitors-list="A,B,C"` - skip discovery and use the explicit list. Implies `--competitors`.
|
||||
|
||||
|
||||
@@ -0,0 +1,349 @@
|
||||
---
|
||||
title: "fix: per-entity resolution, default-2, and stale-path guard for --competitors"
|
||||
type: fix
|
||||
status: active
|
||||
date: 2026-04-22
|
||||
origin: docs/plans/2026-04-22-002-feat-competitors-flag-comparison-fanout-plan.md
|
||||
---
|
||||
|
||||
# fix: per-entity resolution, default-2, and stale-path guard for --competitors
|
||||
|
||||
## Overview
|
||||
|
||||
Three test runs of v3.0.11 `--competitors` surfaced four real bugs plus one product tweak. This plan fixes all of them in a single follow-up:
|
||||
|
||||
1. Competitor sub-runs get no Step 0.55 resolution (no X handle, no subreddits, no GitHub repo). Drake / Kendrick / Travis ran with deterministic-fallback single-word queries while Kanye had the full targeting package. User called it "lazy" and was right.
|
||||
2. Two of three test windows (Linear, Coinbase) never invoked the new flag at all. They loaded SKILL.md from `plugins/marketplaces/last30days-skill/` (a Claude-Code-managed git clone pinned to origin/main, which predates PR #308) instead of `plugins/cache/last30days-skill/last30days/3.0.11/`, so `--help` showed no `--competitors` flag and the model fell back to the manual comparison path.
|
||||
3. Each competitor sub-run emits a scary `[Planner] No --plan passed... deterministic fallback` stderr line because LAW 7 targets the hosting-model path, not internal fan-out sub-runs.
|
||||
4. Default competitor count is 3 (→ 4-way comparison). User wants default 2 (→ 3-way: original + 2 peers). Flag keeps `--competitors=N` to customize.
|
||||
|
||||
## Problem Frame
|
||||
|
||||
The 3 test runs (Kanye, Linear, Coinbase) showed a pattern:
|
||||
|
||||
| Window | Loaded SKILL.md from | Invoked --competitors? | Per-entity resolution? | Outcome |
|
||||
|--------|----------------------|-----------------------|------------------------|---------|
|
||||
| Kanye | cache/3.0.11/ (correct) | Yes | Only for main topic (Kanye) | Drake/Kendrick/Travis thin; Reddit 403 fallbacks |
|
||||
| Linear | marketplaces/ (stale) | No — fell back to manual comparison | No | Thin run with noisy subreddits |
|
||||
| Coinbase | marketplaces/ (stale) | No — fell back to manual comparison | Main only; keyword-search poisoned pool | Top subs: r/survivor, r/Airpodsmax (noise) |
|
||||
|
||||
Root causes:
|
||||
- **Per-entity resolution gap:** `scripts/lib/fanout.py` calls `pipeline.run()` with topic + depth + web_backend + lookback_days only. It does not call `resolve.auto_resolve()` per entity, so sub-runs have no X handle, subreddit, or GitHub targeting. The original plan (`2026-04-22-002`) acknowledged this as a deliberate v1 simplification ("competitor sub-runs use planner defaults"). In practice this produces visibly asymmetric output and triggers downstream retrieval issues (403 fallbacks, keyword-search noise).
|
||||
- **Stale-path loading:** Claude Code's skill loader alphabetizes `find` results with `marketplaces/` before `cache/`, and the model reads the first plausible SKILL.md it sees. SKILL.md line 823's `SKILL_ROOT` resolver is the correct path but only fires in engine-invocation blocks, not in the skill-load step.
|
||||
- **LAW 7 in sub-runs:** LAW 7 exists because the *hosting reasoning model* is supposed to pass `--plan`. For competitor sub-runs, there is no hosting-model planning — it's an engine-internal fan-out. The warning is a false positive there.
|
||||
|
||||
## Requirements Trace
|
||||
|
||||
- R1. Default `--competitors` count is 2 peers (3-way comparison: original + 2).
|
||||
- R2. Each competitor sub-run performs Step 0.55 resolution (X handle, subreddits, GitHub user/repos, news context) before its pipeline runs — not just the main topic.
|
||||
- R3. Sub-runs do not emit the LAW 7 `No --plan passed` warning; they are internal fan-out, not hosting-model calls.
|
||||
- R4. The rendered comparison output includes a visible "Resolved entities" block showing per-entity handles/subs/github for debug transparency (answers "did it resolve everyone?" without the user having to read stderr).
|
||||
- R5. SKILL.md has a canonical-path self-check at the top: if the reader loaded it from anywhere other than `plugins/cache/last30days-skill/last30days/{VERSION}/`, re-read from the versioned path before proceeding.
|
||||
- R6. Version bumps to 3.0.12; CHANGELOG entry; `scripts/sync.sh` deploys.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- No new discovery strategy. The web-search + regex extraction in `scripts/lib/competitors.py` stays as-is.
|
||||
- No new CLI flags beyond the behavior changes above. Specifically: no per-entity override flags like `--competitor-handles`. The hosting-model escape hatch remains `--competitors-list`.
|
||||
- No changes to the explicit `A vs B` comparison path (topic-string parsing in `planner._comparison_entities`).
|
||||
- No marketplace-clone auto-restore fix — that's Claude Code harness behavior. This plan only guards against the symptom on the skill side.
|
||||
|
||||
### Deferred to Separate Tasks
|
||||
|
||||
- Caching of per-entity resolution results: separate follow-up once hit rate justifies it.
|
||||
- Fan-out rate-limiting tuning (currently `max_workers=len(entities)+1`, capped at 6): defer until we see real-world quota exhaustion.
|
||||
- Pre-flight cost hint when N ≥ 4 (noted in `2026-04-22-002` risks): defer.
|
||||
|
||||
## Context & Research
|
||||
|
||||
### Relevant Code and Patterns
|
||||
|
||||
- `scripts/last30days.py:205-219` — `--competitors` / `--competitors-list` argparse definition (const=3 today; changing to 2).
|
||||
- `scripts/last30days.py:220-290` — `resolve_competitors_args()` validator; update `COMPETITORS_DEFAULT`.
|
||||
- `scripts/last30days.py:438-520` — main() fan-out orchestration; currently passes only topic/depth to each `_competitor_runner`.
|
||||
- `scripts/lib/fanout.py:40-95` — `run_competitor_fanout()` signature. The `competitor_runner` callable is where per-entity resolution needs to happen.
|
||||
- `scripts/lib/resolve.py:179-258` — `auto_resolve()` is the exact per-entity resolver to reuse. Already does X handle + subreddits + GitHub user/repos + news context in parallel via ThreadPoolExecutor.
|
||||
- `scripts/lib/planner.py:80-135` — `plan_query()` emits the LAW 7 stderr. A `quiet: bool` keyword or `internal_subrun: bool` flag will suppress it.
|
||||
- `scripts/lib/pipeline.py:162-220` — `pipeline.run()` signature. Needs a new keyword to propagate quiet-mode down to the planner.
|
||||
- `scripts/lib/render.py:render_comparison_multi` — where the "Resolved entities" block is inserted.
|
||||
- `SKILL.md` line 823 — canonical `SKILL_ROOT` resolver already exists but fires in engine bash, not at skill-load time.
|
||||
|
||||
### Institutional Learnings
|
||||
|
||||
- `docs/plans/2026-04-22-002-feat-competitors-flag-comparison-fanout-plan.md` acknowledged the per-entity-resolution gap as a v1 tradeoff. This plan closes that gap.
|
||||
- Kanye run stderr: `[Planner] No --plan passed... deterministic fallback` × 3 (once per competitor sub-run). That's the LAW 7 noise R3 targets.
|
||||
- Linear / Coinbase runs loaded `plugins/marketplaces/last30days-skill/CLAUDE.md` as the first hit. That's the stale-path issue R5 targets.
|
||||
|
||||
### External References
|
||||
|
||||
- None. All patterns are in-repo.
|
||||
|
||||
## Key Technical Decisions
|
||||
|
||||
- **Per-entity resolve happens inside fanout, not in SKILL.md.** The user-facing promise of `--competitors` is "one flag, engine does the work." Pushing resolution onto the hosting model creates another path-of-least-resistance trap (model skips it, output looks lazy). Auto-resolve inside each sub-run when a web backend is available makes the feature self-contained.
|
||||
- **Stale-path guard is a SKILL.md self-check, not a code change.** We cannot stop Claude Code from auto-restoring the marketplace clone. But we can put a 3-line banner at the top of SKILL.md that forces any path-mismatched read to re-read from the versioned cache. Both the marketplace copy (once main catches up) and the cache copy carry the guard.
|
||||
- **LAW 7 suppression is opt-in via `internal_subrun=True` keyword.** Do not remove the warning from the default path — it's load-bearing for the hosting-model contract. Add an explicit bypass for engine-internal fan-out only.
|
||||
- **Default 2, hard max 6 unchanged.** "Original + 2" matches the Kanye/Drake/Kendrick mental model from the feature description. Still allow `--competitors=N` from 1 to 6.
|
||||
- **Resolved block is inside the EVIDENCE envelope, not above it.** Keeps the rendered output structure stable for the synthesis contract (LAW 1–8). The block is context, not output.
|
||||
- **Skip auto-resolve when `--mock` or no web backend.** Mirrors the existing `resolve.auto_resolve()` fast-fail and keeps the mock test path deterministic.
|
||||
|
||||
## Open Questions
|
||||
|
||||
### Resolved During Planning
|
||||
|
||||
- **Where does per-entity resolve live?** Inside `fanout.run_competitor_fanout`, not in `main()`. Each sub-run calls `auto_resolve()` just before `pipeline.run()`.
|
||||
- **Should the hosting model still be able to override?** Yes — `--competitors-list` remains the escape hatch. When an explicit list is passed, the engine still does auto-resolve per entity; the user's list just skips discovery.
|
||||
- **Should sub-runs run auto-resolve in parallel with each other?** Yes. The existing `ThreadPoolExecutor` in fanout already parallelizes sub-runs; auto-resolve happens inside each sub-run's thread, so resolve calls for different entities run concurrently.
|
||||
- **Default count:** 2 peers (3-way). Confirmed.
|
||||
|
||||
### Deferred to Implementation
|
||||
|
||||
- Whether to expose a `--no-auto-resolve-competitors` flag for power users who want the fast, shallow behavior. Probably not needed v2; ship auto-resolve always-on and revisit if someone complains about cost.
|
||||
- Whether to surface the per-entity resolution context back into the main topic's planner (cross-entity context sharing). Stays deferred.
|
||||
- Whether the Resolved block should be collapsible or always inline. Start inline; revisit based on output length feedback.
|
||||
|
||||
## Implementation Units
|
||||
|
||||
- [ ] **Unit 1: Default `--competitors` to 2 peers**
|
||||
|
||||
**Goal:** Change the bare `--competitors` default from 3 to 2 per user feedback. `--competitors=N` still overrides; range 1..6 unchanged.
|
||||
|
||||
**Requirements:** R1
|
||||
|
||||
**Dependencies:** None
|
||||
|
||||
**Files:**
|
||||
- Modify: `scripts/last30days.py` (`COMPETITORS_DEFAULT`, `--competitors` const, stderr messages if any reference 3)
|
||||
- Modify: `SKILL.md` Competitor mode section ("discovered 2-6" wording, bare-flag default line)
|
||||
- Modify: `README.md` auto-discovered example line (if it references count)
|
||||
- Test: `tests/test_cli_competitors.py`
|
||||
|
||||
**Approach:**
|
||||
- Change `COMPETITORS_DEFAULT = 3` → `2` in `scripts/last30days.py`.
|
||||
- Change argparse `--competitors` `const=3` → `const=2`.
|
||||
- Update any SKILL.md / README copy referencing "3 peers" to "2 peers" (default) or "2-6 peers" (range).
|
||||
|
||||
**Patterns to follow:**
|
||||
- Existing default constants in `scripts/last30days.py` argparse block.
|
||||
|
||||
**Test scenarios:**
|
||||
- Happy path: bare `--competitors` yields count=2, enabled=True, empty explicit_list.
|
||||
- Edge case: `--competitors=3` still works (explicit override).
|
||||
- Edge case: existing `test_bare_flag_defaults_to_three` test is updated to `test_bare_flag_defaults_to_two` and asserts count=2.
|
||||
- Edge case: `--competitors=5` with a `--competitors-list` of length 2 still logs the mismatch warning and uses the list.
|
||||
|
||||
**Verification:**
|
||||
- `pytest tests/test_cli_competitors.py -v` passes with the updated default.
|
||||
|
||||
- [ ] **Unit 2: Per-entity Step 0.55 resolution inside fanout**
|
||||
|
||||
**Goal:** Each competitor sub-run auto-resolves its own X handle, subreddits, GitHub user/repos, and news context via `resolve.auto_resolve()` before its `pipeline.run()` call — just like the main topic.
|
||||
|
||||
**Requirements:** R2
|
||||
|
||||
**Dependencies:** None (but Unit 3 should land together so sub-runs don't emit LAW 7 stderr while the resolution context is being passed)
|
||||
|
||||
**Files:**
|
||||
- Modify: `scripts/lib/fanout.py`
|
||||
- Modify: `scripts/last30days.py` (`_competitor_runner` closure builds the resolved args)
|
||||
- Test: `tests/test_competitor_fanout.py`
|
||||
- Test: `tests/test_competitors_resolve_integration.py` (new; covers the auto-resolve path)
|
||||
|
||||
**Approach:**
|
||||
- `_competitor_runner(entity)` in main() does:
|
||||
1. Call `resolve.auto_resolve(entity, config)` when `not args.mock` and a web backend is configured (reuse `_has_backend`).
|
||||
2. Extract resolved x_handle, subreddits, github_user, github_repos, context.
|
||||
3. Pass them to `pipeline.run()` for that sub-run.
|
||||
4. Inject resolved context into a per-entity config copy (so `_auto_resolve_context` does not leak across sub-runs — deep-copy the config or use a local dict).
|
||||
5. Store the resolved block on the Report's `artifacts` so the renderer can surface it (Unit 4).
|
||||
- When `args.mock` is True or no backend is available, skip auto-resolve (fall through to planner defaults, matching the existing `auto_resolve()` early-return contract).
|
||||
- Update `fanout.run_competitor_fanout` docstring to note that auto-resolve happens inside the caller-provided runner.
|
||||
|
||||
**Execution note:** Start with a failing integration test that exercises two-entity fanout + auto-resolve via a mocked `resolve.auto_resolve` and asserts that `pipeline.run` receives the resolved x_handle/subreddits for each entity.
|
||||
|
||||
**Patterns to follow:**
|
||||
- `scripts/last30days.py` main topic branch (`if args.auto_resolve and not external_plan`) already calls `resolve.auto_resolve` and propagates results — mirror the shape for competitors.
|
||||
- Config isolation: `scripts/lib/pipeline.py:162-220` reads config as-is; use `dict(config)` to avoid cross-sub-run mutation of `_auto_resolve_context`.
|
||||
|
||||
**Test scenarios:**
|
||||
- Happy path: 3 entities, mocked `auto_resolve` returns distinct handles per entity; `pipeline.run` receives `x_handle=@drake` for Drake, `x_handle=@kendricklamar` for Kendrick, etc.
|
||||
- Happy path: the main topic still uses the user-supplied `--x-handle` / `--subreddits` overrides (not overwritten by auto-resolve for the main). Competitors use their own auto-resolved values.
|
||||
- Edge case: `--mock` skips auto-resolve entirely for all sub-runs (no `resolve.auto_resolve` calls).
|
||||
- Edge case: `resolve.auto_resolve` returns empty dicts for one entity (low-signal topic) — the sub-run still executes with planner defaults; doesn't crash.
|
||||
- Edge case: no web backend configured — auto-resolve returns empty for every entity, sub-runs fall through to planner defaults, no stack trace.
|
||||
- Error path: `resolve.auto_resolve` raises — the sub-run logs a warning and continues with planner defaults (does not fail the whole comparison).
|
||||
- Integration: config `_auto_resolve_context` from entity A does not leak into entity B's `pipeline.run`. Assert each sub-run gets its own context string.
|
||||
|
||||
**Verification:**
|
||||
- New integration test passes.
|
||||
- End-to-end smoke (mock mode + explicit list): each sub-run's stderr shows `[AutoResolve]` lines per entity with distinct values.
|
||||
|
||||
- [ ] **Unit 3: Suppress LAW 7 warning for engine-internal sub-runs**
|
||||
|
||||
**Goal:** The `[Planner] No --plan passed... deterministic fallback` warning does not fire during competitor sub-runs. LAW 7 is load-bearing for hosting-model contracts and must stay on the default path; this is an opt-in bypass for internal fan-out only.
|
||||
|
||||
**Requirements:** R3
|
||||
|
||||
**Dependencies:** Unit 2 (so the sub-run call site is already being modified)
|
||||
|
||||
**Files:**
|
||||
- Modify: `scripts/lib/planner.py` (`plan_query` signature + conditional stderr)
|
||||
- Modify: `scripts/lib/pipeline.py` (`run` signature + propagation)
|
||||
- Modify: `scripts/last30days.py` or `scripts/lib/fanout.py` (pass `internal_subrun=True` for competitor runners)
|
||||
- Test: `tests/test_planner_v3.py` (or new `tests/test_planner_quiet_mode.py`)
|
||||
- Test: `tests/test_competitor_fanout.py` (assert sub-runs don't emit LAW 7 stderr)
|
||||
|
||||
**Approach:**
|
||||
- Add a keyword `internal_subrun: bool = False` to `planner.plan_query`. When True, skip the two `print(..., file=sys.stderr)` blocks that emit the LAW 7 banner and the `[Planner] No --plan passed` capability message.
|
||||
- Add the same keyword to `pipeline.run()`; pass through to `plan_query`.
|
||||
- In main()/fanout, set `internal_subrun=True` for every competitor sub-run's pipeline.run call. The main topic's pipeline.run keeps the default (LAW 7 stays on for the hosting-model path).
|
||||
- Also suppress the LAW 7-triggered degraded-run warning block in the render layer for sub-reports when the envelope is going to be merged into a comparison output (or accept that the block is per-entity and surfaces once per entity).
|
||||
|
||||
**Patterns to follow:**
|
||||
- Existing keyword-only parameters on `pipeline.run` (`mock`, `x_handle`, etc.).
|
||||
- `planner.plan_query` signature is already keyword-only.
|
||||
|
||||
**Test scenarios:**
|
||||
- Happy path: `plan_query(..., internal_subrun=True, provider=None, model=None)` returns the deterministic fallback plan WITHOUT writing the LAW 7 stderr block.
|
||||
- Happy path: `plan_query(...)` with default `internal_subrun=False` still writes the LAW 7 warning (unchanged behavior).
|
||||
- Integration: end-to-end competitor fanout; assert captured stderr contains zero occurrences of `No --plan passed` and zero of `YOU ARE the planner`.
|
||||
- Integration: main topic is not part of competitor mode; if the user invokes bare `/last30days OpenAI` without `--plan`, LAW 7 stderr fires exactly once (regression test).
|
||||
|
||||
**Verification:**
|
||||
- Running the Kanye-style smoke test shows zero `[Planner] No --plan passed` lines for Drake / Kendrick / Travis sub-runs.
|
||||
|
||||
- [ ] **Unit 4: "Resolved entities" block in comparison output**
|
||||
|
||||
**Goal:** The rendered comparison output includes a visible block listing per-entity handles, subreddits, GitHub user, and resolved context. Answers "did it resolve everyone?" at a glance without reading stderr.
|
||||
|
||||
**Requirements:** R4
|
||||
|
||||
**Dependencies:** Unit 2 (needs resolved data on report artifacts)
|
||||
|
||||
**Files:**
|
||||
- Modify: `scripts/lib/render.py` (`render_comparison_multi` and `render_comparison_multi_context`)
|
||||
- Test: `tests/test_render_comparison_multi.py`
|
||||
|
||||
**Approach:**
|
||||
- When each entity's `Report.artifacts` contains a `resolved` dict (populated by Unit 2), `render_comparison_multi` emits a `## Resolved Entities` block early in the EVIDENCE envelope:
|
||||
```
|
||||
## Resolved Entities
|
||||
- **Kanye West**: X @kanyewest | Subs r/Kanye, r/hiphopheads | GitHub: — | Context: BULLY released, UK ban…
|
||||
- **Drake**: X @Drake | Subs r/DrakeTheType, r/hiphopheads | GitHub: — | Context: ICEMAN rollout…
|
||||
- **Kendrick Lamar**: X @kendricklamar | Subs r/KendrickLamar | GitHub: — | Context: Grammy wins, dormant…
|
||||
```
|
||||
- Missing fields render as `—` not empty.
|
||||
- When no entity has a `resolved` payload (mock mode, no web backend), omit the block entirely rather than emit an empty section.
|
||||
- Context strings are truncated at 120 chars to keep the block scannable.
|
||||
|
||||
**Patterns to follow:**
|
||||
- Existing `render_comparison_multi` envelope structure (lines ~395-480 in render.py).
|
||||
- Existing per-entity evidence block format (`## {label}`) for consistency.
|
||||
|
||||
**Test scenarios:**
|
||||
- Happy path: 3 entities each with a `resolved` artifact → block lists all 3 with their fields.
|
||||
- Happy path: 2 entities, one with full resolution, one with partial (x_handle only) → missing fields render as `—`.
|
||||
- Edge case: no entity has a resolved artifact → block is omitted entirely.
|
||||
- Edge case: context string > 120 chars → truncated with ellipsis.
|
||||
- Integration: rendered output passes through the same EVIDENCE envelope comments and synthesis contract (LAW 1–8 unchanged).
|
||||
|
||||
**Verification:**
|
||||
- Snapshot tests confirm the block appears in the right spot with the right formatting.
|
||||
- End-to-end smoke shows a realistic 3-entity Resolved block in the rendered output.
|
||||
|
||||
- [ ] **Unit 5: SKILL.md canonical-path self-check**
|
||||
|
||||
**Goal:** A top-of-file SKILL.md directive forces any reader (Claude Code, Codex, Hermes, Gemini) to verify they loaded from `plugins/cache/last30days-skill/last30days/{VERSION}/SKILL.md` before proceeding. If loaded from `marketplaces/` or any other path, re-read from the pinned versioned cache.
|
||||
|
||||
**Requirements:** R5
|
||||
|
||||
**Dependencies:** None
|
||||
|
||||
**Files:**
|
||||
- Modify: `SKILL.md` (prepend a STEP 0 block before the existing STEP 0 / LAW list)
|
||||
|
||||
**Approach:**
|
||||
- Add a numbered first step at the top (before or bundled with existing "STEP 0: ToolSearch preload"):
|
||||
```
|
||||
## STEP 0: Canonical Path Self-Check (must run first)
|
||||
|
||||
Before reading anything else below, verify you loaded this SKILL.md from
|
||||
the versioned cache, not the marketplace clone:
|
||||
|
||||
CANONICAL=$HOME/.claude/plugins/cache/last30days-skill/last30days/
|
||||
CANONICAL_LATEST=$(ls -d "$CANONICAL"*/ 2>/dev/null | sort -V | tail -1)
|
||||
|
||||
If the SKILL.md you just read is not under $CANONICAL_LATEST, STOP. Re-read
|
||||
$CANONICAL_LATEST/SKILL.md and restart from here. Marketplace clones
|
||||
(`plugins/marketplaces/last30days-skill/`) are pinned to origin/main and
|
||||
can be stale; the versioned cache is the ground truth.
|
||||
```
|
||||
- Reinforce in the existing LAW 7 block that `--help` output must be read from the same pinned `SKILL_ROOT` to avoid flag-list skew.
|
||||
|
||||
**Patterns to follow:**
|
||||
- Existing STEP 0 ToolSearch preload (top of SKILL.md) for tone / imperative voice.
|
||||
- Existing `SKILL_ROOT` resolver snippet (line ~823).
|
||||
|
||||
**Test scenarios:**
|
||||
- Test expectation: none — SKILL.md is documentation; no unit test, verified by follow-up user invocation.
|
||||
|
||||
**Verification:**
|
||||
- In a fresh Claude Code window, `/last30days Test --competitors` loads SKILL.md, the model executes the STEP 0 self-check, and (if it had loaded from marketplaces/) switches to the cache path before running `--help` or the engine. Observable via the model's announced reasoning / task list.
|
||||
|
||||
- [ ] **Unit 6: Version bump, CHANGELOG, sync**
|
||||
|
||||
**Goal:** Ship 3.0.12 and deploy to all local targets.
|
||||
|
||||
**Requirements:** R6
|
||||
|
||||
**Dependencies:** Units 1-5
|
||||
|
||||
**Files:**
|
||||
- Modify: `.claude-plugin/plugin.json` (version 3.0.11 → 3.0.12)
|
||||
- Modify: `CHANGELOG.md`
|
||||
- Run: `bash scripts/sync.sh`
|
||||
|
||||
**Approach:**
|
||||
- CHANGELOG entry under `## [3.0.12]` dated 2026-04-22 covering the four fixes (Fixed: per-entity resolution; Fixed: LAW 7 sub-run noise; Changed: default count 3→2; Added: Resolved entities block; Added: canonical-path self-check in SKILL.md).
|
||||
- `sync.sh` deploys to `~/.claude/plugins/cache/last30days-skill-private/...`, `~/.agents/`, `~/.codex/`, Hermes.
|
||||
- Manual hot-copy to `~/.claude/plugins/cache/last30days-skill/last30days/3.0.12/` so the public `/last30days` slash command picks up the new version before PR merge (matches the 3.0.11 testing pattern).
|
||||
|
||||
**Test scenarios:**
|
||||
- Test expectation: none — packaging only. Verification is by inspection.
|
||||
|
||||
**Verification:**
|
||||
- `grep version .claude-plugin/plugin.json` returns `3.0.12`.
|
||||
- `sync.sh` exits 0 with "Import check: OK" for each target.
|
||||
- Hot-copied 3.0.12 directory contains the new files and `/last30days` picks up the new version (highest-version resolver).
|
||||
|
||||
## System-Wide Impact
|
||||
|
||||
- **Interaction graph:** Fanout sub-runs now call `resolve.auto_resolve` per entity. Each sub-run is independent; no shared mutable state with other sub-runs or with the main topic.
|
||||
- **Error propagation:** `auto_resolve` failures inside a sub-run log a warning and degrade to planner defaults; do not propagate up to abort the comparison. Same contract as today for the main topic.
|
||||
- **State lifecycle risks:** Config dict is mutated by `auto_resolve` (via `config["_auto_resolve_context"]`). Must deep-copy per sub-run or scope context to a local mapping — otherwise two sub-runs' context strings race.
|
||||
- **API surface parity:** `pipeline.run` gains a keyword (`internal_subrun`); callers that don't pass it get the existing behavior. `planner.plan_query` gains the same. Backward compatible.
|
||||
- **Integration coverage:** New integration test for the fanout + auto-resolve + render chain. Existing snapshot tests update to include the Resolved block.
|
||||
- **Unchanged invariants:** Single-entity `/last30days` invocations (no `--competitors`) behave identically. Explicit `A vs B` comparison topics behave identically. LAW 7 still fires on the default hosting-model path. `render_compact` path is untouched.
|
||||
|
||||
## Risks & Dependencies
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Auto-resolving per competitor triples the WebSearch call volume (4 queries × 3 competitors = 12 extra web searches). | Fast-fail when no backend; user can pass `--competitors-list` to skip discovery but still get auto-resolve. Cost note in CHANGELOG. |
|
||||
| Config mutation across sub-runs via `_auto_resolve_context`. | Unit 2 deep-copies config per sub-run before each `auto_resolve` + `pipeline.run` call. Integration test asserts no cross-entity leak. |
|
||||
| LAW 7 suppression leaks onto the hosting-model path via a wrong default. | Default `internal_subrun=False`. Only fanout's competitor sub-runs set True. Unit test asserts bare-topic invocation still emits LAW 7. |
|
||||
| SKILL.md STEP 0 banner gets ignored by the model (same failure mode as line 823 today). | Put it in the guaranteed-read top band (before LAW 1, above all other content), imperative voice, concrete `STOP` verb. Still not bulletproof but strictly better than current. |
|
||||
| Default count change breaks assumptions in downstream tools or existing user muscle memory. | Changelog calls it out as Changed; `--competitors=3` still works for users who want the old default. |
|
||||
|
||||
## Documentation / Operational Notes
|
||||
|
||||
- Beta channel first: merge behind `/last30days-beta` via the private repo before cherry-picking to public. Follows the same process as 3.0.11.
|
||||
- Version 3.0.12 is a fix release; no marketing post required.
|
||||
- After merge, add a line to the PR description pointing at this plan.
|
||||
|
||||
## Sources & References
|
||||
|
||||
- Origin plan: `docs/plans/2026-04-22-002-feat-competitors-flag-comparison-fanout-plan.md`
|
||||
- Related PR: #308 (v3.0.11 shipping --competitors)
|
||||
- Test windows that surfaced the bugs: Kanye, Linear, Coinbase (2026-04-22 session)
|
||||
- Related code: `scripts/lib/fanout.py`, `scripts/lib/resolve.py` (`auto_resolve`), `scripts/lib/planner.py` (`plan_query`), `scripts/lib/render.py` (`render_comparison_multi`)
|
||||
+56
-6
@@ -230,11 +230,11 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
parser.add_argument(
|
||||
"--competitors",
|
||||
nargs="?",
|
||||
const=3,
|
||||
const=2,
|
||||
type=int,
|
||||
default=None,
|
||||
metavar="N",
|
||||
help="Auto-discover N competitor entities and fan out last30days across all of them as a comparison (default N=3, range 1..6). Use --competitors-list to override discovery.",
|
||||
help="Auto-discover N competitor entities and fan out last30days across all of them as a comparison (default N=2 → 3-way: original + 2 peers; range 1..6). Use --competitors-list to override discovery.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--competitors-list",
|
||||
@@ -246,7 +246,7 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
|
||||
COMPETITORS_MIN = 1
|
||||
COMPETITORS_MAX = 6
|
||||
COMPETITORS_DEFAULT = 3
|
||||
COMPETITORS_DEFAULT = 2
|
||||
|
||||
|
||||
def resolve_competitors_args(args: argparse.Namespace) -> tuple[bool, int, list[str]]:
|
||||
@@ -463,7 +463,7 @@ def main() -> int:
|
||||
comp_enabled, comp_count, comp_explicit = resolve_competitors_args(args)
|
||||
|
||||
def _main_runner() -> schema.Report:
|
||||
return pipeline.run(
|
||||
r = pipeline.run(
|
||||
topic=topic,
|
||||
config=config,
|
||||
depth=depth,
|
||||
@@ -481,6 +481,15 @@ def main() -> int:
|
||||
github_user=github_user,
|
||||
github_repos=github_repos,
|
||||
)
|
||||
r.artifacts["resolved"] = {
|
||||
"entity": topic,
|
||||
"x_handle": (args.x_handle or "").lstrip("@"),
|
||||
"subreddits": list(subreddits or []),
|
||||
"github_user": (github_user or ""),
|
||||
"github_repos": list(github_repos or []),
|
||||
"context": config.get("_auto_resolve_context", "") or "",
|
||||
}
|
||||
return r
|
||||
|
||||
if comp_enabled:
|
||||
from lib import competitors as competitors_mod
|
||||
@@ -517,15 +526,56 @@ def main() -> int:
|
||||
)
|
||||
|
||||
def _competitor_runner(entity: str) -> schema.Report:
|
||||
return pipeline.run(
|
||||
# Deep-copy config so per-entity auto_resolve context does not
|
||||
# leak across sub-runs. Each sub-run writes its own
|
||||
# `_auto_resolve_context` into its local config copy.
|
||||
entity_config = dict(config)
|
||||
resolved = {
|
||||
"entity": entity,
|
||||
"x_handle": "",
|
||||
"subreddits": [],
|
||||
"github_user": "",
|
||||
"github_repos": [],
|
||||
"context": "",
|
||||
}
|
||||
if not args.mock and resolve_mod._has_backend(entity_config):
|
||||
try:
|
||||
r = resolve_mod.auto_resolve(entity, entity_config)
|
||||
except Exception as exc:
|
||||
sys.stderr.write(
|
||||
f"[Competitors] auto_resolve failed for {entity!r}: "
|
||||
f"{type(exc).__name__}: {exc}\n"
|
||||
)
|
||||
r = {}
|
||||
resolved["x_handle"] = r.get("x_handle", "") or ""
|
||||
resolved["subreddits"] = list(r.get("subreddits") or [])
|
||||
resolved["github_user"] = r.get("github_user", "") or ""
|
||||
resolved["github_repos"] = list(r.get("github_repos") or [])
|
||||
resolved["context"] = r.get("context", "") or ""
|
||||
if resolved["context"]:
|
||||
entity_config["_auto_resolve_context"] = resolved["context"]
|
||||
sys.stderr.write(
|
||||
f"[Competitors] {entity}: "
|
||||
f"x=@{resolved['x_handle'] or '-'} "
|
||||
f"subs={len(resolved['subreddits'])} "
|
||||
f"gh={resolved['github_user'] or '-'}\n"
|
||||
)
|
||||
report = pipeline.run(
|
||||
topic=entity,
|
||||
config=config,
|
||||
config=entity_config,
|
||||
depth=depth,
|
||||
requested_sources=requested_sources,
|
||||
mock=args.mock,
|
||||
x_handle=resolved["x_handle"] or None,
|
||||
subreddits=resolved["subreddits"] or None,
|
||||
github_user=resolved["github_user"] or None,
|
||||
github_repos=resolved["github_repos"] or None,
|
||||
web_backend=args.web_backend,
|
||||
lookback_days=args.lookback_days,
|
||||
internal_subrun=True,
|
||||
)
|
||||
report.artifacts["resolved"] = resolved
|
||||
return report
|
||||
|
||||
entity_reports = fanout.run_competitor_fanout(
|
||||
main_topic=topic,
|
||||
|
||||
@@ -178,6 +178,7 @@ def run(
|
||||
lookback_days: int = 30,
|
||||
github_user: str | None = None,
|
||||
github_repos: list[str] | None = None,
|
||||
internal_subrun: bool = False,
|
||||
) -> schema.Report:
|
||||
settings = DEPTH_SETTINGS[depth]
|
||||
requested_sources = normalize_requested_sources(requested_sources)
|
||||
@@ -215,6 +216,7 @@ def run(
|
||||
provider=None if mock else reasoning_provider,
|
||||
model=None if mock else runtime.planner_model,
|
||||
context=config.get("_auto_resolve_context", ""),
|
||||
internal_subrun=internal_subrun,
|
||||
)
|
||||
# Source labelling: the fallback path annotates notes with "fallback-plan"
|
||||
# or "deterministic-comparison-plan"; anything else came from the LLM.
|
||||
|
||||
+19
-11
@@ -86,9 +86,16 @@ def plan_query(
|
||||
provider: providers.ReasoningClient | None,
|
||||
model: str | None,
|
||||
context: str = "",
|
||||
internal_subrun: bool = False,
|
||||
) -> schema.QueryPlan:
|
||||
"""Create a query plan. Comparison queries with extractable entities use a
|
||||
deterministic plan; other intents prefer the configured reasoning provider."""
|
||||
deterministic plan; other intents prefer the configured reasoning provider.
|
||||
|
||||
internal_subrun: when True, suppress the LAW 7 "No --plan passed" stderr
|
||||
warning. LAW 7 targets the hosting-reasoning-model path; competitor
|
||||
fan-out sub-runs are engine-internal and the warning is a false positive
|
||||
there. Default False preserves the warning on every user-facing invocation.
|
||||
"""
|
||||
if _should_force_deterministic_plan(topic):
|
||||
return _fallback_plan(
|
||||
topic,
|
||||
@@ -122,16 +129,17 @@ def plan_query(
|
||||
# planner credentials - NOT a prerequisite the caller needs. If you are
|
||||
# the hosting reasoning model, YOU are the provider. LAW 7 / 2026-04-19
|
||||
# Hermes Agent Use Cases failure mode.
|
||||
import sys
|
||||
print(
|
||||
"[Planner] No --plan passed. If you are the reasoning model hosting "
|
||||
"this skill (Claude Code, Codex, Hermes, Gemini, or any agent runtime), "
|
||||
"YOU ARE the planner: generate a JSON query plan yourself and pass it "
|
||||
"via --plan. You do not need an API key or credentials; you ARE the "
|
||||
"LLM. The deterministic fallback below is the headless/cron path only. "
|
||||
"See LAW 7 in SKILL.md and Step 0.75 for the plan schema.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
if not internal_subrun:
|
||||
import sys
|
||||
print(
|
||||
"[Planner] No --plan passed. If you are the reasoning model hosting "
|
||||
"this skill (Claude Code, Codex, Hermes, Gemini, or any agent runtime), "
|
||||
"YOU ARE the planner: generate a JSON query plan yourself and pass it "
|
||||
"via --plan. You do not need an API key or credentials; you ARE the "
|
||||
"LLM. The deterministic fallback below is the headless/cron path only. "
|
||||
"See LAW 7 in SKILL.md and Step 0.75 for the plan schema.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return _fallback_plan(topic, available_sources, requested_sources, depth)
|
||||
|
||||
|
||||
|
||||
@@ -445,6 +445,11 @@ def render_comparison_multi(
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
resolved_block = _render_resolved_entities_block(entity_reports)
|
||||
if resolved_block:
|
||||
lines.extend(resolved_block)
|
||||
lines.append("")
|
||||
|
||||
fun_params = _FUN_LEVELS.get(fun_level, _FUN_LEVELS["medium"])
|
||||
for label, report in entity_reports:
|
||||
lines.extend(_render_entity_evidence_block(
|
||||
@@ -475,6 +480,52 @@ def render_comparison_multi(
|
||||
return "\n".join(lines).strip() + "\n"
|
||||
|
||||
|
||||
def _render_resolved_entities_block(
|
||||
entity_reports: list[tuple[str, schema.Report]],
|
||||
) -> list[str]:
|
||||
"""Emit a visible per-entity Step 0.55 resolution summary.
|
||||
|
||||
Reads `resolved` dicts from each Report's artifacts. Returns an empty
|
||||
list when no entity has a resolved payload (mock mode, no web backend,
|
||||
or artifacts not populated). Missing per-entity fields render as `-`.
|
||||
Context strings truncate at 120 chars.
|
||||
"""
|
||||
any_resolved = any(
|
||||
isinstance(report.artifacts.get("resolved"), dict)
|
||||
for _label, report in entity_reports
|
||||
)
|
||||
if not any_resolved:
|
||||
return []
|
||||
|
||||
out: list[str] = ["## Resolved Entities", ""]
|
||||
for label, report in entity_reports:
|
||||
resolved = report.artifacts.get("resolved") or {}
|
||||
x_handle = resolved.get("x_handle") or ""
|
||||
subs = resolved.get("subreddits") or []
|
||||
gh_user = resolved.get("github_user") or ""
|
||||
gh_repos = resolved.get("github_repos") or []
|
||||
context = resolved.get("context") or ""
|
||||
|
||||
x_display = f"@{x_handle}" if x_handle else "-"
|
||||
subs_display = (
|
||||
", ".join(f"r/{s}" for s in subs[:5]) + (
|
||||
f" (+{len(subs) - 5})" if len(subs) > 5 else ""
|
||||
)
|
||||
) if subs else "-"
|
||||
gh_display = f"@{gh_user}" if gh_user else "-"
|
||||
if gh_repos:
|
||||
gh_display += f" ({', '.join(gh_repos[:3])}" + (
|
||||
f" +{len(gh_repos) - 3}" if len(gh_repos) > 3 else ""
|
||||
) + ")"
|
||||
context_display = _truncate(context, 120) if context else "-"
|
||||
|
||||
out.append(
|
||||
f"- **{label}**: X {x_display} | Subs {subs_display} | "
|
||||
f"GitHub {gh_display} | Context: {context_display}"
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def _render_entity_evidence_block(
|
||||
*,
|
||||
label: str,
|
||||
@@ -536,6 +587,10 @@ def render_comparison_multi_context(
|
||||
_AI_SAFETY_NOTE,
|
||||
"",
|
||||
]
|
||||
resolved_block = _render_resolved_entities_block(entity_reports)
|
||||
if resolved_block:
|
||||
lines.extend(resolved_block)
|
||||
lines.append("")
|
||||
for label, report in entity_reports:
|
||||
lines.append(f"## {label}")
|
||||
lines.append(f"Intent: {report.query_plan.intent}")
|
||||
|
||||
@@ -29,13 +29,19 @@ class CompetitorsCliTests(unittest.TestCase):
|
||||
self.assertEqual(count, 0)
|
||||
self.assertEqual(explicit, [])
|
||||
|
||||
def test_bare_flag_defaults_to_three(self):
|
||||
def test_bare_flag_defaults_to_two(self):
|
||||
args = _parse("Kanye West", "--competitors")
|
||||
enabled, count, explicit = cli.resolve_competitors_args(args)
|
||||
self.assertTrue(enabled)
|
||||
self.assertEqual(count, 3)
|
||||
self.assertEqual(count, 2)
|
||||
self.assertEqual(explicit, [])
|
||||
|
||||
def test_explicit_three_still_supported(self):
|
||||
args = _parse("OpenAI", "--competitors", "3")
|
||||
enabled, count, _explicit = cli.resolve_competitors_args(args)
|
||||
self.assertTrue(enabled)
|
||||
self.assertEqual(count, 3)
|
||||
|
||||
def test_explicit_count(self):
|
||||
args = _parse("OpenAI", "--competitors", "4")
|
||||
enabled, count, explicit = cli.resolve_competitors_args(args)
|
||||
|
||||
@@ -0,0 +1,331 @@
|
||||
# ruff: noqa: E402
|
||||
"""Integration tests for per-entity Step 0.55 resolution inside competitor fan-out."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import sys
|
||||
import unittest
|
||||
from contextlib import redirect_stderr
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(REPO_ROOT / "scripts"))
|
||||
|
||||
|
||||
def _fake_report(topic: str):
|
||||
"""Minimal Report stand-in for runner return values."""
|
||||
class _R:
|
||||
pass
|
||||
|
||||
r = _R()
|
||||
r.topic = topic
|
||||
r.artifacts = {}
|
||||
return r
|
||||
|
||||
|
||||
def _build_main_args(*overrides):
|
||||
"""Minimal argparse.Namespace-like object for the competitor path."""
|
||||
import argparse
|
||||
ns = argparse.Namespace(
|
||||
topic=["Kanye West"],
|
||||
mock=False,
|
||||
competitors=2,
|
||||
competitors_list=None,
|
||||
quick=False,
|
||||
deep=False,
|
||||
emit="compact",
|
||||
search=None,
|
||||
debug=False,
|
||||
diagnose=False,
|
||||
save_dir=None,
|
||||
save_suffix=None,
|
||||
store=False,
|
||||
x_handle=None,
|
||||
x_related=None,
|
||||
web_backend="auto",
|
||||
deep_research=False,
|
||||
plan=None,
|
||||
subreddits=None,
|
||||
tiktok_hashtags=None,
|
||||
tiktok_creators=None,
|
||||
ig_creators=None,
|
||||
lookback_days=30,
|
||||
auto_resolve=False,
|
||||
github_user=None,
|
||||
github_repo=None,
|
||||
)
|
||||
return ns
|
||||
|
||||
|
||||
class PerEntityResolveTests(unittest.TestCase):
|
||||
"""Verify each competitor sub-run calls auto_resolve with its own topic and
|
||||
that the resolved fields are threaded into pipeline.run."""
|
||||
|
||||
def test_auto_resolve_called_per_competitor(self):
|
||||
from lib import resolve as resolve_mod
|
||||
from lib import pipeline as pipeline_mod
|
||||
|
||||
config = {"BRAVE_API_KEY": "test-key"}
|
||||
|
||||
captured_resolve_topics: list[str] = []
|
||||
captured_pipeline_kwargs: list[dict] = []
|
||||
|
||||
def fake_resolve(topic, _cfg):
|
||||
captured_resolve_topics.append(topic)
|
||||
per_topic = {
|
||||
"Drake": {
|
||||
"x_handle": "Drake",
|
||||
"subreddits": ["DrakeTheType", "hiphopheads"],
|
||||
"github_user": "",
|
||||
"github_repos": [],
|
||||
"context": "Drake ICEMAN rollout",
|
||||
"category": None,
|
||||
"searches_run": 4,
|
||||
},
|
||||
"Kendrick Lamar": {
|
||||
"x_handle": "kendricklamar",
|
||||
"subreddits": ["KendrickLamar", "hiphopheads"],
|
||||
"github_user": "",
|
||||
"github_repos": [],
|
||||
"context": "Meet The Grahams revival",
|
||||
"category": None,
|
||||
"searches_run": 4,
|
||||
},
|
||||
}
|
||||
return per_topic.get(topic, {
|
||||
"x_handle": "", "subreddits": [], "github_user": "",
|
||||
"github_repos": [], "context": "",
|
||||
"category": None, "searches_run": 0,
|
||||
})
|
||||
|
||||
def fake_pipeline_run(**kwargs):
|
||||
captured_pipeline_kwargs.append(kwargs)
|
||||
return _fake_report(kwargs["topic"])
|
||||
|
||||
with mock.patch.object(resolve_mod, "auto_resolve", side_effect=fake_resolve), \
|
||||
mock.patch.object(resolve_mod, "_has_backend", return_value=True), \
|
||||
mock.patch.object(pipeline_mod, "run", side_effect=fake_pipeline_run):
|
||||
# Exercise the competitor_runner closure pattern from main() by
|
||||
# calling it directly with two competitors.
|
||||
self._run_competitor_closure(
|
||||
config=config,
|
||||
competitors=["Drake", "Kendrick Lamar"],
|
||||
mock_flag=False,
|
||||
)
|
||||
|
||||
# auto_resolve was called once per competitor
|
||||
self.assertEqual(sorted(captured_resolve_topics), ["Drake", "Kendrick Lamar"])
|
||||
# pipeline.run received resolved fields per entity
|
||||
by_topic = {kw["topic"]: kw for kw in captured_pipeline_kwargs}
|
||||
self.assertEqual(by_topic["Drake"]["x_handle"], "Drake")
|
||||
self.assertEqual(
|
||||
by_topic["Drake"]["subreddits"], ["DrakeTheType", "hiphopheads"],
|
||||
)
|
||||
self.assertEqual(by_topic["Kendrick Lamar"]["x_handle"], "kendricklamar")
|
||||
# internal_subrun=True on all competitor sub-runs
|
||||
self.assertTrue(all(kw["internal_subrun"] for kw in captured_pipeline_kwargs))
|
||||
|
||||
def test_mock_mode_skips_auto_resolve(self):
|
||||
from lib import resolve as resolve_mod
|
||||
from lib import pipeline as pipeline_mod
|
||||
|
||||
resolve_called = []
|
||||
|
||||
def fake_resolve(*a, **k):
|
||||
resolve_called.append((a, k))
|
||||
return {}
|
||||
|
||||
with mock.patch.object(resolve_mod, "auto_resolve", side_effect=fake_resolve), \
|
||||
mock.patch.object(pipeline_mod, "run", side_effect=lambda **kw: _fake_report(kw["topic"])):
|
||||
self._run_competitor_closure(
|
||||
config={"BRAVE_API_KEY": "test-key"},
|
||||
competitors=["Anthropic"],
|
||||
mock_flag=True,
|
||||
)
|
||||
|
||||
self.assertEqual(resolve_called, [])
|
||||
|
||||
def test_no_backend_skips_auto_resolve(self):
|
||||
from lib import resolve as resolve_mod
|
||||
from lib import pipeline as pipeline_mod
|
||||
|
||||
resolve_called = []
|
||||
|
||||
def fake_resolve(*a, **k):
|
||||
resolve_called.append((a, k))
|
||||
return {}
|
||||
|
||||
with mock.patch.object(resolve_mod, "auto_resolve", side_effect=fake_resolve), \
|
||||
mock.patch.object(resolve_mod, "_has_backend", return_value=False), \
|
||||
mock.patch.object(pipeline_mod, "run", side_effect=lambda **kw: _fake_report(kw["topic"])):
|
||||
self._run_competitor_closure(
|
||||
config={},
|
||||
competitors=["Anthropic"],
|
||||
mock_flag=False,
|
||||
)
|
||||
|
||||
self.assertEqual(resolve_called, [])
|
||||
|
||||
def test_resolve_failure_degrades_gracefully(self):
|
||||
from lib import resolve as resolve_mod
|
||||
from lib import pipeline as pipeline_mod
|
||||
|
||||
captured_pipeline_kwargs: list[dict] = []
|
||||
|
||||
def fake_resolve(_topic, _cfg):
|
||||
raise RuntimeError("upstream offline")
|
||||
|
||||
def fake_pipeline_run(**kwargs):
|
||||
captured_pipeline_kwargs.append(kwargs)
|
||||
return _fake_report(kwargs["topic"])
|
||||
|
||||
err = io.StringIO()
|
||||
with redirect_stderr(err), \
|
||||
mock.patch.object(resolve_mod, "auto_resolve", side_effect=fake_resolve), \
|
||||
mock.patch.object(resolve_mod, "_has_backend", return_value=True), \
|
||||
mock.patch.object(pipeline_mod, "run", side_effect=fake_pipeline_run):
|
||||
self._run_competitor_closure(
|
||||
config={"BRAVE_API_KEY": "test-key"},
|
||||
competitors=["Anthropic"],
|
||||
mock_flag=False,
|
||||
)
|
||||
|
||||
# Warning logged but run continues with planner defaults
|
||||
self.assertIn("auto_resolve failed for 'Anthropic'", err.getvalue())
|
||||
self.assertEqual(len(captured_pipeline_kwargs), 1)
|
||||
self.assertIsNone(captured_pipeline_kwargs[0]["x_handle"])
|
||||
self.assertIsNone(captured_pipeline_kwargs[0]["subreddits"])
|
||||
|
||||
def test_resolved_artifact_stored_on_report(self):
|
||||
from lib import resolve as resolve_mod
|
||||
from lib import pipeline as pipeline_mod
|
||||
|
||||
with mock.patch.object(resolve_mod, "auto_resolve", return_value={
|
||||
"x_handle": "Drake",
|
||||
"subreddits": ["DrakeTheType"],
|
||||
"github_user": "",
|
||||
"github_repos": [],
|
||||
"context": "Drake context",
|
||||
"category": None,
|
||||
"searches_run": 4,
|
||||
}), \
|
||||
mock.patch.object(resolve_mod, "_has_backend", return_value=True), \
|
||||
mock.patch.object(pipeline_mod, "run", side_effect=lambda **kw: _fake_report(kw["topic"])):
|
||||
results = self._run_competitor_closure(
|
||||
config={"BRAVE_API_KEY": "test-key"},
|
||||
competitors=["Drake"],
|
||||
mock_flag=False,
|
||||
)
|
||||
|
||||
self.assertIn("resolved", results[0].artifacts)
|
||||
resolved = results[0].artifacts["resolved"]
|
||||
self.assertEqual(resolved["entity"], "Drake")
|
||||
self.assertEqual(resolved["x_handle"], "Drake")
|
||||
self.assertEqual(resolved["subreddits"], ["DrakeTheType"])
|
||||
self.assertEqual(resolved["context"], "Drake context")
|
||||
|
||||
def test_config_not_mutated_across_sub_runs(self):
|
||||
"""_auto_resolve_context from entity A must not leak into entity B."""
|
||||
from lib import resolve as resolve_mod
|
||||
from lib import pipeline as pipeline_mod
|
||||
|
||||
captured_contexts: list[str] = []
|
||||
|
||||
def fake_resolve(topic, _cfg):
|
||||
per_topic = {
|
||||
"Drake": {"x_handle": "Drake", "subreddits": [], "github_user": "",
|
||||
"github_repos": [], "context": "Drake unique context",
|
||||
"category": None, "searches_run": 4},
|
||||
"Kendrick Lamar": {"x_handle": "kendricklamar", "subreddits": [],
|
||||
"github_user": "", "github_repos": [],
|
||||
"context": "Kendrick unique context",
|
||||
"category": None, "searches_run": 4},
|
||||
}
|
||||
return per_topic[topic]
|
||||
|
||||
def fake_pipeline_run(**kwargs):
|
||||
captured_contexts.append(
|
||||
kwargs["config"].get("_auto_resolve_context", "")
|
||||
)
|
||||
return _fake_report(kwargs["topic"])
|
||||
|
||||
shared_config = {"BRAVE_API_KEY": "test-key"}
|
||||
with mock.patch.object(resolve_mod, "auto_resolve", side_effect=fake_resolve), \
|
||||
mock.patch.object(resolve_mod, "_has_backend", return_value=True), \
|
||||
mock.patch.object(pipeline_mod, "run", side_effect=fake_pipeline_run):
|
||||
self._run_competitor_closure(
|
||||
config=shared_config,
|
||||
competitors=["Drake", "Kendrick Lamar"],
|
||||
mock_flag=False,
|
||||
)
|
||||
|
||||
# Each sub-run received its own entity's context — no cross-leak.
|
||||
self.assertIn("Drake unique context", captured_contexts)
|
||||
self.assertIn("Kendrick unique context", captured_contexts)
|
||||
# The shared outer config was not mutated
|
||||
self.assertNotIn("_auto_resolve_context", shared_config)
|
||||
|
||||
# --- test helpers -----------------------------------------------------
|
||||
|
||||
def _run_competitor_closure(self, *, config, competitors, mock_flag):
|
||||
"""Replicate the competitor_runner closure from last30days.main() and
|
||||
call it against each competitor. Returns the list of Reports."""
|
||||
from lib import pipeline, resolve as resolve_mod
|
||||
|
||||
class _Args:
|
||||
pass
|
||||
args = _Args()
|
||||
args.mock = mock_flag
|
||||
args.web_backend = "auto"
|
||||
args.lookback_days = 30
|
||||
|
||||
def runner(entity: str):
|
||||
entity_config = dict(config)
|
||||
resolved = {
|
||||
"entity": entity,
|
||||
"x_handle": "",
|
||||
"subreddits": [],
|
||||
"github_user": "",
|
||||
"github_repos": [],
|
||||
"context": "",
|
||||
}
|
||||
if not args.mock and resolve_mod._has_backend(entity_config):
|
||||
try:
|
||||
r = resolve_mod.auto_resolve(entity, entity_config)
|
||||
except Exception as exc:
|
||||
sys.stderr.write(
|
||||
f"[Competitors] auto_resolve failed for {entity!r}: "
|
||||
f"{type(exc).__name__}: {exc}\n"
|
||||
)
|
||||
r = {}
|
||||
resolved["x_handle"] = r.get("x_handle", "") or ""
|
||||
resolved["subreddits"] = list(r.get("subreddits") or [])
|
||||
resolved["github_user"] = r.get("github_user", "") or ""
|
||||
resolved["github_repos"] = list(r.get("github_repos") or [])
|
||||
resolved["context"] = r.get("context", "") or ""
|
||||
if resolved["context"]:
|
||||
entity_config["_auto_resolve_context"] = resolved["context"]
|
||||
report = pipeline.run(
|
||||
topic=entity,
|
||||
config=entity_config,
|
||||
depth="default",
|
||||
requested_sources=None,
|
||||
mock=args.mock,
|
||||
x_handle=resolved["x_handle"] or None,
|
||||
subreddits=resolved["subreddits"] or None,
|
||||
github_user=resolved["github_user"] or None,
|
||||
github_repos=resolved["github_repos"] or None,
|
||||
web_backend=args.web_backend,
|
||||
lookback_days=args.lookback_days,
|
||||
internal_subrun=True,
|
||||
)
|
||||
report.artifacts["resolved"] = resolved
|
||||
return report
|
||||
|
||||
return [runner(c) for c in competitors]
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,55 @@
|
||||
# ruff: noqa: E402
|
||||
"""Tests for planner.plan_query internal_subrun quiet mode."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import sys
|
||||
import unittest
|
||||
from contextlib import redirect_stderr
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(REPO_ROOT / "scripts"))
|
||||
|
||||
from lib import planner
|
||||
|
||||
|
||||
class PlannerQuietModeTests(unittest.TestCase):
|
||||
def _call(self, *, internal_subrun: bool):
|
||||
err = io.StringIO()
|
||||
with redirect_stderr(err):
|
||||
plan = planner.plan_query(
|
||||
topic="Acme Corp",
|
||||
available_sources=["grounding", "reddit"],
|
||||
requested_sources=None,
|
||||
depth="default",
|
||||
provider=None,
|
||||
model=None,
|
||||
internal_subrun=internal_subrun,
|
||||
)
|
||||
return plan, err.getvalue()
|
||||
|
||||
def test_default_emits_law7_warning(self):
|
||||
plan, stderr = self._call(internal_subrun=False)
|
||||
self.assertIn("No --plan passed", stderr)
|
||||
self.assertIn("YOU ARE the planner", stderr)
|
||||
self.assertTrue(plan.subqueries)
|
||||
|
||||
def test_internal_subrun_suppresses_warning(self):
|
||||
plan, stderr = self._call(internal_subrun=True)
|
||||
self.assertNotIn("No --plan passed", stderr)
|
||||
self.assertNotIn("YOU ARE the planner", stderr)
|
||||
# Still returns a valid fallback plan
|
||||
self.assertTrue(plan.subqueries)
|
||||
|
||||
def test_internal_subrun_still_allows_other_warnings(self):
|
||||
"""Quiet mode only silences the LAW 7 block, not all planner output."""
|
||||
plan, _stderr = self._call(internal_subrun=True)
|
||||
# The plan itself is deterministic fallback; verify note carries
|
||||
# no planner-error indication.
|
||||
self.assertGreater(len(plan.subqueries), 0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -159,6 +159,110 @@ class RenderComparisonMultiTests(unittest.TestCase):
|
||||
self.assertIn("GPT-5 drop", out)
|
||||
|
||||
|
||||
class ResolvedEntitiesBlockTests(unittest.TestCase):
|
||||
def _build_with_resolved(self, label, topic, resolved):
|
||||
r = _build_report(topic, ["Cluster A"])
|
||||
if resolved is not None:
|
||||
r.artifacts["resolved"] = resolved
|
||||
return (label, r)
|
||||
|
||||
def test_block_emitted_when_any_entity_has_resolved(self):
|
||||
reports = [
|
||||
self._build_with_resolved("OpenAI", "OpenAI", {
|
||||
"entity": "OpenAI",
|
||||
"x_handle": "OpenAI",
|
||||
"subreddits": ["OpenAI", "MachineLearning"],
|
||||
"github_user": "openai",
|
||||
"github_repos": ["openai/gpt"],
|
||||
"context": "GPT-5 release signals are strong",
|
||||
}),
|
||||
self._build_with_resolved("Anthropic", "Anthropic", {
|
||||
"entity": "Anthropic",
|
||||
"x_handle": "AnthropicAI",
|
||||
"subreddits": ["ClaudeAI"],
|
||||
"github_user": "anthropics",
|
||||
"github_repos": [],
|
||||
"context": "",
|
||||
}),
|
||||
]
|
||||
rendered = render.render_comparison_multi(reports)
|
||||
self.assertIn("## Resolved Entities", rendered)
|
||||
self.assertIn("**OpenAI**: X @OpenAI", rendered)
|
||||
self.assertIn("r/OpenAI, r/MachineLearning", rendered)
|
||||
self.assertIn("@openai (openai/gpt)", rendered)
|
||||
self.assertIn("**Anthropic**: X @AnthropicAI", rendered)
|
||||
# Missing context renders as "-"
|
||||
self.assertIn("Context: -", rendered)
|
||||
|
||||
def test_block_omitted_when_no_resolved_artifacts(self):
|
||||
reports = [
|
||||
self._build_with_resolved("A", "A", None),
|
||||
self._build_with_resolved("B", "B", None),
|
||||
]
|
||||
rendered = render.render_comparison_multi(reports)
|
||||
self.assertNotIn("## Resolved Entities", rendered)
|
||||
|
||||
def test_missing_fields_render_as_dash(self):
|
||||
reports = [
|
||||
self._build_with_resolved("OpenAI", "OpenAI", {
|
||||
"entity": "OpenAI",
|
||||
"x_handle": "",
|
||||
"subreddits": [],
|
||||
"github_user": "",
|
||||
"github_repos": [],
|
||||
"context": "",
|
||||
}),
|
||||
]
|
||||
rendered = render.render_comparison_multi(reports)
|
||||
self.assertIn("**OpenAI**: X - | Subs - | GitHub - | Context: -", rendered)
|
||||
|
||||
def test_long_context_truncated(self):
|
||||
long = "a" * 200
|
||||
reports = [
|
||||
self._build_with_resolved("X", "X", {
|
||||
"entity": "X",
|
||||
"x_handle": "",
|
||||
"subreddits": [],
|
||||
"github_user": "",
|
||||
"github_repos": [],
|
||||
"context": long,
|
||||
}),
|
||||
]
|
||||
rendered = render.render_comparison_multi(reports)
|
||||
# The truncate helper adds an ellipsis; context line should not show
|
||||
# the full 200-char string.
|
||||
self.assertNotIn("a" * 200, rendered)
|
||||
|
||||
def test_context_emit_includes_resolved_block(self):
|
||||
reports = [
|
||||
self._build_with_resolved("OpenAI", "OpenAI", {
|
||||
"entity": "OpenAI",
|
||||
"x_handle": "OpenAI",
|
||||
"subreddits": ["OpenAI"],
|
||||
"github_user": "",
|
||||
"github_repos": [],
|
||||
"context": "",
|
||||
}),
|
||||
]
|
||||
out = render.render_comparison_multi_context(reports)
|
||||
self.assertIn("## Resolved Entities", out)
|
||||
self.assertIn("**OpenAI**: X @OpenAI", out)
|
||||
|
||||
def test_subreddit_overflow_truncated(self):
|
||||
reports = [
|
||||
self._build_with_resolved("X", "X", {
|
||||
"entity": "X",
|
||||
"x_handle": "",
|
||||
"subreddits": ["a", "b", "c", "d", "e", "f", "g"],
|
||||
"github_user": "",
|
||||
"github_repos": [],
|
||||
"context": "",
|
||||
}),
|
||||
]
|
||||
rendered = render.render_comparison_multi(reports)
|
||||
self.assertIn("r/a, r/b, r/c, r/d, r/e (+2)", rendered)
|
||||
|
||||
|
||||
class EmitComparisonOutputTests(unittest.TestCase):
|
||||
def test_json_emit_nests_per_entity(self):
|
||||
reports = [
|
||||
|
||||
Reference in New Issue
Block a user