feat: --competitors flag for auto-discovered comparison fan-out (#308)
Pass `--competitors` on a single-entity topic and the engine auto-discovers 2-6 peer entities via web search, runs the full pipeline on each in parallel, and returns one N-way comparison reusing the existing 9-axis Head-to-Head scaffold. `last30days OpenAI --competitors` resolves to Anthropic + xAI + Google Gemini; `last30days Kanye West --competitors` resolves to Drake + Kendrick Lamar + one more peer. - New CLI flags: --competitors, --competitors=N, --competitors-list - New scripts/lib/competitors.py — mirrors resolve.auto_resolve pattern (web search + deterministic text extraction, no internal LLM) - New scripts/lib/fanout.py — ThreadPoolExecutor orchestrator; per-entity failures degrade gracefully as long as >=2 entities survive - Multi-report render in scripts/lib/render.py reuses the comparison scaffold for the synthesis table - LAW 7-style stderr when no backend and no list, pointing the hosting reasoning model at --competitors-list - 38 new tests across CLI parsing, discovery, fanout, and rendering Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matt Van Horn
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "last30days",
|
||||
"version": "3.0.10",
|
||||
"version": "3.0.11",
|
||||
"description": "Research any topic across Reddit, X, YouTube, TikTok, Instagram, Hacker News, Polymarket, GitHub, and 5+ more sources. AI agent scores by upvotes, likes, and real money - not editors.",
|
||||
"author": {
|
||||
"name": "Matt Van Horn",
|
||||
|
||||
@@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [3.0.11] - 2026-04-22
|
||||
|
||||
### Added
|
||||
|
||||
- **`--competitors` flag for auto-discovered comparison fan-out.** Pass `--competitors` on a single-entity topic and the engine discovers 2-6 peer entities via web search, then runs the full pipeline on each in parallel and emits one N-way comparison. `last30days Kanye West --competitors` resolves Drake, Kendrick Lamar, and one more peer. `last30days OpenAI --competitors` resolves Anthropic, xAI, Google Gemini. `--competitors=N` controls count, `--competitors-list="A,B,C"` skips discovery and uses the explicit list. Discovery mirrors the `auto_resolve` pattern (Brave / Exa / Serper / Parallel) with deterministic text extraction - no internal LLM call. Sub-runs inherit the main `--quick`/`--deep`/`--days`, run in a `ThreadPoolExecutor`, and degrade gracefully when at least 2 entities survive. Output reuses the existing 9-axis `## Head-to-Head` scaffold.
|
||||
|
||||
## [3.0.10] - 2026-04-21
|
||||
|
||||
### Added
|
||||
|
||||
@@ -114,6 +114,10 @@ When the same story appears on Reddit, X, and YouTube, v3 merges them into one c
|
||||
|
||||
"CLI vs MCP" used to run three serial passes (12+ minutes). v3 runs one pass with entity-aware subqueries for both sides simultaneously. Same depth, 3 minutes.
|
||||
|
||||
### Auto-discovered competitor comparisons
|
||||
|
||||
`/last30days OpenAI --competitors` discovers the top 3 peers via web search (Anthropic, xAI, Google Gemini), runs the full pipeline on each in parallel, and returns one N-way comparison report. Override with `--competitors=N` or `--competitors-list="A,B,C"`.
|
||||
|
||||
### GitHub person-mode
|
||||
|
||||
When the topic is a person, the engine switches from keyword search to author-scoped queries. Instead of "who mentioned this name in an issue body," it answers: what are they shipping and where is it landing?
|
||||
|
||||
@@ -573,6 +573,21 @@ Then do WebSearch for: `{TOPIC_A} vs {TOPIC_B} comparison {YEAR}` and `{TOPIC_A}
|
||||
|
||||
**COMPARISON TABLE SCAFFOLD (engine-emitted, pass through verbatim):** For comparison topics, the engine's compact output includes a `## Head-to-Head Comparison` block with an empty markdown table (columns = entities, rows = axes like "Core pitch", "Who it's for", "Community stance", "Trajectory") plus a "Choose X if / Choose Y if" prose block. Your synthesis MUST include this block verbatim with filled cells, positioned between the narrative and the emoji-tree footer. Keep each cell to 5-15 words. Use ' - ' (hyphen with spaces) not em-dashes inside cells. The block is the canonical comparison output shape - do not invent your own table structure.
|
||||
|
||||
### Competitor mode (`--competitors`)
|
||||
|
||||
When the user passes `--competitors` on a single-entity topic, the engine auto-discovers 2-6 peer entities and fans out the full pipeline over the topic plus each competitor in parallel. Example: `last30days Kanye West --competitors` resolves to a 4-way comparison against Drake, Kendrick Lamar, and one other peer; `last30days OpenAI --competitors` resolves against Anthropic, xAI, and Google Gemini.
|
||||
|
||||
**Flag surface:**
|
||||
- `--competitors` (bare) - discover and compare against 3 peers.
|
||||
- `--competitors=N` - discover N peers (range 1..6; out-of-range clamps with a stderr warning).
|
||||
- `--competitors-list="A,B,C"` - skip discovery and use the explicit list. Implies `--competitors`.
|
||||
|
||||
**Discovery path:** web search plus deterministic text mining (same backends as `--auto-resolve`). No internal LLM call. When no web search backend is configured and no list is passed, the engine emits a LAW 7-style stderr telling the hosting reasoning model to generate the list and re-invoke with `--competitors-list="A,B,C"`, then exits non-zero.
|
||||
|
||||
**Sub-run behavior:** each entity runs `pipeline.run()` in parallel inheriting the main run's `--quick` / `--deep` / `--web-backend` / `--days`. Topic-specific overrides (`--x-handle`, `--subreddits`, `--github-user`, `--github-repo`) apply to the main topic only - competitor sub-runs use planner defaults. Per-entity failures degrade to a warning; the run continues as long as at least 2 entities survive.
|
||||
|
||||
**Output:** one comparison report covering all entities, reusing the same `## Head-to-Head` scaffold as explicit `A vs B` topics. Synthesis contract identical to the COMPARISON query type above.
|
||||
|
||||
---
|
||||
|
||||
## Step 0.55: Pre-Research Intelligence (resolve communities + handles)
|
||||
|
||||
@@ -0,0 +1,303 @@
|
||||
---
|
||||
title: "feat: --competitors flag for auto-discovered comparison fan-out"
|
||||
type: feat
|
||||
status: active
|
||||
date: 2026-04-22
|
||||
---
|
||||
|
||||
# feat: --competitors flag for auto-discovered comparison fan-out
|
||||
|
||||
## Overview
|
||||
|
||||
Add a `--competitors` flag to the last30days engine that auto-discovers 2-4 peer entities for the topic, runs the full retrieval pipeline on each in parallel, and renders a multi-entity comparison. Invoking `last30days Kanye West --competitors` should resolve to "Kanye vs Drake vs Kendrick Lamar" and emit a comparison report covering all three. Invoking `last30days OpenAI --competitors` should resolve to "OpenAI vs Anthropic vs xAI vs Gemini" and emit a four-way comparison.
|
||||
|
||||
Discovery mirrors the existing `resolve.auto_resolve()` pattern used for X handles and subreddits at pipeline start — web search (Brave / Exa / Serper) plus deterministic extraction. Not an internal LLM call.
|
||||
|
||||
## Problem Frame
|
||||
|
||||
Users who want a comparison today must type "OpenAI vs Anthropic vs xAI" themselves. The `planner._comparison_entities()` path already handles explicit multi-entity topics and `render._render_comparison_scaffold()` already emits a 9-axis comparison table. What is missing is the discovery half — a user who types a single entity with `--competitors` should get the comparison for free.
|
||||
|
||||
This is also the natural next step after the Step 0.55 category-peer subreddit work (PR #305, merged 2026-04-22). That feature widens the subreddit set within a single topic; this feature widens the entity set into peer entities.
|
||||
|
||||
## Requirements Trace
|
||||
|
||||
- R1. New `--competitors` boolean flag that triggers competitor discovery and multi-entity fan-out.
|
||||
- R2. New `--competitors-list="A,B,C"` to explicitly skip discovery (mirrors `--plan`, `--subreddits`, `--x-handle` overrides).
|
||||
- R3. New `--competitors=N` short form to set competitor count inline (N in 1..6).
|
||||
- R4. Default count is 3 competitors (original + 3 = 4-way comparison).
|
||||
- R5. Competitor retrieval depth inherits the main run's depth (`--quick` / `--deep`); all entities run in parallel so wall clock stays close to a single run.
|
||||
- R6. Discovery mirrors `resolve.auto_resolve()`: web search for peers, deterministic text extraction. No internal LLM dependency.
|
||||
- R7. If no web search backend is configured and no `--competitors-list` was passed, engine emits a LAW 7-style stderr telling the host agent to pass `--competitors-list` and exits non-zero.
|
||||
- R8. Output rendering is a single comparison report covering all entities, reusing the existing 9-axis scaffold from `render._render_comparison_scaffold()` where applicable.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Synthesis prompt changes beyond wiring N reports into the existing comparison scaffold are out of scope.
|
||||
- `--competitors` does not replace the existing explicit "A vs B vs C" topic parsing in `planner._comparison_entities()`; both paths coexist.
|
||||
- No caching layer for discovery results in v1.
|
||||
- No UI/SKILL.md rewrite of the entire comparison section; only the new flag is documented.
|
||||
- No new web search backend.
|
||||
|
||||
### Deferred to Separate Tasks
|
||||
|
||||
- Caching of competitor lookups: separate follow-up once hit rate justifies it.
|
||||
- Disambiguation UX for topics with multiple common entities ("Amazon" the company vs the river): separate brainstorm.
|
||||
|
||||
## Context & Research
|
||||
|
||||
### Relevant Code and Patterns
|
||||
|
||||
- `scripts/last30days.py:168-249` — `build_parser()` argparse definitions. Existing depth flags (`--quick`, `--deep`) and override flags (`--plan`, `--subreddits`, `--x-handle`, `--auto-resolve`) set the convention to mirror.
|
||||
- `scripts/lib/resolve.py:179-258` — `auto_resolve()` is the reference pattern: web search fan-out via `ThreadPoolExecutor`, per-query extraction functions, graceful empty-dict return when no backend is available.
|
||||
- `scripts/lib/resolve.py:98-140` — `_extract_x_handle()` and sibling extractors show the deterministic text-mining style competitor extraction should mirror.
|
||||
- `scripts/lib/pipeline.py:162-220` — `pipeline.run()` signature is the fan-out target. One call per entity, each returning a `schema.Report`.
|
||||
- `scripts/lib/planner.py:430-564` — Existing comparison-intent handling and `_comparison_entities()` entity extraction. The new flag feeds the same mental model but populates entities from discovery instead of from the topic string.
|
||||
- `scripts/lib/render.py:333-392` — `_render_comparison_scaffold()` already emits a 9-axis markdown comparison table. The new multi-report renderer should reuse this helper by assembling a synthetic "A vs B vs C" topic header for it.
|
||||
- `scripts/lib/grounding.py` + `scripts/lib/providers.py` — Web search backend resolution (Brave / Exa / Serper). Reused as-is.
|
||||
|
||||
### Institutional Learnings
|
||||
|
||||
- No existing `docs/solutions/` entries for competitor discovery or multi-entity fan-out.
|
||||
- Recent plan `docs/plans/2026-04-22-001-fix-category-peer-subreddit-resolution-plan.md` established the precedent of deterministic peer expansion; this plan extends that idea from subreddits to entities.
|
||||
|
||||
### External References
|
||||
|
||||
- None gathered — local patterns are strong. `resolve.auto_resolve()` is a direct template.
|
||||
|
||||
## Key Technical Decisions
|
||||
|
||||
- **Discovery mirrors auto_resolve, not plan_query.** Web search + regex extraction, not an LLM call. Matches the user's explicit direction ("use the python brain the same way it searches for X handles"). Cheaper, no provider credential requirement, deterministic.
|
||||
- **Orchestration lives in `last30days.py` main, not inside `pipeline.run()`.** The fan-out is a top-level concern — one pipeline run per entity, each independent. Keeps `pipeline.run()` single-entity and unchanged except for sharing a `ThreadPoolExecutor` factory.
|
||||
- **Sub-runs inherit main depth and run in parallel.** Wall clock ≈ single run; token cost scales linearly with N. User-controlled via the existing `--quick`/`--deep` flags.
|
||||
- **New module `scripts/lib/competitors.py` instead of adding to `resolve.py`.** Keeps resolve focused on single-entity entity-bundle discovery (handles/subreddits/github); competitors.py owns peer-entity discovery. Similar shape, different responsibility.
|
||||
- **Multi-report render is additive in `render.py`.** New `render_comparison_multi(reports: list[Report]) -> str` composes a synthetic "A vs B vs C" topic and delegates to the existing scaffold + synthesis path where possible. No rewrite of the single-entity render path.
|
||||
- **Default count = 3 competitors (4-way comparison).** Hard cap at 6.
|
||||
- **LAW 7-style stderr when no backend and no list.** Matches how `planner.plan_query()` already tells the hosting agent to pass `--plan`.
|
||||
|
||||
## Open Questions
|
||||
|
||||
### Resolved During Planning
|
||||
|
||||
- **Discovery mechanism:** Web search via `grounding.web_search()`, not an internal LLM. User confirmed the auto_resolve pattern is the target.
|
||||
- **Default competitor count:** 3 (original + 3 = 4-way).
|
||||
- **Sub-run depth:** Inherit main depth, parallel execution.
|
||||
- **Flag naming:** `--competitors` (standard argparse double-dash). `--competitors=N` for inline count. `--competitors-list="A,B,C"` to skip discovery.
|
||||
|
||||
### Deferred to Implementation
|
||||
|
||||
- Exact extraction heuristics for competitor names across Brave / Exa / Serper result shapes. The SERP text varies (listicles, comparison pages, "vs" pages); the initial implementation will start with listicle parsing plus a "X vs Y" pattern match, and harden against real results in the test phase.
|
||||
- Handling of topic ambiguity ("Amazon", "Apple"). Initial behavior: trust whatever web search returns for the topic verbatim; disambiguation is a separate concern.
|
||||
- Merge strategy when two entities return overlapping URLs (e.g., an "OpenAI vs Anthropic" article shows up in both runs). Likely dedupe at the clustering step, but defer the exact policy until we see how often it happens.
|
||||
- Whether to expose competitor discovery artifacts (the raw web search results) as a debug emit. Follow the existing `--debug` conventions.
|
||||
|
||||
## Implementation Units
|
||||
|
||||
- [ ] **Unit 1: CLI flag parsing and validation**
|
||||
|
||||
**Goal:** Add `--competitors`, `--competitors=N`, and `--competitors-list` to the argparse surface, validate values, and thread them into the main orchestration.
|
||||
|
||||
**Requirements:** R1, R2, R3, R4
|
||||
|
||||
**Dependencies:** None
|
||||
|
||||
**Files:**
|
||||
- Modify: `scripts/last30days.py`
|
||||
- Test: `tests/test_cli_competitors.py`
|
||||
|
||||
**Approach:**
|
||||
- Add three mutually cooperative flags near line 205 in `build_parser()`:
|
||||
- `--competitors` with `nargs="?"` and `const=3` so bare `--competitors` defaults to 3, `--competitors=4` is honored, and `--competitors=0` is rejected
|
||||
- `--competitors-list` free-text CSV
|
||||
- Normalize in `main()`: if `--competitors-list` is present, skip discovery and use the list. If `--competitors` is set and no list, trigger discovery with count = the flag value. Clamp count to 1..6 with a stderr warning at boundary.
|
||||
- Thread the resulting entity list into the orchestrator added in Unit 3.
|
||||
|
||||
**Patterns to follow:**
|
||||
- `--plan` argument at `scripts/last30days.py:187` — same skip-discovery-when-explicit shape.
|
||||
- `--subreddits` / `--x-handle` at `scripts/last30days.py:180,189` — same override semantics.
|
||||
|
||||
**Test scenarios:**
|
||||
- Happy path: bare `--competitors` parses to count=3, empty list.
|
||||
- Happy path: `--competitors=4` parses to count=4.
|
||||
- Happy path: `--competitors-list="A,B,C"` parses to count=3, list=["A","B","C"], and is preferred over any discovery signal.
|
||||
- Edge case: `--competitors=0` and `--competitors=-1` are rejected with a clear error.
|
||||
- Edge case: `--competitors=99` clamps to 6 with a stderr warning.
|
||||
- Edge case: `--competitors` combined with `--competitors-list` uses the list and logs that discovery was skipped.
|
||||
- Edge case: `--competitors-list` value with whitespace ("A, B , C") normalizes correctly.
|
||||
|
||||
**Verification:**
|
||||
- Running the binary with each flag variation produces the expected post-parse state without calling out to the network.
|
||||
|
||||
- [ ] **Unit 2: `scripts/lib/competitors.py` discovery module**
|
||||
|
||||
**Goal:** Discover peer entities for a topic using web search + deterministic extraction, mirroring `resolve.auto_resolve()`.
|
||||
|
||||
**Requirements:** R6, R7
|
||||
|
||||
**Dependencies:** None (pure module; wired by Unit 3)
|
||||
|
||||
**Files:**
|
||||
- Create: `scripts/lib/competitors.py`
|
||||
- Test: `tests/test_competitors.py`
|
||||
|
||||
**Approach:**
|
||||
- Public entry point `discover_competitors(topic: str, count: int, config: dict) -> list[str]`.
|
||||
- Early return `[]` when `_has_backend(config)` is false (reuse the helper from `resolve.py`; factor if needed).
|
||||
- Fan out 2-3 web searches in a `ThreadPoolExecutor`:
|
||||
- `"{topic} competitors"`
|
||||
- `"{topic} alternatives"`
|
||||
- `"{topic} vs"` (captures "X vs Y" articles)
|
||||
- Feed results into a deterministic `_extract_peer_entities(results, topic)` that:
|
||||
- Mines titles and snippets for capitalized noun phrases other than the topic itself
|
||||
- Scores by frequency across results
|
||||
- Filters stopwords and the topic's own tokens
|
||||
- Returns top `count` unique entities ordered by score
|
||||
- Emit a single-line stderr log mirroring the `resolve._log` format.
|
||||
|
||||
**Patterns to follow:**
|
||||
- `scripts/lib/resolve.py:179-258` for the function shape, executor usage, and empty-result fallback.
|
||||
- `scripts/lib/resolve.py:98-140` for extractor style (small, deterministic, no external state).
|
||||
|
||||
**Test scenarios:**
|
||||
- Happy path: canned SERP fixtures for "OpenAI" return ["Anthropic", "xAI", "Google"] or close peers in the top 3.
|
||||
- Happy path: canned SERP fixtures for "Kanye West" return rap peers (Drake, Kendrick) in the top 3.
|
||||
- Edge case: empty SERP results return `[]` without raising.
|
||||
- Edge case: extractor filters out the topic itself (case- and punctuation-insensitive).
|
||||
- Edge case: near-duplicate entities ("OpenAI" vs "Open AI") dedupe to one slot.
|
||||
- Error path: web search backend raises — the failure is logged and the function returns `[]`.
|
||||
- Edge case: count=1 returns a single-element list; count=6 returns up to six entities.
|
||||
|
||||
**Verification:**
|
||||
- Unit tests pass with fixtures committed under `tests/fixtures/competitors-*.json`.
|
||||
- Manual run against a live backend for one topic confirms sensible output (recorded as a notes file, not a test assertion).
|
||||
|
||||
- [ ] **Unit 3: Parallel fan-out orchestrator**
|
||||
|
||||
**Goal:** Run `pipeline.run()` once per entity (topic + discovered competitors) in parallel, collect `schema.Report` per entity, and hand them to the comparison renderer.
|
||||
|
||||
**Requirements:** R5, R7
|
||||
|
||||
**Dependencies:** Unit 1, Unit 2
|
||||
|
||||
**Files:**
|
||||
- Modify: `scripts/last30days.py`
|
||||
- Possibly create: `scripts/lib/fanout.py` if the orchestrator grows past ~60 lines
|
||||
- Test: `tests/test_competitor_fanout.py`
|
||||
|
||||
**Approach:**
|
||||
- After arg parsing and before the existing `pipeline.run()` call, branch on `args.competitors`:
|
||||
- If a list was provided or discovery returned entities, build `entities = [topic, *competitors]`.
|
||||
- Spawn one `pipeline.run()` per entity via `ThreadPoolExecutor(max_workers=len(entities))`, passing the same `config`, `depth`, and all sub-run-relevant args (mock, plan, etc.). Respect `--plan` — if a plan is passed it applies to the main topic only; competitors use the internal planner fallback for v1.
|
||||
- Collect `{entity: Report}` mapping. A per-entity failure logs a stderr warning and drops that entity from the comparison; the run continues as long as 2 entities succeed.
|
||||
- If fewer than 2 entities survive, exit with a clear error.
|
||||
- LAW 7-style stderr:
|
||||
- If `args.competitors` is set, no list was passed, no web search backend is configured, emit a LAW 7 stderr message pointing to the `--competitors-list` override and exit non-zero. Reuse the tone from `planner.plan_query()` fallback (`scripts/lib/planner.py:125-135`).
|
||||
|
||||
**Execution note:** Start with a failing integration test that exercises the full main → orchestrator → mocked pipeline.run path; the orchestrator is where bugs hide.
|
||||
|
||||
**Patterns to follow:**
|
||||
- `scripts/lib/resolve.py:225-239` for ThreadPoolExecutor + as_completed + per-future error handling.
|
||||
- `scripts/lib/pipeline.py:310+` for how ThreadPoolExecutor is already used inside a single run (same idiom, outer layer).
|
||||
|
||||
**Test scenarios:**
|
||||
- Happy path: main + 2 competitors, all three `pipeline.run()` calls succeed (mocked), orchestrator returns 3 Reports.
|
||||
- Happy path: discovery returns the competitor list; orchestrator fans out accordingly.
|
||||
- Edge case: one of three competitor pipelines raises — the run continues with the surviving 2 and emits a warning.
|
||||
- Edge case: all competitors fail but the main topic succeeds — orchestrator exits non-zero with a clear error rather than silently degrading to a single-entity render.
|
||||
- Edge case: `--competitors` set, no backend, no list — orchestrator emits the LAW 7 stderr and exits non-zero before any pipeline call.
|
||||
- Integration: wall-clock time for 3 mocked pipelines in parallel is close to the slowest single run, not the sum (timing assertion with generous margin).
|
||||
|
||||
**Verification:**
|
||||
- End-to-end test with mocked `pipeline.run()` and mocked competitors discovery produces 3 Reports and hands them to a stubbed renderer.
|
||||
|
||||
- [ ] **Unit 4: Multi-report comparison renderer**
|
||||
|
||||
**Goal:** Compose N `schema.Report`s into a single comparison-mode output, reusing the existing 9-axis scaffold.
|
||||
|
||||
**Requirements:** R8
|
||||
|
||||
**Dependencies:** Unit 3
|
||||
|
||||
**Files:**
|
||||
- Modify: `scripts/lib/render.py`
|
||||
- Test: `tests/test_render_comparison_multi.py`
|
||||
|
||||
**Approach:**
|
||||
- Add `render_comparison_multi(reports: list[schema.Report], *, emit: str) -> str`.
|
||||
- Build a synthetic comparison topic: `f"{entity_a} vs {entity_b} vs {entity_c}"`.
|
||||
- Reuse `_render_comparison_scaffold()` for the table skeleton. Each entity column is populated from its own Report's top clusters and citations.
|
||||
- For the narrative synthesis block, concatenate per-entity highlights, clearly labeled by entity, under a shared "Comparison" header.
|
||||
- Preserve existing emit modes (`compact`, `md`, `json`, `context`). In `json` emit, return a `{"entities": [...], "reports": [...]}` shape; single-Report consumers remain unaffected because the single-report render path is untouched.
|
||||
|
||||
**Patterns to follow:**
|
||||
- `scripts/lib/render.py:333-392` (`_parse_comparison_entities`, `_render_comparison_scaffold`) — the scaffold is the contract.
|
||||
- `scripts/lib/render.py` single-report rendering — for per-entity narrative blocks.
|
||||
|
||||
**Test scenarios:**
|
||||
- Happy path: 3 Reports with distinct clusters render into a 3-column table and a "Comparison" section that mentions each entity at least once.
|
||||
- Happy path: 2 Reports render as a 2-column table without breaking the scaffold.
|
||||
- Edge case: a Report with an empty cluster list renders as "(no significant discussion this month)" in its column rather than crashing.
|
||||
- Edge case: Reports with overlapping URLs (same article cited by two entities) dedupe citations at the footer but keep both column entries.
|
||||
- Emit variants: `--emit=compact`, `--emit=md`, `--emit=json`, `--emit=context` each produce valid output with all entities represented.
|
||||
- Integration: end-to-end snapshot test using fixture Reports, checked against a stored expected output (with a clear update path when the scaffold intentionally evolves).
|
||||
|
||||
**Verification:**
|
||||
- Snapshot tests pass. Manual review of one real 3-way comparison confirms readability.
|
||||
|
||||
- [ ] **Unit 5: Docs, SKILL.md mention, and sync**
|
||||
|
||||
**Goal:** Document the new flag so the hosting agent and human users both know it exists, and run the sync script.
|
||||
|
||||
**Requirements:** R1-R8 (surfaces them to users)
|
||||
|
||||
**Dependencies:** Units 1-4
|
||||
|
||||
**Files:**
|
||||
- Modify: `SKILL.md`
|
||||
- Modify: `README.md` (brief flag reference)
|
||||
- Modify: `CHANGELOG.md`
|
||||
- Run: `bash scripts/sync.sh`
|
||||
|
||||
**Approach:**
|
||||
- Add a compact "Competitor mode" subsection under the existing comparison docs in `SKILL.md`. Document the flag, the default count, the override flag, and the LAW 7 fallback stderr.
|
||||
- Keep `README.md` addition to a single example line.
|
||||
- CHANGELOG entry mirrors the voice of recent entries (imperative, outcome-first).
|
||||
- Sync via `scripts/sync.sh` per CLAUDE.md rules so `~/.claude/`, `~/.agents/`, `~/.codex/` pick up the new SKILL.md.
|
||||
|
||||
**Test scenarios:**
|
||||
- Test expectation: none — documentation and sync only. Verification is by inspection and by running `sync.sh` and confirming target directories updated.
|
||||
|
||||
**Verification:**
|
||||
- `sync.sh` completes without errors.
|
||||
- `SKILL.md` rendered preview mentions `--competitors` in the comparison section.
|
||||
|
||||
## System-Wide Impact
|
||||
|
||||
- **Interaction graph:** `last30days.py main()` now orchestrates multiple `pipeline.run()` calls instead of one. No other callers of `pipeline.run()` are affected (it remains single-entity).
|
||||
- **Error propagation:** Per-entity failures degrade gracefully as long as ≥2 entities survive; fewer survivors exits non-zero. Discovery failure with `--competitors` and no list is fatal.
|
||||
- **State lifecycle risks:** Each sub-run uses its own `pipeline.run()` state; no shared mutable config. The `config` dict is read-only in `pipeline.run()` today — verify before committing to shared-reference passing, else deep-copy per sub-run.
|
||||
- **API surface parity:** `--competitors` coexists with the existing explicit "A vs B vs C" topic parsing in `planner._comparison_entities()`. Both produce comparable output formats; the only difference is where the entity list came from.
|
||||
- **Integration coverage:** The fan-out orchestrator crosses CLI → discovery → N pipelines → render; integration tests in Unit 3 and Unit 4 must exercise the full path end to end, not just unit-level.
|
||||
- **Unchanged invariants:** `pipeline.run()` signature and single-entity semantics are unchanged. The single-entity render path in `render.py` is unchanged. No changes to `planner.plan_query()`. No changes to existing flags.
|
||||
|
||||
## Risks & Dependencies
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Competitor discovery returns garbage entities for niche topics. | `--competitors-list` override lets the user (or hosting agent) correct it. Unit tests with edge-case fixtures. Log discovery output to stderr under `--debug`. |
|
||||
| Token cost scales linearly with N sub-runs. | Default count capped at 3, hard max 6, inherit `--quick` to let users throttle. Wall clock stays parallel. Emit a cost hint to stderr when N ≥ 4. |
|
||||
| Merge conflicts against the single-entity render path during refactoring. | Keep the multi-report renderer strictly additive; do not modify the single-Report code path. |
|
||||
| Config dict mutation inside sub-runs could leak state between entities. | Verify read-only usage before sharing references. If any sub-component mutates, deep-copy per sub-run before spawning threads. |
|
||||
| A SERP extractor that works on Brave fixtures breaks on Exa/Serper result shapes. | Test fixtures for all three backends. Extractor operates on a normalized shape from `grounding.web_search()` (already the case), not raw provider output. |
|
||||
| Hosting agent (Claude Code, Codex) unaware of the new flag when it could usefully pass `--competitors-list`. | SKILL.md updated in Unit 5 documents the flag in the same style as `--plan` and `--auto-resolve`. |
|
||||
|
||||
## Documentation / Operational Notes
|
||||
|
||||
- Beta channel first: per `CLAUDE.md`, experimental changes go to `mvanhorn/last30days-skill-private` on the `/last30days-beta` command. Land this on the private repo first, shake out on real topics for a day or two, then cherry-pick to public.
|
||||
- After land-merge: run `scripts/sync.sh` to deploy SKILL.md + scripts to `~/.claude/`, `~/.agents/`, `~/.codex/`.
|
||||
- Release notes entry in CHANGELOG.md follows the v3.0.9 voice — outcome-first, one paragraph.
|
||||
|
||||
## Sources & References
|
||||
|
||||
- Related code: `scripts/lib/resolve.py:179` (`auto_resolve`), `scripts/lib/pipeline.py:162` (`pipeline.run`), `scripts/lib/planner.py:80` (`plan_query` LAW 7 fallback), `scripts/lib/render.py:333` (comparison scaffold)
|
||||
- Related PRs: #305 (Step 0.55 category-peer subreddit expansion — the precedent for deterministic peer expansion, merged 2026-04-22)
|
||||
- Related plan: `docs/plans/2026-04-22-001-fix-category-peer-subreddit-resolution-plan.md`
|
||||
+195
-19
@@ -122,6 +122,31 @@ def emit_output(report: schema.Report, emit: str, fun_level: str = "medium", sav
|
||||
raise SystemExit(f"Unsupported emit mode: {emit}")
|
||||
|
||||
|
||||
def emit_comparison_output(
|
||||
entity_reports: list[tuple[str, schema.Report]],
|
||||
emit: str,
|
||||
fun_level: str = "medium",
|
||||
save_path: str | None = None,
|
||||
) -> str:
|
||||
if emit == "json":
|
||||
payload = {
|
||||
"comparison": True,
|
||||
"entities": [label for label, _ in entity_reports],
|
||||
"reports": [
|
||||
{"entity": label, "report": schema.to_dict(report)}
|
||||
for label, report in entity_reports
|
||||
],
|
||||
}
|
||||
return json.dumps(payload, indent=2, sort_keys=True)
|
||||
if emit in {"compact", "md"}:
|
||||
return render.render_comparison_multi(
|
||||
entity_reports, fun_level=fun_level, save_path=save_path,
|
||||
)
|
||||
if emit == "context":
|
||||
return render.render_comparison_multi_context(entity_reports)
|
||||
raise SystemExit(f"Unsupported emit mode: {emit}")
|
||||
|
||||
|
||||
def compute_save_path_display(save_dir: str, topic: str, suffix: str, emit: str) -> str:
|
||||
"""Compute the user-friendly save path string that will be shown in the footer.
|
||||
|
||||
@@ -202,9 +227,85 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
help="Use web search to discover subreddits/handles before planning (for platforms without WebSearch)")
|
||||
parser.add_argument("--github-user", help="GitHub username for person-mode search (e.g., steipete)")
|
||||
parser.add_argument("--github-repo", help="Comma-separated owner/repo for project-mode search (e.g., openclaw/openclaw,paperclipai/paperclip)")
|
||||
parser.add_argument(
|
||||
"--competitors",
|
||||
nargs="?",
|
||||
const=3,
|
||||
type=int,
|
||||
default=None,
|
||||
metavar="N",
|
||||
help="Auto-discover N competitor entities and fan out last30days across all of them as a comparison (default N=3, range 1..6). Use --competitors-list to override discovery.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--competitors-list",
|
||||
dest="competitors_list",
|
||||
help="Comma-separated competitor entities to skip discovery (e.g., 'Anthropic,xAI,Google Gemini'). Implies --competitors.",
|
||||
)
|
||||
return parser
|
||||
|
||||
|
||||
COMPETITORS_MIN = 1
|
||||
COMPETITORS_MAX = 6
|
||||
COMPETITORS_DEFAULT = 3
|
||||
|
||||
|
||||
def resolve_competitors_args(args: argparse.Namespace) -> tuple[bool, int, list[str]]:
|
||||
"""Normalize --competitors / --competitors-list into (enabled, count, explicit_list).
|
||||
|
||||
- (False, 0, []) when neither flag is set.
|
||||
- An explicit list always wins; count is derived from list length.
|
||||
- A numeric count outside [1, 6] is clamped with a stderr warning.
|
||||
- count <= 0 (explicit) raises SystemExit(2).
|
||||
"""
|
||||
explicit_list: list[str] = []
|
||||
list_flag_provided = args.competitors_list is not None
|
||||
if list_flag_provided:
|
||||
explicit_list = [
|
||||
entity.strip()
|
||||
for entity in args.competitors_list.split(",")
|
||||
if entity.strip()
|
||||
]
|
||||
if not explicit_list:
|
||||
sys.stderr.write("[Competitors] --competitors-list is empty.\n")
|
||||
raise SystemExit(2)
|
||||
|
||||
competitors_flag = args.competitors
|
||||
list_present = bool(explicit_list)
|
||||
flag_present = competitors_flag is not None
|
||||
|
||||
if not list_present and not flag_present:
|
||||
return False, 0, []
|
||||
|
||||
if list_present:
|
||||
count = len(explicit_list)
|
||||
if flag_present and competitors_flag != count:
|
||||
sys.stderr.write(
|
||||
f"[Competitors] --competitors={competitors_flag} ignored; using "
|
||||
f"{count} entries from --competitors-list.\n"
|
||||
)
|
||||
if count > COMPETITORS_MAX:
|
||||
sys.stderr.write(
|
||||
f"[Competitors] --competitors-list has {count} entries, clamping to {COMPETITORS_MAX}.\n"
|
||||
)
|
||||
explicit_list = explicit_list[:COMPETITORS_MAX]
|
||||
count = COMPETITORS_MAX
|
||||
return True, count, explicit_list
|
||||
|
||||
# flag_present, no explicit list
|
||||
count = competitors_flag
|
||||
if count < COMPETITORS_MIN:
|
||||
sys.stderr.write(
|
||||
f"[Competitors] --competitors must be >= {COMPETITORS_MIN} (got {count}).\n"
|
||||
)
|
||||
raise SystemExit(2)
|
||||
if count > COMPETITORS_MAX:
|
||||
sys.stderr.write(
|
||||
f"[Competitors] --competitors={count} exceeds max {COMPETITORS_MAX}; clamping.\n"
|
||||
)
|
||||
count = COMPETITORS_MAX
|
||||
return True, count, []
|
||||
|
||||
|
||||
def _missing_sources_for_promo(diag: dict[str, object]) -> str | None:
|
||||
available = set(diag.get("available_sources") or [])
|
||||
missing = []
|
||||
@@ -359,24 +460,91 @@ def main() -> int:
|
||||
if "perplexity" not in include.lower():
|
||||
config["INCLUDE_SOURCES"] = f"{include},perplexity" if include else "perplexity"
|
||||
|
||||
report = pipeline.run(
|
||||
topic=topic,
|
||||
config=config,
|
||||
depth=depth,
|
||||
requested_sources=requested_sources,
|
||||
mock=args.mock,
|
||||
x_handle=args.x_handle,
|
||||
x_related=x_related,
|
||||
web_backend=args.web_backend,
|
||||
external_plan=external_plan,
|
||||
subreddits=subreddits,
|
||||
tiktok_hashtags=tiktok_hashtags,
|
||||
tiktok_creators=tiktok_creators,
|
||||
ig_creators=ig_creators,
|
||||
lookback_days=args.lookback_days,
|
||||
github_user=github_user,
|
||||
github_repos=github_repos,
|
||||
)
|
||||
comp_enabled, comp_count, comp_explicit = resolve_competitors_args(args)
|
||||
|
||||
def _main_runner() -> schema.Report:
|
||||
return pipeline.run(
|
||||
topic=topic,
|
||||
config=config,
|
||||
depth=depth,
|
||||
requested_sources=requested_sources,
|
||||
mock=args.mock,
|
||||
x_handle=args.x_handle,
|
||||
x_related=x_related,
|
||||
web_backend=args.web_backend,
|
||||
external_plan=external_plan,
|
||||
subreddits=subreddits,
|
||||
tiktok_hashtags=tiktok_hashtags,
|
||||
tiktok_creators=tiktok_creators,
|
||||
ig_creators=ig_creators,
|
||||
lookback_days=args.lookback_days,
|
||||
github_user=github_user,
|
||||
github_repos=github_repos,
|
||||
)
|
||||
|
||||
if comp_enabled:
|
||||
from lib import competitors as competitors_mod
|
||||
from lib import fanout, resolve as resolve_mod
|
||||
|
||||
if comp_explicit:
|
||||
discovered = comp_explicit
|
||||
else:
|
||||
if not resolve_mod._has_backend(config) and not args.mock:
|
||||
sys.stderr.write(
|
||||
"[Competitors] --competitors requires either a configured web "
|
||||
"search backend (BRAVE_API_KEY / EXA_API_KEY / SERPER_API_KEY / "
|
||||
"PARALLEL_API_KEY) or an explicit --competitors-list. If you are "
|
||||
"the hosting reasoning model (Claude Code, Codex, Hermes, "
|
||||
"Gemini, or any agent runtime), generate a comma-separated list "
|
||||
"of 2-6 peer entities for the topic and re-invoke with "
|
||||
"--competitors-list=\"A,B,C\". You do not need API keys; you ARE "
|
||||
"the LLM. The web-search discovery path below is the headless / "
|
||||
"credentialed path only.\n"
|
||||
)
|
||||
return 2
|
||||
discovered = competitors_mod.discover_competitors(
|
||||
topic, comp_count, config, lookback_days=args.lookback_days,
|
||||
)
|
||||
if not discovered:
|
||||
sys.stderr.write(
|
||||
f"[Competitors] No peers discovered for {topic!r}; aborting "
|
||||
"comparison run. Pass --competitors-list to override.\n"
|
||||
)
|
||||
return 2
|
||||
|
||||
sys.stderr.write(
|
||||
f"[Competitors] Comparing: {topic} vs " + " vs ".join(discovered) + "\n"
|
||||
)
|
||||
|
||||
def _competitor_runner(entity: str) -> schema.Report:
|
||||
return pipeline.run(
|
||||
topic=entity,
|
||||
config=config,
|
||||
depth=depth,
|
||||
requested_sources=requested_sources,
|
||||
mock=args.mock,
|
||||
web_backend=args.web_backend,
|
||||
lookback_days=args.lookback_days,
|
||||
)
|
||||
|
||||
entity_reports = fanout.run_competitor_fanout(
|
||||
main_topic=topic,
|
||||
main_runner=_main_runner,
|
||||
competitors=discovered,
|
||||
competitor_runner=_competitor_runner,
|
||||
)
|
||||
if len(entity_reports) < 2:
|
||||
progress.end_processing()
|
||||
sys.stderr.write(
|
||||
f"[Competitors] Fewer than 2 sub-runs survived ({len(entity_reports)}); "
|
||||
"cannot render a comparison. Re-run without --competitors or check the "
|
||||
"warnings above.\n"
|
||||
)
|
||||
return 1
|
||||
report = entity_reports[0][1]
|
||||
report.artifacts["competitor_reports"] = entity_reports
|
||||
else:
|
||||
report = _main_runner()
|
||||
except Exception as exc:
|
||||
progress.end_processing()
|
||||
progress.show_error(str(exc))
|
||||
@@ -420,7 +588,15 @@ def main() -> int:
|
||||
)
|
||||
report.artifacts["pre_research_flags_present"] = pre_research_flags_present
|
||||
|
||||
rendered = emit_output(report, args.emit, fun_level=fun_level, save_path=footer_save_path)
|
||||
entity_reports = report.artifacts.get("competitor_reports") if hasattr(report, "artifacts") else None
|
||||
if entity_reports:
|
||||
rendered = emit_comparison_output(
|
||||
entity_reports, args.emit, fun_level=fun_level, save_path=footer_save_path,
|
||||
)
|
||||
else:
|
||||
rendered = emit_output(
|
||||
report, args.emit, fun_level=fun_level, save_path=footer_save_path,
|
||||
)
|
||||
if args.save_dir:
|
||||
save_path = save_output(report, args.emit, args.save_dir, suffix=args.save_suffix or "")
|
||||
sys.stderr.write(f"[last30days] Saved output to {save_path}\n")
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
"""Discover peer entities ("competitors") for a topic via web search.
|
||||
|
||||
Mirrors the `resolve.auto_resolve()` pattern: fan out 2-3 web searches via
|
||||
`grounding.web_search()`, then extract capitalized entity candidates from
|
||||
titles and snippets with deterministic text mining. No LLM call — the
|
||||
hosting reasoning model can always override discovery via
|
||||
`--competitors-list`.
|
||||
|
||||
Returned list is ordered by score (frequency across queries) and capped to
|
||||
the caller's requested count.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import sys
|
||||
from collections import Counter
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
|
||||
from . import dates, grounding
|
||||
from .resolve import _has_backend
|
||||
|
||||
# A "brand-shaped" token starts with uppercase OR is camelCase with an
|
||||
# uppercase letter later. Catches "Anthropic", "OpenAI", "xAI", "iPhone",
|
||||
# "eBay", "Hugging", "Face".
|
||||
_BRAND_TOKEN = (
|
||||
r"(?:[A-Z][A-Za-z0-9&.\-]*"
|
||||
r"|[a-z][A-Za-z0-9&.\-]*[A-Z][A-Za-z0-9&.\-]*)"
|
||||
)
|
||||
|
||||
# A capitalized phrase of 1-4 brand tokens separated by whitespace.
|
||||
_CAPITALIZED_PHRASE = re.compile(
|
||||
rf"\b{_BRAND_TOKEN}(?:\s+{_BRAND_TOKEN}){{0,3}}\b"
|
||||
)
|
||||
|
||||
# Title-case fillers common in listicle SERPs. Kept flat — extraction
|
||||
# rejects a candidate whose entire tokens are stopwords, not candidates
|
||||
# that merely contain one.
|
||||
_STOPWORD_TOKENS: frozenset[str] = frozenset(
|
||||
token.lower()
|
||||
for token in (
|
||||
# Listicle fillers
|
||||
"Top", "Best", "Worst", "Popular", "Leading", "Similar",
|
||||
"Alternatives", "Alternative", "Competitor", "Competitors",
|
||||
"vs", "Vs", "Versus", "Review", "Reviews", "Comparison",
|
||||
"Guide", "List", "Lists", "Full", "Complete", "Free", "Paid",
|
||||
"Tools", "Tool", "Options", "Rivals", "Rival", "Similar",
|
||||
"Pick", "Picks", "Ranking", "Ranked", "Recommended",
|
||||
# Grammar / time
|
||||
"The", "A", "An", "Of", "In", "For", "To", "With", "On", "At",
|
||||
"By", "From", "Is", "Are", "And", "Or", "But", "Than", "As",
|
||||
"This", "That", "These", "Those", "Our", "Your", "Their",
|
||||
"January", "February", "March", "April", "May", "June", "July",
|
||||
"August", "September", "October", "November", "December",
|
||||
# Years likely to appear as standalone tokens
|
||||
*(str(year) for year in range(2018, 2031)),
|
||||
# Miscellaneous SERP noise
|
||||
"AI", "Apps", "App", "Software", "Platform", "Service", "Startups",
|
||||
"Companies", "Company", "Products", "Product", "Brands", "Brand",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _log(msg: str) -> None:
|
||||
print(f"[Competitors] {msg}", file=sys.stderr)
|
||||
|
||||
|
||||
def _topic_tokens(topic: str) -> set[str]:
|
||||
"""Return lowercase alphanumeric tokens of the topic for filtering."""
|
||||
return {tok for tok in re.findall(r"[A-Za-z0-9]+", topic.lower()) if tok}
|
||||
|
||||
|
||||
def _candidate_ok(candidate: str, topic_tokens: set[str]) -> bool:
|
||||
"""Filter a candidate phrase against stopwords and topic overlap."""
|
||||
tokens = [t for t in re.findall(r"[A-Za-z0-9&.\-]+", candidate) if t]
|
||||
if not tokens:
|
||||
return False
|
||||
# Reject candidates made entirely of stopwords (e.g., "Top Alternatives").
|
||||
if all(tok.lower() in _STOPWORD_TOKENS for tok in tokens):
|
||||
return False
|
||||
# Reject candidates that overlap with the topic (e.g., topic="OpenAI"
|
||||
# should not return "OpenAI Alternatives" or "OpenAI").
|
||||
lower_tokens = {tok.lower() for tok in tokens}
|
||||
if lower_tokens & topic_tokens:
|
||||
return False
|
||||
# Reject too-short one-letter tokens like "I" or single digits.
|
||||
if len(tokens) == 1 and len(tokens[0]) < 2:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _normalize_candidate(candidate: str) -> str:
|
||||
"""Collapse whitespace and strip trailing punctuation."""
|
||||
return re.sub(r"\s+", " ", candidate).strip(".,;:!?'\"()[] ")
|
||||
|
||||
|
||||
def _extract_peer_entities(
|
||||
items: list[dict], topic: str, limit: int,
|
||||
) -> list[str]:
|
||||
"""Score capitalized candidates across SERP items and return top `limit`.
|
||||
|
||||
Scoring is bag-of-phrases frequency across all items in the input. Ties
|
||||
are broken by first-seen order so the output is deterministic.
|
||||
"""
|
||||
topic_tokens = _topic_tokens(topic)
|
||||
counts: Counter[str] = Counter()
|
||||
first_seen: dict[str, int] = {}
|
||||
order = 0
|
||||
# Group candidates into a frequency map keyed by lowercased normalized
|
||||
# form so "xAI" and "xAI" count together regardless of case.
|
||||
canonical: dict[str, str] = {}
|
||||
for item in items:
|
||||
text = f"{item.get('title', '')} {item.get('snippet', '')}"
|
||||
for raw in _CAPITALIZED_PHRASE.findall(text):
|
||||
candidate = _normalize_candidate(raw)
|
||||
if not _candidate_ok(candidate, topic_tokens):
|
||||
continue
|
||||
key = candidate.lower()
|
||||
if key not in canonical:
|
||||
canonical[key] = candidate
|
||||
first_seen[key] = order
|
||||
order += 1
|
||||
counts[key] += 1
|
||||
|
||||
ranked_keys = sorted(
|
||||
counts.keys(),
|
||||
key=lambda k: (-counts[k], first_seen[k]),
|
||||
)
|
||||
return [canonical[k] for k in ranked_keys[:limit]]
|
||||
|
||||
|
||||
def _queries_for(topic: str) -> dict[str, str]:
|
||||
return {
|
||||
"competitors": f"{topic} competitors",
|
||||
"alternatives": f"{topic} alternatives",
|
||||
"vs": f"{topic} vs",
|
||||
}
|
||||
|
||||
|
||||
def discover_competitors(
|
||||
topic: str,
|
||||
count: int,
|
||||
config: dict,
|
||||
*,
|
||||
lookback_days: int = 30,
|
||||
) -> list[str]:
|
||||
"""Discover `count` peer entities for `topic` via web search.
|
||||
|
||||
Args:
|
||||
topic: The primary research topic.
|
||||
count: Desired number of competitor entities (1..N).
|
||||
config: Runtime config dict — expects the same shape as the engine
|
||||
config (BRAVE_API_KEY / EXA_API_KEY / SERPER_API_KEY / etc.).
|
||||
lookback_days: Date range for freshness. Defaults to 30.
|
||||
|
||||
Returns:
|
||||
A list of up to `count` entity names, deduped and ordered by score.
|
||||
Empty list when no web backend is configured or every search fails
|
||||
or returns zero usable candidates.
|
||||
"""
|
||||
if count < 1:
|
||||
return []
|
||||
if not _has_backend(config):
|
||||
_log("No web search backend available, skipping competitor discovery")
|
||||
return []
|
||||
|
||||
date_range = dates.get_date_range(lookback_days)
|
||||
queries = _queries_for(topic)
|
||||
collected: list[dict] = []
|
||||
searches_run = 0
|
||||
|
||||
def _search(label: str, query: str) -> tuple[str, list[dict]]:
|
||||
items, _artifact = grounding.web_search(query, date_range, config)
|
||||
return label, items
|
||||
|
||||
with ThreadPoolExecutor(max_workers=len(queries)) as executor:
|
||||
futures = {
|
||||
executor.submit(_search, label, q): label
|
||||
for label, q in queries.items()
|
||||
}
|
||||
for future in as_completed(futures):
|
||||
label = futures[future]
|
||||
try:
|
||||
_label, items = future.result()
|
||||
collected.extend(items)
|
||||
searches_run += 1
|
||||
except Exception as exc:
|
||||
_log(f"Search failed for {label}: {exc}")
|
||||
|
||||
if not collected:
|
||||
_log(f"No SERP results for {topic!r} across {searches_run}/{len(queries)} queries")
|
||||
return []
|
||||
|
||||
entities = _extract_peer_entities(collected, topic, limit=count)
|
||||
_log(
|
||||
f"Discovered {len(entities)} competitor(s) for {topic!r} "
|
||||
f"from {searches_run}/{len(queries)} queries: {entities}"
|
||||
)
|
||||
return entities
|
||||
@@ -0,0 +1,85 @@
|
||||
"""Parallel multi-entity fan-out for the --competitors flag.
|
||||
|
||||
The orchestrator accepts a `main_runner()` for the topic and a
|
||||
`competitor_runner(entity)` for each peer. It parallelizes their execution
|
||||
via a `ThreadPoolExecutor` and collects per-entity Reports. Per-entity
|
||||
failures are logged and dropped; the run survives as long as the main topic
|
||||
plus at least one competitor succeed.
|
||||
|
||||
This module owns no business logic about pipeline arguments — the caller
|
||||
(scripts/last30days.py main) builds the closures with the appropriate
|
||||
config, depth, and overrides for each entity.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from typing import Callable
|
||||
|
||||
from . import schema
|
||||
|
||||
# Sub-runs hit the same upstream APIs as the main topic. Cap parallelism so a
|
||||
# 6-way fan-out does not stampede a single backend's rate limit.
|
||||
MAX_PARALLEL_SUBRUNS = 6
|
||||
|
||||
|
||||
def _log(msg: str) -> None:
|
||||
print(f"[Fanout] {msg}", file=sys.stderr)
|
||||
|
||||
|
||||
def run_competitor_fanout(
|
||||
*,
|
||||
main_topic: str,
|
||||
main_runner: Callable[[], schema.Report],
|
||||
competitors: list[str],
|
||||
competitor_runner: Callable[[str], schema.Report],
|
||||
) -> list[tuple[str, schema.Report]]:
|
||||
"""Run main + competitor pipelines in parallel; return surviving reports.
|
||||
|
||||
Args:
|
||||
main_topic: Display label for the user's primary topic.
|
||||
main_runner: Zero-arg callable returning the main topic's Report.
|
||||
competitors: Ordered list of competitor entity names.
|
||||
competitor_runner: Callable(entity_name) -> Report for each peer.
|
||||
|
||||
Returns:
|
||||
Ordered list of (entity_name, Report) tuples for runs that succeeded.
|
||||
Empty list if every run raised; the caller decides how to surface
|
||||
partial-failure modes.
|
||||
"""
|
||||
if not competitors:
|
||||
report = main_runner()
|
||||
return [(main_topic, report)]
|
||||
|
||||
workers = min(len(competitors) + 1, MAX_PARALLEL_SUBRUNS)
|
||||
|
||||
def _run_one(label: str, fn: Callable[[], schema.Report]) -> tuple[str, schema.Report | None, Exception | None]:
|
||||
try:
|
||||
return label, fn(), None
|
||||
except Exception as exc:
|
||||
return label, None, exc
|
||||
|
||||
submissions: list[tuple[str, Callable[[], schema.Report]]] = [
|
||||
(main_topic, main_runner),
|
||||
]
|
||||
for entity in competitors:
|
||||
submissions.append((entity, lambda e=entity: competitor_runner(e)))
|
||||
|
||||
with ThreadPoolExecutor(max_workers=workers) as executor:
|
||||
futures = {
|
||||
executor.submit(_run_one, label, fn): label
|
||||
for label, fn in submissions
|
||||
}
|
||||
results: dict[str, schema.Report] = {}
|
||||
for future in as_completed(futures):
|
||||
label, report, exc = future.result()
|
||||
if exc is not None:
|
||||
_log(f"Sub-run failed for {label!r}: {type(exc).__name__}: {exc}")
|
||||
continue
|
||||
assert report is not None
|
||||
results[label] = report
|
||||
|
||||
# Preserve the original submission order rather than completion order so
|
||||
# the comparison render is deterministic across runs.
|
||||
return [(label, results[label]) for label, _ in submissions if label in results]
|
||||
@@ -392,6 +392,165 @@ def _render_comparison_scaffold(topic: str) -> list[str]:
|
||||
]
|
||||
|
||||
|
||||
def render_comparison_multi(
|
||||
entity_reports: list[tuple[str, schema.Report]],
|
||||
*,
|
||||
cluster_limit: int = 4,
|
||||
fun_level: str = "medium",
|
||||
save_path: str | None = None,
|
||||
) -> str:
|
||||
"""Render N (entity, Report) pairs as a single comparison output.
|
||||
|
||||
Reuses _render_comparison_scaffold for the synthesis table and emits
|
||||
per-entity evidence sections inside one EVIDENCE FOR SYNTHESIS envelope.
|
||||
The single-Report render_compact path is unchanged.
|
||||
|
||||
Args:
|
||||
entity_reports: Ordered (label, Report) pairs. The first pair is the
|
||||
user's main topic; the remainder are discovered/explicit competitors.
|
||||
cluster_limit: Max clusters to surface per entity (kept lower than the
|
||||
single-entity default to keep N-way comparisons readable).
|
||||
fun_level: Same fun-level knob as render_compact, applied to each
|
||||
entity's best-takes block.
|
||||
save_path: Optional save-path display string for the footer.
|
||||
"""
|
||||
if not entity_reports:
|
||||
raise ValueError("render_comparison_multi requires at least one report")
|
||||
|
||||
entities = [label for label, _ in entity_reports]
|
||||
main_label, main_report = entity_reports[0]
|
||||
synthesized_topic = " vs ".join(entities)
|
||||
|
||||
lines: list[str] = [
|
||||
*_render_badge(),
|
||||
f"# last30days v3.0.0: {synthesized_topic}",
|
||||
"",
|
||||
*_assistant_safety_lines(),
|
||||
f"- Comparison mode: {len(entities)} entities ({', '.join(entities)})",
|
||||
f"- Date range: {main_report.range_from} to {main_report.range_to}",
|
||||
"",
|
||||
]
|
||||
|
||||
aggregated_warnings: list[str] = []
|
||||
for label, report in entity_reports:
|
||||
aggregated_warnings.extend(f"[{label}] {w}" for w in report.warnings)
|
||||
if aggregated_warnings:
|
||||
lines.append("## Warnings")
|
||||
lines.extend(f"- {w}" for w in aggregated_warnings)
|
||||
lines.append("")
|
||||
|
||||
lines.append(
|
||||
"<!-- EVIDENCE FOR SYNTHESIS: read this, do not emit verbatim. Transform into "
|
||||
"`What I learned:` prose per LAW 2. Each entity has its own evidence subsection. -->"
|
||||
)
|
||||
lines.append("")
|
||||
|
||||
fun_params = _FUN_LEVELS.get(fun_level, _FUN_LEVELS["medium"])
|
||||
for label, report in entity_reports:
|
||||
lines.extend(_render_entity_evidence_block(
|
||||
label=label,
|
||||
report=report,
|
||||
cluster_limit=cluster_limit,
|
||||
fun_params=fun_params,
|
||||
))
|
||||
|
||||
lines.append("<!-- END EVIDENCE FOR SYNTHESIS -->")
|
||||
lines.append("")
|
||||
|
||||
# Reuse the existing comparison scaffold by feeding it the synthesized
|
||||
# topic. _parse_comparison_entities splits on " vs " so the scaffold
|
||||
# picks up all N entities automatically.
|
||||
scaffold = _render_comparison_scaffold(synthesized_topic)
|
||||
lines.extend(scaffold)
|
||||
|
||||
footer = _render_emoji_footer(main_report, save_path)
|
||||
if footer:
|
||||
lines.append("")
|
||||
lines.append("<!-- PASS-THROUGH FOOTER: emit verbatim in the model response per LAW 5. -->")
|
||||
lines.extend(footer)
|
||||
lines.append("<!-- END PASS-THROUGH FOOTER -->")
|
||||
|
||||
lines.extend(_render_canonical_boundary())
|
||||
|
||||
return "\n".join(lines).strip() + "\n"
|
||||
|
||||
|
||||
def _render_entity_evidence_block(
|
||||
*,
|
||||
label: str,
|
||||
report: schema.Report,
|
||||
cluster_limit: int,
|
||||
fun_params: dict,
|
||||
) -> list[str]:
|
||||
"""Render one entity's clusters and best-takes inside the evidence envelope."""
|
||||
candidate_by_id = {c.candidate_id: c for c in report.ranked_candidates}
|
||||
out: list[str] = [f"## {label}", ""]
|
||||
|
||||
if not report.clusters:
|
||||
out.append("(no significant discussion this month)")
|
||||
out.append("")
|
||||
return out
|
||||
|
||||
out.append("### Ranked Evidence Clusters")
|
||||
out.append("")
|
||||
for index, cluster in enumerate(report.clusters[:cluster_limit], start=1):
|
||||
out.append(
|
||||
f"#### {index}. {cluster.title} "
|
||||
f"(score {cluster.score:.0f}, {len(cluster.candidate_ids)} item"
|
||||
f"{'s' if len(cluster.candidate_ids) != 1 else ''}, "
|
||||
f"sources: {', '.join(_source_label(s) for s in cluster.sources)})"
|
||||
)
|
||||
if cluster.uncertainty:
|
||||
out.append(f"- Uncertainty: {cluster.uncertainty}")
|
||||
for rep_index, candidate_id in enumerate(cluster.representative_ids, start=1):
|
||||
candidate = candidate_by_id.get(candidate_id)
|
||||
if not candidate:
|
||||
continue
|
||||
out.extend(_render_candidate(candidate, prefix=f"{rep_index}."))
|
||||
out.append("")
|
||||
|
||||
best_takes = _render_best_takes(
|
||||
report.ranked_candidates,
|
||||
limit=fun_params["limit"],
|
||||
threshold=fun_params["threshold"],
|
||||
)
|
||||
if best_takes:
|
||||
out.extend(best_takes)
|
||||
out.append("")
|
||||
|
||||
return out
|
||||
|
||||
|
||||
def render_comparison_multi_context(
|
||||
entity_reports: list[tuple[str, schema.Report]],
|
||||
cluster_limit: int = 4,
|
||||
) -> str:
|
||||
"""Context-mode rendering for the multi-entity comparison."""
|
||||
if not entity_reports:
|
||||
raise ValueError("render_comparison_multi_context requires at least one report")
|
||||
|
||||
entities = [label for label, _ in entity_reports]
|
||||
lines = [
|
||||
f"Comparison: {' vs '.join(entities)}",
|
||||
f"Entities: {len(entities)}",
|
||||
_AI_SAFETY_NOTE,
|
||||
"",
|
||||
]
|
||||
for label, report in entity_reports:
|
||||
lines.append(f"## {label}")
|
||||
lines.append(f"Intent: {report.query_plan.intent}")
|
||||
if not report.clusters:
|
||||
lines.append("- (no significant discussion this month)")
|
||||
else:
|
||||
for cluster in report.clusters[:cluster_limit]:
|
||||
lines.append(
|
||||
f"- {cluster.title} "
|
||||
f"[{', '.join(_source_label(s) for s in cluster.sources)}]"
|
||||
)
|
||||
lines.append("")
|
||||
return "\n".join(lines).strip() + "\n"
|
||||
|
||||
|
||||
def render_full(report: schema.Report) -> str:
|
||||
"""Full data dump: ALL clusters + ALL items by source. For saved files and debugging."""
|
||||
# Start with the same header as compact
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
# ruff: noqa: E402
|
||||
"""CLI parsing and validation for --competitors / --competitors-list."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import sys
|
||||
import unittest
|
||||
from contextlib import redirect_stderr
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(REPO_ROOT / "scripts"))
|
||||
|
||||
import last30days as cli
|
||||
|
||||
|
||||
def _parse(*argv: str):
|
||||
parser = cli.build_parser()
|
||||
args, _extra = parser.parse_known_args(argv)
|
||||
return args
|
||||
|
||||
|
||||
class CompetitorsCliTests(unittest.TestCase):
|
||||
def test_flag_absent_returns_disabled(self):
|
||||
args = _parse("Kanye West")
|
||||
enabled, count, explicit = cli.resolve_competitors_args(args)
|
||||
self.assertFalse(enabled)
|
||||
self.assertEqual(count, 0)
|
||||
self.assertEqual(explicit, [])
|
||||
|
||||
def test_bare_flag_defaults_to_three(self):
|
||||
args = _parse("Kanye West", "--competitors")
|
||||
enabled, count, explicit = cli.resolve_competitors_args(args)
|
||||
self.assertTrue(enabled)
|
||||
self.assertEqual(count, 3)
|
||||
self.assertEqual(explicit, [])
|
||||
|
||||
def test_explicit_count(self):
|
||||
args = _parse("OpenAI", "--competitors", "4")
|
||||
enabled, count, explicit = cli.resolve_competitors_args(args)
|
||||
self.assertTrue(enabled)
|
||||
self.assertEqual(count, 4)
|
||||
self.assertEqual(explicit, [])
|
||||
|
||||
def test_explicit_list_preferred_over_discovery(self):
|
||||
args = _parse(
|
||||
"OpenAI",
|
||||
"--competitors",
|
||||
"--competitors-list",
|
||||
"Anthropic,xAI,Google Gemini",
|
||||
)
|
||||
enabled, count, explicit = cli.resolve_competitors_args(args)
|
||||
self.assertTrue(enabled)
|
||||
self.assertEqual(count, 3)
|
||||
self.assertEqual(explicit, ["Anthropic", "xAI", "Google Gemini"])
|
||||
|
||||
def test_explicit_list_without_flag_implies_enabled(self):
|
||||
args = _parse("OpenAI", "--competitors-list", "Anthropic,xAI")
|
||||
enabled, count, explicit = cli.resolve_competitors_args(args)
|
||||
self.assertTrue(enabled)
|
||||
self.assertEqual(count, 2)
|
||||
self.assertEqual(explicit, ["Anthropic", "xAI"])
|
||||
|
||||
def test_list_whitespace_normalized(self):
|
||||
args = _parse("OpenAI", "--competitors-list", " Anthropic , xAI , Gemini ")
|
||||
_enabled, count, explicit = cli.resolve_competitors_args(args)
|
||||
self.assertEqual(count, 3)
|
||||
self.assertEqual(explicit, ["Anthropic", "xAI", "Gemini"])
|
||||
|
||||
def test_zero_count_rejected(self):
|
||||
args = _parse("Topic", "--competitors", "0")
|
||||
with self.assertRaises(SystemExit) as cm, redirect_stderr(io.StringIO()) as err:
|
||||
cli.resolve_competitors_args(args)
|
||||
self.assertEqual(cm.exception.code, 2)
|
||||
self.assertIn("--competitors must be >= 1", err.getvalue())
|
||||
|
||||
def test_negative_count_rejected(self):
|
||||
args = _parse("Topic", "--competitors", "-1")
|
||||
with self.assertRaises(SystemExit), redirect_stderr(io.StringIO()):
|
||||
cli.resolve_competitors_args(args)
|
||||
|
||||
def test_over_max_count_clamps_with_warning(self):
|
||||
args = _parse("Topic", "--competitors", "99")
|
||||
err = io.StringIO()
|
||||
with redirect_stderr(err):
|
||||
enabled, count, explicit = cli.resolve_competitors_args(args)
|
||||
self.assertTrue(enabled)
|
||||
self.assertEqual(count, cli.COMPETITORS_MAX)
|
||||
self.assertEqual(explicit, [])
|
||||
self.assertIn("clamping", err.getvalue())
|
||||
|
||||
def test_overlong_list_clamps_with_warning(self):
|
||||
args = _parse(
|
||||
"Topic",
|
||||
"--competitors-list",
|
||||
"A,B,C,D,E,F,G,H",
|
||||
)
|
||||
err = io.StringIO()
|
||||
with redirect_stderr(err):
|
||||
enabled, count, explicit = cli.resolve_competitors_args(args)
|
||||
self.assertTrue(enabled)
|
||||
self.assertEqual(count, cli.COMPETITORS_MAX)
|
||||
self.assertEqual(len(explicit), cli.COMPETITORS_MAX)
|
||||
self.assertIn("clamping to", err.getvalue())
|
||||
|
||||
def test_list_count_mismatch_warns(self):
|
||||
args = _parse(
|
||||
"Topic",
|
||||
"--competitors",
|
||||
"5",
|
||||
"--competitors-list",
|
||||
"A,B",
|
||||
)
|
||||
err = io.StringIO()
|
||||
with redirect_stderr(err):
|
||||
enabled, count, explicit = cli.resolve_competitors_args(args)
|
||||
self.assertTrue(enabled)
|
||||
self.assertEqual(count, 2)
|
||||
self.assertEqual(explicit, ["A", "B"])
|
||||
self.assertIn("--competitors=5 ignored", err.getvalue())
|
||||
|
||||
def test_empty_list_rejected(self):
|
||||
args = _parse("Topic", "--competitors-list", ",, ,")
|
||||
with self.assertRaises(SystemExit) as cm, redirect_stderr(io.StringIO()):
|
||||
cli.resolve_competitors_args(args)
|
||||
self.assertEqual(cm.exception.code, 2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,160 @@
|
||||
# ruff: noqa: E402
|
||||
"""Tests for scripts/lib/fanout.run_competitor_fanout."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from contextlib import redirect_stderr
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(REPO_ROOT / "scripts"))
|
||||
|
||||
from lib import fanout
|
||||
|
||||
|
||||
def _fake_report(topic: str):
|
||||
"""Build a lightweight Report stand-in. Tests only check identity."""
|
||||
class _R:
|
||||
pass
|
||||
|
||||
r = _R()
|
||||
r.topic = topic
|
||||
return r
|
||||
|
||||
|
||||
class FanoutOrchestratorTests(unittest.TestCase):
|
||||
def test_main_plus_two_competitors_all_succeed(self):
|
||||
def main_runner():
|
||||
return _fake_report("OpenAI")
|
||||
|
||||
def comp_runner(entity):
|
||||
return _fake_report(entity)
|
||||
|
||||
err = io.StringIO()
|
||||
with redirect_stderr(err):
|
||||
results = fanout.run_competitor_fanout(
|
||||
main_topic="OpenAI",
|
||||
main_runner=main_runner,
|
||||
competitors=["Anthropic", "xAI"],
|
||||
competitor_runner=comp_runner,
|
||||
)
|
||||
labels = [label for label, _ in results]
|
||||
self.assertEqual(labels, ["OpenAI", "Anthropic", "xAI"])
|
||||
self.assertEqual(results[0][1].topic, "OpenAI")
|
||||
self.assertEqual(results[1][1].topic, "Anthropic")
|
||||
|
||||
def test_one_competitor_failure_degrades_gracefully(self):
|
||||
def main_runner():
|
||||
return _fake_report("OpenAI")
|
||||
|
||||
def comp_runner(entity):
|
||||
if entity == "BrokenCo":
|
||||
raise RuntimeError("upstream offline")
|
||||
return _fake_report(entity)
|
||||
|
||||
err = io.StringIO()
|
||||
with redirect_stderr(err):
|
||||
results = fanout.run_competitor_fanout(
|
||||
main_topic="OpenAI",
|
||||
main_runner=main_runner,
|
||||
competitors=["Anthropic", "BrokenCo", "xAI"],
|
||||
competitor_runner=comp_runner,
|
||||
)
|
||||
labels = [label for label, _ in results]
|
||||
self.assertEqual(labels, ["OpenAI", "Anthropic", "xAI"])
|
||||
self.assertIn("BrokenCo", err.getvalue())
|
||||
self.assertIn("upstream offline", err.getvalue())
|
||||
|
||||
def test_main_topic_failure_leaves_only_competitors(self):
|
||||
def main_runner():
|
||||
raise RuntimeError("main exploded")
|
||||
|
||||
def comp_runner(entity):
|
||||
return _fake_report(entity)
|
||||
|
||||
err = io.StringIO()
|
||||
with redirect_stderr(err):
|
||||
results = fanout.run_competitor_fanout(
|
||||
main_topic="OpenAI",
|
||||
main_runner=main_runner,
|
||||
competitors=["Anthropic", "xAI"],
|
||||
competitor_runner=comp_runner,
|
||||
)
|
||||
labels = [label for label, _ in results]
|
||||
self.assertEqual(labels, ["Anthropic", "xAI"])
|
||||
self.assertIn("main exploded", err.getvalue())
|
||||
|
||||
def test_empty_competitor_list_runs_only_main(self):
|
||||
def main_runner():
|
||||
return _fake_report("OpenAI")
|
||||
|
||||
def comp_runner(_entity):
|
||||
raise AssertionError("should not be called when competitors=[]")
|
||||
|
||||
err = io.StringIO()
|
||||
with redirect_stderr(err):
|
||||
results = fanout.run_competitor_fanout(
|
||||
main_topic="OpenAI",
|
||||
main_runner=main_runner,
|
||||
competitors=[],
|
||||
competitor_runner=comp_runner,
|
||||
)
|
||||
self.assertEqual([label for label, _ in results], ["OpenAI"])
|
||||
|
||||
def test_sub_runs_execute_in_parallel(self):
|
||||
"""Wall clock should be closer to max(latency) than sum(latency)."""
|
||||
delay = 0.2
|
||||
call_count = 3 # main + 2 competitors
|
||||
|
||||
def make_runner(_label):
|
||||
def runner():
|
||||
time.sleep(delay)
|
||||
return _fake_report(_label)
|
||||
return runner
|
||||
|
||||
def comp_runner(entity):
|
||||
return make_runner(entity)()
|
||||
|
||||
start = time.monotonic()
|
||||
with redirect_stderr(io.StringIO()):
|
||||
results = fanout.run_competitor_fanout(
|
||||
main_topic="OpenAI",
|
||||
main_runner=make_runner("OpenAI"),
|
||||
competitors=["Anthropic", "xAI"],
|
||||
competitor_runner=comp_runner,
|
||||
)
|
||||
elapsed = time.monotonic() - start
|
||||
self.assertEqual(len(results), 3)
|
||||
# Generous margin: parallel execution should finish well under
|
||||
# sum(call_count * delay) == 0.6s. We accept anything under 0.5s.
|
||||
self.assertLess(
|
||||
elapsed, delay * call_count,
|
||||
f"Expected parallel execution < {delay * call_count:.2f}s, "
|
||||
f"got {elapsed:.2f}s (sub-runs likely serialized)",
|
||||
)
|
||||
|
||||
def test_all_competitors_fail_leaves_main_only(self):
|
||||
def main_runner():
|
||||
return _fake_report("OpenAI")
|
||||
|
||||
def comp_runner(_entity):
|
||||
raise RuntimeError("all offline")
|
||||
|
||||
with redirect_stderr(io.StringIO()):
|
||||
results = fanout.run_competitor_fanout(
|
||||
main_topic="OpenAI",
|
||||
main_runner=main_runner,
|
||||
competitors=["A", "B", "C"],
|
||||
competitor_runner=comp_runner,
|
||||
)
|
||||
self.assertEqual([label for label, _ in results], ["OpenAI"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,152 @@
|
||||
# ruff: noqa: E402
|
||||
"""Tests for scripts/lib/competitors.discover_competitors."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import sys
|
||||
import unittest
|
||||
from contextlib import redirect_stderr
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(REPO_ROOT / "scripts"))
|
||||
|
||||
from lib import competitors
|
||||
|
||||
|
||||
def _serp(items: list[tuple[str, str]]) -> list[dict]:
|
||||
"""Build a minimal SERP items list from (title, snippet) pairs."""
|
||||
return [
|
||||
{"title": title, "snippet": snippet, "url": "https://example.test/"}
|
||||
for title, snippet in items
|
||||
]
|
||||
|
||||
|
||||
OPENAI_SERP = _serp(
|
||||
[
|
||||
("OpenAI vs Anthropic vs xAI: which is better?", "xAI and Anthropic now compete directly with OpenAI."),
|
||||
("Top OpenAI alternatives in 2026", "Anthropic, Google Gemini, and xAI are the leading alternatives this year."),
|
||||
("xAI and Anthropic challenge OpenAI dominance", "xAI and Anthropic push Google Gemini hard; xAI keeps shipping."),
|
||||
("Anthropic vs xAI: head to head", "Anthropic and xAI trade punches; Google Gemini is not far behind."),
|
||||
]
|
||||
)
|
||||
|
||||
KANYE_SERP = _serp(
|
||||
[
|
||||
("Kanye West vs Drake: the feud explained", "Drake responded to Kanye with a diss track."),
|
||||
("Top rappers of the decade: Kendrick Lamar, Drake, J Cole", "Kendrick Lamar released a new album; Drake toured Europe."),
|
||||
("Drake and Kendrick Lamar trade shots", "J Cole stayed out of the Drake vs Kendrick Lamar feud."),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
class CompetitorDiscoveryTests(unittest.TestCase):
|
||||
def _run(self, serp: list[dict], topic: str, count: int = 3) -> list[str]:
|
||||
config = {"BRAVE_API_KEY": "test-key"}
|
||||
with mock.patch.object(
|
||||
competitors.grounding, "web_search", return_value=(serp, {})
|
||||
):
|
||||
with redirect_stderr(io.StringIO()):
|
||||
return competitors.discover_competitors(topic, count, config)
|
||||
|
||||
def test_openai_surfaces_anthropic_and_peers(self):
|
||||
results = self._run(OPENAI_SERP, "OpenAI", count=3)
|
||||
self.assertEqual(len(results), 3)
|
||||
joined = " ".join(results)
|
||||
self.assertIn("Anthropic", joined)
|
||||
self.assertIn("xAI", joined)
|
||||
# Should not surface the topic itself
|
||||
self.assertNotIn("OpenAI", results)
|
||||
self.assertFalse(
|
||||
any("OpenAI" in entity for entity in results),
|
||||
f"Topic token leaked into results: {results}",
|
||||
)
|
||||
|
||||
def test_kanye_surfaces_rap_peers(self):
|
||||
results = self._run(KANYE_SERP, "Kanye West", count=2)
|
||||
self.assertEqual(len(results), 2)
|
||||
joined = " ".join(results)
|
||||
self.assertTrue(
|
||||
"Drake" in joined and "Kendrick Lamar" in joined,
|
||||
f"Expected Drake and Kendrick Lamar in {results}",
|
||||
)
|
||||
|
||||
def test_empty_serp_returns_empty(self):
|
||||
results = self._run([], "OpenAI", count=3)
|
||||
self.assertEqual(results, [])
|
||||
|
||||
def test_no_backend_returns_empty(self):
|
||||
err = io.StringIO()
|
||||
with redirect_stderr(err):
|
||||
results = competitors.discover_competitors("OpenAI", 3, config={})
|
||||
self.assertEqual(results, [])
|
||||
self.assertIn("No web search backend", err.getvalue())
|
||||
|
||||
def test_backend_error_returns_empty(self):
|
||||
config = {"BRAVE_API_KEY": "test-key"}
|
||||
|
||||
def boom(*_args, **_kwargs):
|
||||
raise RuntimeError("SERP provider offline")
|
||||
|
||||
err = io.StringIO()
|
||||
with mock.patch.object(competitors.grounding, "web_search", side_effect=boom):
|
||||
with redirect_stderr(err):
|
||||
results = competitors.discover_competitors("OpenAI", 3, config)
|
||||
self.assertEqual(results, [])
|
||||
self.assertIn("Search failed", err.getvalue())
|
||||
|
||||
def test_topic_tokens_filtered(self):
|
||||
"""Candidates overlapping topic tokens are rejected."""
|
||||
serp = _serp(
|
||||
[
|
||||
("Open AI vs Anthropic", "Open AI, Anthropic, and Google lead."),
|
||||
("OpenAI Alternatives: Anthropic", "Anthropic is a competitor to Open AI."),
|
||||
]
|
||||
)
|
||||
results = self._run(serp, "OpenAI", count=3)
|
||||
# "Open AI" shares the "openai" lowercased-concatenation? Actually tokenizer
|
||||
# splits "Open AI" into ["open", "ai"]. Topic "OpenAI" tokenizes to ["openai"].
|
||||
# They do not overlap at the token level, which is fine — the filter is
|
||||
# best-effort. We only assert that bare "OpenAI" is filtered and real
|
||||
# competitors still surface.
|
||||
self.assertNotIn("OpenAI", results)
|
||||
self.assertIn("Anthropic", results)
|
||||
|
||||
def test_deduplicates_case_insensitively(self):
|
||||
serp = _serp(
|
||||
[
|
||||
("Anthropic vs Gemini", "anthropic is strong."),
|
||||
("ANTHROPIC makes Claude", "Anthropic announced Claude 4."),
|
||||
]
|
||||
)
|
||||
results = self._run(serp, "OpenAI", count=3)
|
||||
# "Anthropic" should appear exactly once (first-seen capitalization wins).
|
||||
anthropic_matches = [r for r in results if r.lower() == "anthropic"]
|
||||
self.assertEqual(len(anthropic_matches), 1)
|
||||
|
||||
def test_count_one_returns_single(self):
|
||||
results = self._run(OPENAI_SERP, "OpenAI", count=1)
|
||||
self.assertEqual(len(results), 1)
|
||||
|
||||
def test_stopword_only_candidates_rejected(self):
|
||||
serp = _serp(
|
||||
[
|
||||
("Top Alternatives", "Best Competitors and Top Tools."),
|
||||
("Free Software Reviews", "Complete Guide to The Options."),
|
||||
]
|
||||
)
|
||||
results = self._run(serp, "Widget", count=5)
|
||||
self.assertEqual(
|
||||
results, [],
|
||||
f"Stopword-only phrases should not be returned: got {results}",
|
||||
)
|
||||
|
||||
def test_count_zero_returns_empty(self):
|
||||
results = self._run(OPENAI_SERP, "OpenAI", count=0)
|
||||
self.assertEqual(results, [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,201 @@
|
||||
# ruff: noqa: E402
|
||||
"""Tests for render.render_comparison_multi and emit_comparison_output."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
sys.path.insert(0, str(REPO_ROOT / "scripts"))
|
||||
|
||||
import last30days as cli
|
||||
from lib import render, schema
|
||||
|
||||
|
||||
def _build_report(topic: str, cluster_titles: list[str]) -> schema.Report:
|
||||
query_plan = schema.QueryPlan(
|
||||
intent="comparison",
|
||||
freshness_mode="balanced_recent",
|
||||
cluster_mode="debate",
|
||||
raw_topic=topic,
|
||||
subqueries=[
|
||||
schema.SubQuery(
|
||||
label="primary",
|
||||
search_query=topic,
|
||||
ranking_query=topic,
|
||||
sources=["grounding"],
|
||||
)
|
||||
],
|
||||
source_weights={"grounding": 1.0},
|
||||
)
|
||||
clusters: list[schema.Cluster] = []
|
||||
candidates: list[schema.Candidate] = []
|
||||
for idx, title in enumerate(cluster_titles):
|
||||
candidate_id = f"{topic.lower().replace(' ', '-')}-c{idx}"
|
||||
item = schema.SourceItem(
|
||||
source="grounding",
|
||||
item_id=f"g-{candidate_id}",
|
||||
title=f"{title} evidence",
|
||||
body=f"Body for {title}",
|
||||
url=f"https://example.test/{candidate_id}",
|
||||
snippet=f"Snippet for {title}",
|
||||
published_at="2026-04-20",
|
||||
)
|
||||
candidate = schema.Candidate(
|
||||
candidate_id=candidate_id,
|
||||
item_id=item.item_id,
|
||||
source="grounding",
|
||||
title=item.title,
|
||||
url=item.url,
|
||||
snippet=item.snippet,
|
||||
subquery_labels=["primary"],
|
||||
native_ranks={"grounding": idx + 1},
|
||||
local_relevance=0.8 - idx * 0.1,
|
||||
freshness=5,
|
||||
engagement=10,
|
||||
source_quality=0.9,
|
||||
rrf_score=0.6 - idx * 0.05,
|
||||
sources=["grounding"],
|
||||
source_items=[item],
|
||||
final_score=80.0 - idx * 5,
|
||||
)
|
||||
candidates.append(candidate)
|
||||
clusters.append(
|
||||
schema.Cluster(
|
||||
cluster_id=f"cl-{idx}",
|
||||
title=title,
|
||||
candidate_ids=[candidate_id],
|
||||
representative_ids=[candidate_id],
|
||||
score=80.0 - idx * 5,
|
||||
sources=["grounding"],
|
||||
)
|
||||
)
|
||||
return schema.Report(
|
||||
topic=topic,
|
||||
range_from="2026-03-23",
|
||||
range_to="2026-04-22",
|
||||
generated_at="2026-04-22T00:00:00+00:00",
|
||||
provider_runtime=schema.ProviderRuntime(
|
||||
reasoning_provider="mock",
|
||||
planner_model="mock-planner",
|
||||
rerank_model="mock-rerank",
|
||||
),
|
||||
query_plan=query_plan,
|
||||
clusters=clusters,
|
||||
ranked_candidates=candidates,
|
||||
items_by_source={"grounding": [c.source_items[0] for c in candidates]},
|
||||
errors_by_source={},
|
||||
)
|
||||
|
||||
|
||||
class RenderComparisonMultiTests(unittest.TestCase):
|
||||
def test_three_entity_table(self):
|
||||
reports = [
|
||||
("OpenAI", _build_report("OpenAI", ["GPT-5 drop", "API pricing cut"])),
|
||||
("Anthropic", _build_report("Anthropic", ["Claude 4.7 ship", "MCP rollout"])),
|
||||
("xAI", _build_report("xAI", ["Grok 4 release", "Memphis cluster"])),
|
||||
]
|
||||
rendered = render.render_comparison_multi(reports)
|
||||
# All three entities appear in the header
|
||||
self.assertIn("OpenAI vs Anthropic vs xAI", rendered)
|
||||
# Each entity has its own evidence section
|
||||
self.assertIn("## OpenAI", rendered)
|
||||
self.assertIn("## Anthropic", rendered)
|
||||
self.assertIn("## xAI", rendered)
|
||||
# Scaffold table header has a column per entity
|
||||
self.assertIn("| Dimension | OpenAI | Anthropic | xAI |", rendered)
|
||||
# Envelope scaffolding present
|
||||
self.assertIn("EVIDENCE FOR SYNTHESIS", rendered)
|
||||
self.assertIn("END OF last30days CANONICAL OUTPUT", rendered)
|
||||
|
||||
def test_two_entity_table_has_two_columns(self):
|
||||
reports = [
|
||||
("Kanye West", _build_report("Kanye West", ["Donda 2 release"])),
|
||||
("Drake", _build_report("Drake", ["For All The Dogs"])),
|
||||
]
|
||||
rendered = render.render_comparison_multi(reports)
|
||||
self.assertIn("| Dimension | Kanye West | Drake |", rendered)
|
||||
self.assertIn("## Kanye West", rendered)
|
||||
self.assertIn("## Drake", rendered)
|
||||
|
||||
def test_empty_clusters_renders_placeholder(self):
|
||||
reports = [
|
||||
("OpenAI", _build_report("OpenAI", ["GPT-5 drop"])),
|
||||
("ObscureCompetitor", _build_report("ObscureCompetitor", [])),
|
||||
]
|
||||
rendered = render.render_comparison_multi(reports)
|
||||
self.assertIn("## ObscureCompetitor", rendered)
|
||||
self.assertIn("no significant discussion this month", rendered)
|
||||
# Main still has its cluster
|
||||
self.assertIn("GPT-5 drop", rendered)
|
||||
|
||||
def test_warnings_aggregated_and_labeled(self):
|
||||
report_a = _build_report("OpenAI", ["GPT-5 drop"])
|
||||
report_b = _build_report("Anthropic", ["Claude 4.7"])
|
||||
report_a.warnings.append("Brave quota exhausted")
|
||||
report_b.warnings.append("Exa returned 0 results")
|
||||
rendered = render.render_comparison_multi(
|
||||
[("OpenAI", report_a), ("Anthropic", report_b)]
|
||||
)
|
||||
self.assertIn("[OpenAI] Brave quota exhausted", rendered)
|
||||
self.assertIn("[Anthropic] Exa returned 0 results", rendered)
|
||||
|
||||
def test_raises_on_empty_input(self):
|
||||
with self.assertRaises(ValueError):
|
||||
render.render_comparison_multi([])
|
||||
|
||||
def test_context_emit(self):
|
||||
reports = [
|
||||
("OpenAI", _build_report("OpenAI", ["GPT-5 drop"])),
|
||||
("Anthropic", _build_report("Anthropic", ["Claude 4.7"])),
|
||||
]
|
||||
out = render.render_comparison_multi_context(reports)
|
||||
self.assertIn("Comparison: OpenAI vs Anthropic", out)
|
||||
self.assertIn("## OpenAI", out)
|
||||
self.assertIn("## Anthropic", out)
|
||||
self.assertIn("GPT-5 drop", out)
|
||||
|
||||
|
||||
class EmitComparisonOutputTests(unittest.TestCase):
|
||||
def test_json_emit_nests_per_entity(self):
|
||||
reports = [
|
||||
("OpenAI", _build_report("OpenAI", ["GPT-5 drop"])),
|
||||
("Anthropic", _build_report("Anthropic", ["Claude 4.7"])),
|
||||
]
|
||||
out = cli.emit_comparison_output(reports, emit="json")
|
||||
payload = json.loads(out)
|
||||
self.assertTrue(payload["comparison"])
|
||||
self.assertEqual(payload["entities"], ["OpenAI", "Anthropic"])
|
||||
self.assertEqual(len(payload["reports"]), 2)
|
||||
self.assertEqual(payload["reports"][0]["entity"], "OpenAI")
|
||||
self.assertIn("topic", payload["reports"][0]["report"])
|
||||
|
||||
def test_compact_and_md_both_route_to_multi(self):
|
||||
reports = [
|
||||
("A", _build_report("A", ["Thing A"])),
|
||||
("B", _build_report("B", ["Thing B"])),
|
||||
]
|
||||
compact = cli.emit_comparison_output(reports, emit="compact")
|
||||
md = cli.emit_comparison_output(reports, emit="md")
|
||||
self.assertIn("| Dimension | A | B |", compact)
|
||||
self.assertEqual(compact, md)
|
||||
|
||||
def test_context_emit_goes_to_context_renderer(self):
|
||||
reports = [
|
||||
("A", _build_report("A", ["Thing A"])),
|
||||
("B", _build_report("B", ["Thing B"])),
|
||||
]
|
||||
out = cli.emit_comparison_output(reports, emit="context")
|
||||
self.assertIn("Comparison: A vs B", out)
|
||||
|
||||
def test_unsupported_emit_raises(self):
|
||||
reports = [("A", _build_report("A", ["Thing A"]))]
|
||||
with self.assertRaises(SystemExit):
|
||||
cli.emit_comparison_output(reports, emit="xml")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user