* feat: vs mode runs N full passes; --competitors wraps vs with auto-discovery
Unifies vs-mode and --competitors onto one fanout architecture. A topic
containing "vs" / "versus" now runs N full pipeline.run() calls in parallel
(reverting the one-pass latency optimization that removed per-entity
depth); --competitors becomes a SKILL.md-level shortcut where the hosting
reasoning model (Claude Code, Codex, Hermes, Gemini) discovers N peers via
its own WebSearch, runs Step 0.55 per entity, and invokes the engine with
a vs-topic + --competitors-plan JSON.
Changed:
- vs-mode: N full passes in parallel via fanout (was 1 merged pass).
- --competitors: SKILL.md shortcut for vs-mode-with-discovery. Engine flag
kept for headless/cron use. LAW 7-style stderr reframed to lead with the
hosting-model path (use WebSearch + --competitors-plan) instead of
BRAVE_API_KEY. Footer BRAVE/SERPER nudge suppressed when --plan or
--competitors-plan present (hosting model already has WebSearch).
Added:
- --competitors-plan JSON flag: per-entity {x_handle, x_related, subreddits,
github_user, github_repos, context}. Accepts inline JSON or file path.
subrun_kwargs_for helper is the single source of truth for per-entity
kwargs — no closure-default fallthrough from main scope.
- Per-entity save files: each entity's sub-run produces its own
{slug}-raw.md with a single-row Resolved Entities block.
- --polymarket-keywords filter for ambiguous single-token topics.
Fixed:
- test_competitor_subrun_isolation regression suite locks in 3.0.12's
no-leak invariant (main flags do not inherit into peer sub-runs).
- Updates test_regression.py for the new comparison-mode payload shape.
Bumps plugin.json to 3.0.13. 1,219 tests passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: comparison title attribution — (Last 30 Days) → (/Last30Days)
User feedback on 3.0.13 dogfood runs (Kanye vs Drake, Mercer Island,
Figma): the comparison-mode synthesis title should attribute to the
slash command rather than restate the date range.
Three SKILL.md occurrences updated. Pure documentation change. Bumps to
3.0.14.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four fixes based on 2026-04-22 test-window feedback on v3.0.11 --competitors:
- Each competitor sub-run now runs Step 0.55 (X handle / subreddits /
GitHub) via resolve.auto_resolve inside the fanout closure. Deep-copied
config per entity prevents _auto_resolve_context leak across sub-runs.
Resolved data stored on report.artifacts["resolved"] for the renderer.
- New internal_subrun keyword on planner.plan_query and pipeline.run
suppresses the LAW 7 "No --plan passed" stderr for engine-internal
fan-out only. Default path unchanged.
- Default --competitors count is now 2 (3-way total). --competitors=N
still customizes; range 1..6.
- SKILL.md STEP 0 canonical-path self-check forces readers who loaded
from marketplaces/ (auto-restored to origin/main, stale) to re-read
from plugins/cache/last30days-skill/last30days/{VERSION}/SKILL.md.
Two of three 2026-04-22 test windows hit this stale-path trap.
- New ## Resolved Entities block in render_comparison_multi shows
per-entity handles/subs/github for debug visibility.
Bumps plugin.json to 3.0.12. 12 new tests; 1,175 total passing.
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds xurl (https://github.com/openclaw/xurl) as a third X search
backend, sitting after xAI API and Bird/GraphQL in the priority chain.
xurl uses the official X API v2 with OAuth2+PKCE authentication,
requiring only a free X Developer App. It auto-refreshes tokens and
works reliably as a stable fallback when xAI API key or browser
cookies are not available.
Limitations:
- X API search/recent returns last 7 days only (vs Bird's full archive)
- No AI-powered relevance scoring (uses token_overlap_relevance instead)
- Free tier: 180 requests per 15-minute window
New files:
- scripts/lib/xurl_x.py: xurl CLI wrapper with search + parse
- tests/test_xurl_x.py: 30 unit tests (all passing)
Modified files:
- scripts/lib/env.py: detect xurl in get_x_source_with_method(),
get_missing_keys(), and get_x_source_status()
- scripts/last30days.py: add xurl_x import and xurl branch in
_search_x() priority chain
- SKILL.md: document xurl setup option
The stderr [Planner] warning from PR #285 doesn't reach the user because
Claude and other reasoning agents hide stderr from their synthesis. The
2026-04-19 Hermes Agent Use Cases Run 1 produced source=deterministic
and the user never saw it.
Adds a user-visible stdout block that the model's LAW 5 pass-through
contract forces into the response. Fires only when plan_source is
deterministic AND no pre-research flags were passed AND the topic is
pre-research-eligible (named entity). Cron jobs on abstract topics
don't trigger it.
Position: BEFORE the EVIDENCE FOR SYNTHESIS envelope so the model sees
it as the first non-badge content. Wrapped in a new USER-VISIBLE BANNER
envelope matching the EVIDENCE/PASS-THROUGH envelope pattern from Unit 1
of PR #285.
Runtime-agnostic language: explicitly enumerates Claude Code, Codex,
Hermes, Gemini so the hosting reasoning model recognizes itself
regardless of runtime.
pipeline.py now persists plan_source to report.artifacts so the
renderer can consume it. Adds 7 tests covering fire conditions,
suppression conditions (external/llm plan source, flags present,
abstract topic), and correct position relative to the evidence envelope.
The prior pipeline.py only logged the planner outcome when an external
--plan was passed ("[Planner] Using external plan (N subqueries)").
The internal LLM planner and the deterministic fallback ran silently,
so retrieval-breadth failures were invisible without --debug.
After plan finalization, emit a unified trace:
[Planner] Plan: intent=X, freshness=Y, cluster_mode=Z, subqueries=N, source=external|llm|deterministic
[Planner] sq1 label=... search="..." sources=[...]
[Planner] sq2 ...
Stderr only; does not touch the user-facing stdout synthesis. The
source= annotation distinguishes --plan (external), provider-backed
(llm), and deterministic paths — so when the 2026-04-19 Hermes Agent
Use Cases failure mode recurs, the trace tells the user which path ran
and what subqueries it produced.
Tests: added test_planner_trace_always_fires_on_mock_run which captures
stderr on a mock pipeline run and asserts the summary + per-subquery
lines appear.
Consolidates seven beta-validated plans into the public release. Validated
on nine+ topics across GENERAL, COMPARISON, RECOMMENDATIONS, and
demographic-shopping classes before ship.
Plans bundled in this release:
- 003 Engine-emitted Pre-Research Status warning + Polymarket summarization
+ VOICE CONTRACT LAW 1-5 + Step 0.55 MANDATORY
- 004 WebSearch deferred-tool loading (ToolSearch STEP 0) + LAW 5 universal
+ top-of-file imperative
- 005 Supplement floor (2-3 minimum) separate from Step 0.55 pre-research
- 006 Step 2.5 MANDATORY raw-file append with canonical format example +
count-equality self-check
- 007 Restored April 9 canonical comparison template with Quick Verdict,
per-entity Strengths/Weaknesses, 9-axis Head-to-Head, Bottom Line,
emerging stack + LAW 2/4 COMPARISON exceptions
- 008 Person-topic GitHub handle resolution MANDATORY + LAW 1 reinforcement
at Step 2 tail and Step 2.5 entry + RECOMMENDATIONS signal-weighted
ranking rewrite + Polymarket post-merge topic filter (engine change,
filter_items_against_topic helper + vs/versus in _NOISE_WORDS)
- 009 Unified pre-flight CHECKLIST + VOICE CONTRACT formatting-authority
preface + Step 0.45 Query Quality Pre-Flight (4 keyword-trap classes) +
post-synthesis Sources-block self-check
Beta validation topics (2026-04-18): Kanye West, Matt Van Horn, CLI vs MCP,
OpenClaw vs Paperclip vs Hermes, Paperclip vs Hermes vs Open Claw, Garry
Tan, Israel vs Lebanon, Best programming language for AI agents, Peter
Steinberger post plan 009, Birthday gift for 42 year old man (Class 1
pre-flight fired correctly), Vincent Koc (passed).
No breaking changes. No new CLI flags. No new public API. Plugin name
(last30days) and marketplace name (last30days-skill) unchanged.
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(normalize): pass YouTube top_comments through with Reddit-compatible shape
_normalize_youtube silently dropped top_comments after enrich_with_comments
populated them, so the downstream signals/render/entity layers never saw
YouTube comments. Map likes->score and text->excerpt so the existing
Reddit-compatible readers Just Work.
Shared _remap_comments helper will be reused for TikTok in a later commit.
* feat(tiktok): fetch top comments via ScrapeCreators when opted in
Mirrors the youtube_comments pattern: new env.is_tiktok_comments_available
gate (requires SCRAPECREATORS_API_KEY + tiktok_comments in INCLUDE_SOURCES),
tiktok.enrich_with_comments ranks posts and fetches via
GET /v1/tiktok/video/comments. Vote field is digg_count; text and user.nickname
come across verbatim. Pipeline calls the enricher right after TikTok search
when the gate is open.
Comment-fetch errors never crash the pipeline — the enricher returns an
empty list on 4xx/5xx.
* feat(normalize): pass TikTok top_comments through with digg_count->score mapping
Instagram uses the same shortform normalizer and has no comment fetcher
today, so the key is harmlessly absent there — no Instagram regression.
* feat(signals): add YouTube + TikTok top-comment score to engagement formula
Mirrors Reddit's 10% top-comment slot. Without top_comments present, the
formula reduces to views-dominant weighting; with a high-signal comment,
the item gets a meaningful bump (log1p(10k) ~ 9.2, weighted 0.10 = ~0.92
on the engagement score).
Updated the existing dominant-weight and missing-fields tests to the new
weights (0.45/0.32/0.13 for YT, 0.45/0.27/0.18 for TT). Views still dominate.
* feat(render): source-aware thresholds and vote labels for top comments
10 upvotes on Reddit signals community interest; 10 likes on a viral
TikTok is noise. Introduce per-source minimums (reddit 10, youtube 50,
tiktok 500) and native vote labels ('upvotes' for Reddit, 'likes' for
YT/TT). First-pass numbers — tune after live observation.
* docs: generalize top-comment quoting to YouTube + TikTok, add tiktok_comments opt-in
Synthesis instructions previously called out Reddit top comments only.
Now cover Reddit/YouTube/TikTok uniformly with source-appropriate vote
labels (upvotes vs likes), and explicitly frame YT transcript highlights
and comments as complementary signals. README and setup-wizard copy
document the new tiktok_comments INCLUDE_SOURCES token.
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
The dedup hot path recomputed normalize_text() 4 times per comparison
and recomputed item_text() on every inner-loop iteration. Pre-computing
n-gram sets and token sets into a _PreparedText cache cuts dedup time
by 6x (2.16s to 0.39s on 300 unique items).
Bird handle searches spawned one Node process per handle sequentially.
Now uses ThreadPoolExecutor so N handles run concurrently. Same pattern
applied to YouTube comment enrichment (was serial, Reddit was already
parallel) and the retry-thin-sources phase in the pipeline.
Clustering now pre-computes candidate text and uses prepared_similarity
for the O(n^2) grouping and MMR representative selection loops.
Minor: _is_wsl() cached with lru_cache, Bundle.add_items() uses
extend() instead of list concatenation.
End-to-end: 5.2s -> 3.7s (29% faster) on a typical 4-source query.
Add Xquik (xquik.com) as a new X/Twitter search source that uses a REST
API with full engagement metrics (likes, retweets, replies, quotes,
views, bookmarks). Uses stdlib urllib only -- no new dependencies.
- scripts/lib/xquik.py: source module with search, parse, query expansion
- tests/test_xquik.py: 32 unit tests covering all functions
- env.py: XQUIK_API_KEY config and availability check
- pipeline.py: source registration and retrieve dispatch
- normalize.py: reuses _normalize_x (same item format as Bird)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Matt Van Horn