fix: Step 0.55 category-peer subreddit expansion (#305)
* feat(resolve): category-peer subreddit map for Step 0.55 Introduces scripts/lib/categories.py with a curated category->peer-subs map and wires scripts/lib/resolve.py auto_resolve() to merge peers into the WebSearch-extracted subreddit list. Named 2026-04-22 failure mode: a "Prompting GPT Image 2" run resolved only r/OpenAI + r/ChatGPT and missed r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt where prompting techniques actually live. Map is static, curated, ~11 categories (ai_image_generation, ai_video_generation, ai_music_generation, ai_coding_agent, ai_agent_framework, ai_chat_model, saas_screen_recording, saas_productivity, prediction_markets, crypto_defi, dev_tool_cli). First-match-wins ordering from most-specific to least-specific. Compound-term patterns only (no bare common nouns like "image", "ai"). auto_resolve now: - calls detect_category(topic) after _extract_subreddits - merges peer_subs case-insensitively, caps at MAX_SUBS (10) - preserves every WebSearch-returned sub (freshest signal) - emits [Resolve] Matched category=<id>, adding peers: <list> on stderr only when peers were actually added - returns new "category" key in the result dict for observability - wraps classifier in try/except so failures degrade to unwidened list Includes drive-by: test_full_resolve / test_partial_failure searches_run expectations bumped from 3->4 / 2->3 to match the current queries dict (subreddit + news + x_handle + github). * feat(skill): Step 0.55 category-peer expansion and self-check Adds Section 2a (category-peer expansion, MANDATORY for product topics) and the Step 0.55 self-check checkpoint that fires immediately before the Resolved block displays. Structural mirror of the engine-side categories.py map: same categories, same peer subs, same priority order. The model-side path now: - Applies category-peer expansion to the WebSearch-resolved subs on every product-in-a-known-category run. - Emits the (+ <category_id> peers) annotation on the Reddit line of the Resolved block as the observable contract. Absence on a product-in-a-known-category topic is a Step 0.55 regression. - Runs a self-check before emitting Resolved: "does the resolved list include at least 2 peer subs for the matched category? if not, widen NOW and do not run the engine yet." Mirror of the Python map lives inside Step 0.55 as a table for the model to pattern-match against; extrapolation to unlisted categories is explicitly allowed. Worked example (the exact failing query) appears below the table so reviewers can see before/after at a glance. Both changes land inside the existing Step 0.55 block. No new top-level section, no new LAW. LAWs 1-6 wording unchanged. * test: end-to-end regression for GPT Image 2 failure mode Stubs grounding.web_search to return the OpenAI-only subs that caused the 2026-04-22 failure, then asserts that auto_resolve widens to include the image-gen peers and emits the [Resolve] Matched category=ai_image_generation stderr line. Covers the cap boundary and the uncategorized-topic no-op path. Fixture tests/fixtures/prompting-gpt-image-2-resolved-block.md is documentation-grade (not parsed by tests) and shows the pre-fix vs post-fix Resolved block shape so reviewers can evaluate future categories.py edits against the original bug. --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
This commit is contained in:
+67
-5
@@ -11,14 +11,64 @@ import re
|
||||
import sys
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
from . import dates, grounding
|
||||
from . import categories, dates, grounding
|
||||
|
||||
MAX_SUBS = 10
|
||||
|
||||
|
||||
def _log(msg: str) -> None:
|
||||
print(f"[Resolve] {msg}", file=sys.stderr)
|
||||
|
||||
|
||||
def _merge_category_peers(topic: str, subreddits: list[str]) -> tuple[list[str], Optional[str]]:
|
||||
"""Extend the WebSearch-extracted subreddit list with category peers.
|
||||
|
||||
Classifies the topic, fetches the category's peer subs, dedupes
|
||||
case-insensitively against the existing list, and appends missing
|
||||
peers in priority order. Caps the final list at MAX_SUBS, preserving
|
||||
every WebSearch-returned sub (they are the freshest signal) and
|
||||
trimming from the peer-additions end.
|
||||
|
||||
Returns a tuple of (merged_subs, matched_category_id_or_None).
|
||||
Emits a [Resolve] Matched category log line only when peers were
|
||||
actually added (not when every peer was already in the WebSearch set).
|
||||
|
||||
Classification failures degrade to "no match" — the unwidened list
|
||||
is returned and a warning is logged.
|
||||
"""
|
||||
try:
|
||||
category = categories.detect_category(topic)
|
||||
except Exception as exc:
|
||||
_log(f"Category classification failed: {exc}")
|
||||
return list(subreddits)[:MAX_SUBS], None
|
||||
|
||||
if category is None:
|
||||
return list(subreddits)[:MAX_SUBS], None
|
||||
|
||||
peers = categories.peer_subs_for(category)
|
||||
if not peers:
|
||||
return list(subreddits)[:MAX_SUBS], category
|
||||
|
||||
existing_lower = {s.lower() for s in subreddits}
|
||||
merged = list(subreddits)
|
||||
added: list[str] = []
|
||||
for peer in peers:
|
||||
if len(merged) >= MAX_SUBS:
|
||||
break
|
||||
if peer.lower() in existing_lower:
|
||||
continue
|
||||
merged.append(peer)
|
||||
existing_lower.add(peer.lower())
|
||||
added.append(peer)
|
||||
|
||||
if added:
|
||||
_log(f"Matched category={category}, adding peers: {', '.join(added)}")
|
||||
|
||||
return merged, category
|
||||
|
||||
|
||||
def _has_backend(config: dict) -> bool:
|
||||
"""Check if any web search backend is available."""
|
||||
return bool(
|
||||
@@ -134,10 +184,19 @@ def auto_resolve(topic: str, config: dict) -> dict:
|
||||
config: Dict with API keys (BRAVE_API_KEY, EXA_API_KEY, SERPER_API_KEY).
|
||||
|
||||
Returns:
|
||||
Dict with keys: subreddits, x_handle, context, searches_run.
|
||||
Returns empty result if no web search backend is available.
|
||||
Dict with keys: subreddits, x_handle, github_user, github_repos,
|
||||
context, category, searches_run. Returns empty result if no web
|
||||
search backend is available.
|
||||
"""
|
||||
empty = {"subreddits": [], "x_handle": "", "context": "", "searches_run": 0}
|
||||
empty = {
|
||||
"subreddits": [],
|
||||
"x_handle": "",
|
||||
"github_user": "",
|
||||
"github_repos": [],
|
||||
"context": "",
|
||||
"category": None,
|
||||
"searches_run": 0,
|
||||
}
|
||||
|
||||
if not _has_backend(config):
|
||||
_log("No web search backend available, skipping resolve")
|
||||
@@ -184,7 +243,9 @@ def auto_resolve(topic: str, config: dict) -> dict:
|
||||
github_repos = _extract_github_repos(results.get("github", []))
|
||||
context = _build_context_summary(results.get("news", []))
|
||||
|
||||
_log(f"Resolved {len(subreddits)} subreddits, x_handle={x_handle!r}, github_user={github_user!r}, github_repos={github_repos!r}, context_len={len(context)}")
|
||||
subreddits, category = _merge_category_peers(topic, subreddits)
|
||||
|
||||
_log(f"Resolved {len(subreddits)} subreddits, x_handle={x_handle!r}, github_user={github_user!r}, github_repos={github_repos!r}, context_len={len(context)}, category={category!r}")
|
||||
|
||||
return {
|
||||
"subreddits": subreddits,
|
||||
@@ -192,5 +253,6 @@ def auto_resolve(topic: str, config: dict) -> dict:
|
||||
"github_user": github_user,
|
||||
"github_repos": github_repos,
|
||||
"context": context,
|
||||
"category": category,
|
||||
"searches_run": searches_run,
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user