sawz/last30days-skill
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: Step 0.55 category-peer subreddit expansion (#305)

Browse Source 
* feat(resolve): category-peer subreddit map for Step 0.55

Introduces scripts/lib/categories.py with a curated category->peer-subs
map and wires scripts/lib/resolve.py auto_resolve() to merge peers into
the WebSearch-extracted subreddit list. Named 2026-04-22 failure mode:
a "Prompting GPT Image 2" run resolved only r/OpenAI + r/ChatGPT and
missed r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt where
prompting techniques actually live.

Map is static, curated, ~11 categories (ai_image_generation,
ai_video_generation, ai_music_generation, ai_coding_agent,
ai_agent_framework, ai_chat_model, saas_screen_recording,
saas_productivity, prediction_markets, crypto_defi, dev_tool_cli).
First-match-wins ordering from most-specific to least-specific.
Compound-term patterns only (no bare common nouns like "image", "ai").

auto_resolve now:
- calls detect_category(topic) after _extract_subreddits
- merges peer_subs case-insensitively, caps at MAX_SUBS (10)
- preserves every WebSearch-returned sub (freshest signal)
- emits [Resolve] Matched category=<id>, adding peers: <list> on stderr
  only when peers were actually added
- returns new "category" key in the result dict for observability
- wraps classifier in try/except so failures degrade to unwidened list

Includes drive-by: test_full_resolve / test_partial_failure
searches_run expectations bumped from 3->4 / 2->3 to match the current
queries dict (subreddit + news + x_handle + github).

* feat(skill): Step 0.55 category-peer expansion and self-check

Adds Section 2a (category-peer expansion, MANDATORY for product topics)
and the Step 0.55 self-check checkpoint that fires immediately before
the Resolved block displays. Structural mirror of the engine-side
categories.py map: same categories, same peer subs, same priority
order.

The model-side path now:
- Applies category-peer expansion to the WebSearch-resolved subs on
  every product-in-a-known-category run.
- Emits the (+ <category_id> peers) annotation on the Reddit line of
  the Resolved block as the observable contract. Absence on a
  product-in-a-known-category topic is a Step 0.55 regression.
- Runs a self-check before emitting Resolved: "does the resolved list
  include at least 2 peer subs for the matched category? if not,
  widen NOW and do not run the engine yet."

Mirror of the Python map lives inside Step 0.55 as a table for the
model to pattern-match against; extrapolation to unlisted categories
is explicitly allowed. Worked example (the exact failing query)
appears below the table so reviewers can see before/after at a glance.

Both changes land inside the existing Step 0.55 block. No new
top-level section, no new LAW. LAWs 1-6 wording unchanged.

* test: end-to-end regression for GPT Image 2 failure mode

Stubs grounding.web_search to return the OpenAI-only subs that caused
the 2026-04-22 failure, then asserts that auto_resolve widens to
include the image-gen peers and emits the [Resolve] Matched
category=ai_image_generation stderr line. Covers the cap boundary
and the uncategorized-topic no-op path.

Fixture tests/fixtures/prompting-gpt-image-2-resolved-block.md is
documentation-grade (not parsed by tests) and shows the pre-fix vs
post-fix Resolved block shape so reviewers can evaluate future
categories.py edits against the original bug.

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
This commit is contained in:
Matt Van Horn
2026-04-22 14:31:39 -07:00
committed by GitHub
parent 952a876536
commit 4e91f4e754
7 changed files with 929 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/lib/categories.py
+283
View File
@@ -0,0 +1,283 @@
"""Category-peer subreddit map for Step 0.55 community resolution.
When a topic is a product in a known category (AI image generation, AI coding
agents, SaaS screen recording, etc.), brand-specific subreddits returned by
WebSearch are insufficient: cross-product technique discussion lives in
category-peer subs. This module classifies a topic into a category by matching
compound-term patterns against the lowercased topic string, then returns the
priority-ordered peer subreddit list for that category.
The map is intentionally small, curated, and code-reviewed. Adding a new
category is a code change; there is no user-editable override surface.
False-positive guard: every pattern is either a multi-word compound (e.g.
"image generation", "text to image") or a domain-specific single word
(e.g. "midjourney", "stablediffusion"). Bare common nouns like "image",
"ai", or "model" are never used as patterns.
First-match-wins: categories are evaluated in declared order. Entries are
sorted from most-specific to least-specific so narrower categories claim a
topic before broader ones. For example, `ai_image_generation` appears
before `ai_chat_model` so "gpt image 2" matches the image-gen category.
"""
from __future__ import annotations
from typing import List, Optional, TypedDict
class _CategoryEntry(TypedDict):
patterns: List[str]
peer_subs: List[str]
CATEGORY_PEERS: dict[str, _CategoryEntry] = {
"ai_image_generation": {
"patterns": [
"image generation",
"image gen",
"text to image",
"text-to-image",
"gpt image",
"gpt-image",
"nano banana",
"midjourney",
"stable diffusion",
"stablediffusion",
"dall-e",
"dalle",
"flux.1",
"flux schnell",
"imagen",
"seedance",
"ideogram",
"recraft",
],
"peer_subs": [
"StableDiffusion",
"midjourney",
"dalle2",
"aiArt",
"PromptEngineering",
"MediaSynthesis",
],
},
"ai_video_generation": {
"patterns": [
"video generation",
"text to video",
"text-to-video",
"sora",
"veo 3",
"veo3",
"runway gen",
"kling",
"pika labs",
"luma dream machine",
"hailuo",
],
"peer_subs": [
"aivideo",
"StableDiffusion",
"runwayml",
"singularity",
"MediaSynthesis",
],
},
"ai_music_generation": {
"patterns": [
"music generation",
"ai music",
"suno",
"udio",
"riffusion",
"stable audio",
],
"peer_subs": [
"SunoAI",
"udiomusic",
"aimusic",
"artificial",
],
},
"ai_coding_agent": {
"patterns": [
"claude code",
"cursor ide",
"github copilot",
"windsurf",
"aider",
"cline",
"openclaw",
"hermes agent",
"continue.dev",
"codeium",
"sweep ai",
"devin ai",
"coding agent",
"coding assistant",
],
"peer_subs": [
"ChatGPTCoding",
"LocalLLaMA",
"singularity",
"PromptEngineering",
],
},
"ai_agent_framework": {
"patterns": [
"agent framework",
"agentic framework",
"langchain",
"langgraph",
"crewai",
"autogen",
"llamaindex",
"dspy",
"smolagents",
],
"peer_subs": [
"LangChain",
"LocalLLaMA",
"AI_Agents",
"MachineLearning",
],
},
"ai_chat_model": {
"patterns": [
"gpt-5",
"gpt-4",
"claude opus",
"claude sonnet",
"claude haiku",
"gemini pro",
"gemini flash",
"llama 3",
"llama 4",
"deepseek",
"qwen",
"mistral large",
"grok",
],
"peer_subs": [
"LocalLLaMA",
"ChatGPT",
"ClaudeAI",
"singularity",
"artificial",
],
},
"saas_screen_recording": {
"patterns": [
"screen recording",
"screen recorder",
"loom video",
"tella screen",
"vidyard",
"screen capture tool",
],
"peer_subs": [
"SaaS",
"screenrecording",
"productivity",
"Entrepreneur",
],
},
"saas_productivity": {
"patterns": [
"notion app",
"obsidian plugin",
"obsidian app",
"linear app",
"asana",
"clickup",
"productivity app",
],
"peer_subs": [
"productivity",
"SaaS",
"ObsidianMD",
"Notion",
],
},
"prediction_markets": {
"patterns": [
"polymarket",
"kalshi",
"prediction market",
"event contracts",
"manifold markets",
],
"peer_subs": [
"Polymarket",
"Kalshi",
"predictionmarkets",
],
},
"crypto_defi": {
"patterns": [
"defi protocol",
"yield farming",
"liquidity pool",
"stablecoin",
"ethereum layer",
"layer 2",
"l2 rollup",
],
"peer_subs": [
"defi",
"ethfinance",
"CryptoCurrency",
"ethereum",
],
},
"dev_tool_cli": {
"patterns": [
"cli tool",
"command line tool",
"terminal app",
"dev tool",
],
"peer_subs": [
"commandline",
"programming",
"webdev",
],
},
}
def detect_category(topic: Optional[str]) -> Optional[str]:
"""Classify a topic into a known category by compound-term match.
Returns the category id (e.g. "ai_image_generation") or None if no
category's patterns match. Matching is case-insensitive substring over
the lowercased topic. Declaration order wins (first-match-wins), so the
map is ordered from most-specific to least-specific.
A None or empty topic returns None. Classification never raises on
normal string inputs; callers do not need to wrap in try/except for
typical paths, though defensive callers may.
"""
if not topic:
return None
lowered = topic.lower()
for category_id, entry in CATEGORY_PEERS.items():
for pattern in entry["patterns"]:
if pattern in lowered:
return category_id
return None
def peer_subs_for(category_id: Optional[str]) -> List[str]:
"""Return the priority-ordered peer subreddit list for a category.
Returns an empty list for None or unknown category ids. The returned
list is a fresh copy; callers may safely mutate it.
"""
if not category_id:
return []
entry = CATEGORY_PEERS.get(category_id)
if not entry:
return []
return list(entry["peer_subs"])
scripts/lib/resolve.py
+67 -5
View File
@@ -11,14 +11,64 @@ import re
import sys
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import datetime, timezone
from typing import Optional
from . import dates, grounding
from . import categories, dates, grounding
MAX_SUBS = 10
def _log(msg: str) -> None:
print(f"[Resolve] {msg}", file=sys.stderr)
def _merge_category_peers(topic: str, subreddits: list[str]) -> tuple[list[str], Optional[str]]:
"""Extend the WebSearch-extracted subreddit list with category peers.
Classifies the topic, fetches the category's peer subs, dedupes
case-insensitively against the existing list, and appends missing
peers in priority order. Caps the final list at MAX_SUBS, preserving
every WebSearch-returned sub (they are the freshest signal) and
trimming from the peer-additions end.
Returns a tuple of (merged_subs, matched_category_id_or_None).
Emits a [Resolve] Matched category log line only when peers were
actually added (not when every peer was already in the WebSearch set).
Classification failures degrade to "no match" — the unwidened list
is returned and a warning is logged.
"""
try:
category = categories.detect_category(topic)
except Exception as exc:
_log(f"Category classification failed: {exc}")
return list(subreddits)[:MAX_SUBS], None
if category is None:
return list(subreddits)[:MAX_SUBS], None
peers = categories.peer_subs_for(category)
if not peers:
return list(subreddits)[:MAX_SUBS], category
existing_lower = {s.lower() for s in subreddits}
merged = list(subreddits)
added: list[str] = []
for peer in peers:
if len(merged) >= MAX_SUBS:
break
if peer.lower() in existing_lower:
continue
merged.append(peer)
existing_lower.add(peer.lower())
added.append(peer)
if added:
_log(f"Matched category={category}, adding peers: {', '.join(added)}")
return merged, category
def _has_backend(config: dict) -> bool:
"""Check if any web search backend is available."""
return bool(
@@ -134,10 +184,19 @@ def auto_resolve(topic: str, config: dict) -> dict:
config: Dict with API keys (BRAVE_API_KEY, EXA_API_KEY, SERPER_API_KEY).
Returns:
Dict with keys: subreddits, x_handle, context, searches_run.
Returns empty result if no web search backend is available.
Dict with keys: subreddits, x_handle, github_user, github_repos,
context, category, searches_run. Returns empty result if no web
search backend is available.
"""
empty = {"subreddits": [], "x_handle": "", "context": "", "searches_run": 0}
empty = {
"subreddits": [],
"x_handle": "",
"github_user": "",
"github_repos": [],
"context": "",
"category": None,
"searches_run": 0,
}
if not _has_backend(config):
_log("No web search backend available, skipping resolve")
@@ -184,7 +243,9 @@ def auto_resolve(topic: str, config: dict) -> dict:
github_repos = _extract_github_repos(results.get("github", []))
context = _build_context_summary(results.get("news", []))
_log(f"Resolved {len(subreddits)} subreddits, x_handle={x_handle!r}, github_user={github_user!r}, github_repos={github_repos!r}, context_len={len(context)}")
subreddits, category = _merge_category_peers(topic, subreddits)
_log(f"Resolved {len(subreddits)} subreddits, x_handle={x_handle!r}, github_user={github_user!r}, github_repos={github_repos!r}, context_len={len(context)}, category={category!r}")
return {
"subreddits": subreddits,
@@ -192,5 +253,6 @@ def auto_resolve(topic: str, config: dict) -> dict:
"github_user": github_user,
"github_repos": github_repos,
"context": context,
"category": category,
"searches_run": searches_run,
}