fix: Step 0.55 category-peer subreddit expansion (#305)
* feat(resolve): category-peer subreddit map for Step 0.55 Introduces scripts/lib/categories.py with a curated category->peer-subs map and wires scripts/lib/resolve.py auto_resolve() to merge peers into the WebSearch-extracted subreddit list. Named 2026-04-22 failure mode: a "Prompting GPT Image 2" run resolved only r/OpenAI + r/ChatGPT and missed r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt where prompting techniques actually live. Map is static, curated, ~11 categories (ai_image_generation, ai_video_generation, ai_music_generation, ai_coding_agent, ai_agent_framework, ai_chat_model, saas_screen_recording, saas_productivity, prediction_markets, crypto_defi, dev_tool_cli). First-match-wins ordering from most-specific to least-specific. Compound-term patterns only (no bare common nouns like "image", "ai"). auto_resolve now: - calls detect_category(topic) after _extract_subreddits - merges peer_subs case-insensitively, caps at MAX_SUBS (10) - preserves every WebSearch-returned sub (freshest signal) - emits [Resolve] Matched category=<id>, adding peers: <list> on stderr only when peers were actually added - returns new "category" key in the result dict for observability - wraps classifier in try/except so failures degrade to unwidened list Includes drive-by: test_full_resolve / test_partial_failure searches_run expectations bumped from 3->4 / 2->3 to match the current queries dict (subreddit + news + x_handle + github). * feat(skill): Step 0.55 category-peer expansion and self-check Adds Section 2a (category-peer expansion, MANDATORY for product topics) and the Step 0.55 self-check checkpoint that fires immediately before the Resolved block displays. Structural mirror of the engine-side categories.py map: same categories, same peer subs, same priority order. The model-side path now: - Applies category-peer expansion to the WebSearch-resolved subs on every product-in-a-known-category run. - Emits the (+ <category_id> peers) annotation on the Reddit line of the Resolved block as the observable contract. Absence on a product-in-a-known-category topic is a Step 0.55 regression. - Runs a self-check before emitting Resolved: "does the resolved list include at least 2 peer subs for the matched category? if not, widen NOW and do not run the engine yet." Mirror of the Python map lives inside Step 0.55 as a table for the model to pattern-match against; extrapolation to unlisted categories is explicitly allowed. Worked example (the exact failing query) appears below the table so reviewers can see before/after at a glance. Both changes land inside the existing Step 0.55 block. No new top-level section, no new LAW. LAWs 1-6 wording unchanged. * test: end-to-end regression for GPT Image 2 failure mode Stubs grounding.web_search to return the OpenAI-only subs that caused the 2026-04-22 failure, then asserts that auto_resolve widens to include the image-gen peers and emits the [Resolve] Matched category=ai_image_generation stderr line. Covers the cap boundary and the uncategorized-topic no-op path. Fixture tests/fixtures/prompting-gpt-image-2-resolved-block.md is documentation-grade (not parsed by tests) and shows the pre-fix vs post-fix Resolved block shape so reviewers can evaluate future categories.py edits against the original bug. --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
This commit is contained in:
Matt Van Horn
@@ -596,6 +596,43 @@ The first search finds subreddits. The second gives you current events context (
|
||||
|
||||
Extract 3-5 subreddit names from the results. Store as `RESOLVED_SUBREDDITS` (comma-separated, no r/ prefix).
|
||||
|
||||
**2a. Category-peer expansion (MANDATORY for product topics).** If the topic is a product in a recognizable category (AI image generation, AI video generation, AI coding agents, AI music, AI chat models, SaaS screen recording, prediction markets, etc.), the brand-specific subreddits that WebSearch returned are INSUFFICIENT. Add 2-3 peer subreddits from the category. Peer subs are where cross-product technique discussion actually lives. Missing them is the 2026-04-22 `GPT Image 2` failure mode: the model resolved `r/OpenAI, r/ChatGPT, r/singularity, r/ChatGPTpromptengineering` (all OpenAI-brand) and missed `r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt` where prompting techniques are actually shared. The user had to manually prompt "check image generation reddits too" to get a usable run.
|
||||
|
||||
Canonical category peers (single source of truth; `scripts/lib/categories.py` mirrors this for the `--auto-resolve` engine path):
|
||||
|
||||
| Category | Trigger keywords | Peer subs (priority order) |
|
||||
|----------|------------------|---------------------------|
|
||||
| `ai_image_generation` | image generation, text to image, GPT Image, Nano Banana, Midjourney, Stable Diffusion, DALL-E, Flux.1, Imagen, Seedance, Ideogram, Recraft | `StableDiffusion, midjourney, dalle2, aiArt, PromptEngineering, MediaSynthesis` |
|
||||
| `ai_video_generation` | video generation, text to video, Sora, Veo 3, Runway Gen, Kling, Pika Labs, Luma Dream Machine, Hailuo | `aivideo, StableDiffusion, runwayml, singularity, MediaSynthesis` |
|
||||
| `ai_music_generation` | music generation, ai music, Suno, Udio, Riffusion, Stable Audio | `SunoAI, udiomusic, aimusic, artificial` |
|
||||
| `ai_coding_agent` | Claude Code, Cursor IDE, GitHub Copilot, Windsurf, Aider, Cline, OpenClaw, Hermes Agent, Continue.dev, Codeium, Devin | `ChatGPTCoding, LocalLLaMA, singularity, PromptEngineering` |
|
||||
| `ai_agent_framework` | agent framework, LangChain, LangGraph, CrewAI, AutoGen, LlamaIndex, DSPy, smolagents | `LangChain, LocalLLaMA, AI_Agents, MachineLearning` |
|
||||
| `ai_chat_model` | GPT-5/4, Claude Opus/Sonnet/Haiku, Gemini Pro/Flash, Llama 3/4, DeepSeek, Qwen, Mistral Large, Grok | `LocalLLaMA, ChatGPT, ClaudeAI, singularity, artificial` |
|
||||
| `saas_screen_recording` | screen recording, screen recorder, Loom video, Tella screen, Vidyard | `SaaS, screenrecording, productivity, Entrepreneur` |
|
||||
| `saas_productivity` | Notion app, Obsidian, Linear app, Asana, ClickUp, productivity app | `productivity, SaaS, ObsidianMD, Notion` |
|
||||
| `prediction_markets` | Polymarket, Kalshi, prediction market, event contracts, Manifold Markets | `Polymarket, Kalshi, predictionmarkets` |
|
||||
| `crypto_defi` | DeFi protocol, yield farming, liquidity pool, stablecoin, layer 2, L2 rollup | `defi, ethfinance, CryptoCurrency, ethereum` |
|
||||
|
||||
**Merging rule.** Start with WebSearch-returned subs. Append 2-3 category peers in the priority order shown. Dedupe case-insensitively (don't list `midjourney` twice if WebSearch already returned it). Cap total at 10: if adding all peers would exceed the cap, keep every WebSearch-returned sub (they are the freshest signal) and drop peers from the end of the priority list.
|
||||
|
||||
**Extrapolation.** If the topic is a product in a category NOT listed in the table (new AI tool, niche SaaS), use the same spirit: pick the 2-3 most active cross-product communities where technique discussion happens. A new image-gen tool still gets `r/StableDiffusion, r/midjourney, r/aiArt`. A new code editor still gets `r/ChatGPTCoding, r/LocalLLaMA`.
|
||||
|
||||
**Worked example — the failing query.** Topic: `Prompting GPT Image 2`.
|
||||
|
||||
Before (the 2026-04-22 failure mode):
|
||||
```
|
||||
Resolved:
|
||||
- Reddit: r/OpenAI, r/ChatGPT, r/singularity, r/ChatGPTpromptengineering, r/artificial
|
||||
```
|
||||
|
||||
After (with category-peer expansion):
|
||||
```
|
||||
Resolved:
|
||||
- Reddit: r/OpenAI, r/ChatGPT, r/singularity, r/ChatGPTpromptengineering, r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt (+ ai_image_generation peers)
|
||||
```
|
||||
|
||||
The parenthetical `(+ ai_image_generation peers)` is the observable contract of the new Resolved block format. See Step 0.55 self-check below.
|
||||
|
||||
**3. TikTok hashtags + creators** - **INFER these from your topic knowledge. Do NOT WebSearch for "{PERSON} TikTok account" - most people/CEOs don't have TikTok, and the search is wasted.**
|
||||
|
||||
- **Hashtags:** Infer 2-3 from the topic name + category. Examples: "Kanye West" → `kanyewest,ye,bully`. "Claude Code" → `claudecode,aiagent,aicoding`. "Sam Altman" → `samaltman,openai,chatgpt`.
|
||||
@@ -658,17 +695,19 @@ Passing the resolved block visibly (per-entity, all 4 types each) is the observa
|
||||
|
||||
**If you can't infer targeting for a platform, skip that flag -- the Python engine will fall back to keyword search.**
|
||||
|
||||
**Step 0.55 self-check: category-peer coverage.** Before emitting the Resolved block, re-read your resolved subreddit list. Does the topic match any category in the Section 2a table (or fit the spirit of one — AI image gen, AI coding, AI music, etc.)? If YES: does your list include AT LEAST 2 peer subs from that category? If NO, widen the list NOW — do not run the engine yet. The observable contract is the `(+ {category_id} peers)` annotation on the Reddit line in the Resolved block. Its absence on a product-in-a-known-category topic is a Step 0.55 regression — the named 2026-04-22 failure mode. Person topics, music artists, news stories, and topics outside any category are exempt; omit the annotation.
|
||||
|
||||
**After resolving all handles and communities, display what you found before moving on.** This shows the user that intelligent pre-research happened:
|
||||
|
||||
```
|
||||
Resolved:
|
||||
- X: @{HANDLE} (+ @{COMPANY}, @{COMMENTATOR})
|
||||
- Reddit: r/{sub1}, r/{sub2}, r/{sub3}
|
||||
- Reddit: r/{sub1}, r/{sub2}, r/{sub3}, r/{peer1}, r/{peer2} (+ {category_id} peers)
|
||||
- TikTok: #{hashtag1}, #{hashtag2}
|
||||
- YouTube: {query1}, {query2}
|
||||
```
|
||||
|
||||
Only show lines for platforms where something was resolved. Skip empty lines. This display replaces the old "Parsed intent" block with something more useful.
|
||||
Only show lines for platforms where something was resolved. Skip empty lines. On the Reddit line, the trailing `(+ {category_id} peers)` annotation appears when Step 0.55 Section 2a added category-peer subs. Omit the annotation when the topic had no matching category. This display replaces the old "Parsed intent" block with something more useful.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,283 @@
|
||||
"""Category-peer subreddit map for Step 0.55 community resolution.
|
||||
|
||||
When a topic is a product in a known category (AI image generation, AI coding
|
||||
agents, SaaS screen recording, etc.), brand-specific subreddits returned by
|
||||
WebSearch are insufficient: cross-product technique discussion lives in
|
||||
category-peer subs. This module classifies a topic into a category by matching
|
||||
compound-term patterns against the lowercased topic string, then returns the
|
||||
priority-ordered peer subreddit list for that category.
|
||||
|
||||
The map is intentionally small, curated, and code-reviewed. Adding a new
|
||||
category is a code change; there is no user-editable override surface.
|
||||
|
||||
False-positive guard: every pattern is either a multi-word compound (e.g.
|
||||
"image generation", "text to image") or a domain-specific single word
|
||||
(e.g. "midjourney", "stablediffusion"). Bare common nouns like "image",
|
||||
"ai", or "model" are never used as patterns.
|
||||
|
||||
First-match-wins: categories are evaluated in declared order. Entries are
|
||||
sorted from most-specific to least-specific so narrower categories claim a
|
||||
topic before broader ones. For example, `ai_image_generation` appears
|
||||
before `ai_chat_model` so "gpt image 2" matches the image-gen category.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import List, Optional, TypedDict
|
||||
|
||||
|
||||
class _CategoryEntry(TypedDict):
|
||||
patterns: List[str]
|
||||
peer_subs: List[str]
|
||||
|
||||
|
||||
CATEGORY_PEERS: dict[str, _CategoryEntry] = {
|
||||
"ai_image_generation": {
|
||||
"patterns": [
|
||||
"image generation",
|
||||
"image gen",
|
||||
"text to image",
|
||||
"text-to-image",
|
||||
"gpt image",
|
||||
"gpt-image",
|
||||
"nano banana",
|
||||
"midjourney",
|
||||
"stable diffusion",
|
||||
"stablediffusion",
|
||||
"dall-e",
|
||||
"dalle",
|
||||
"flux.1",
|
||||
"flux schnell",
|
||||
"imagen",
|
||||
"seedance",
|
||||
"ideogram",
|
||||
"recraft",
|
||||
],
|
||||
"peer_subs": [
|
||||
"StableDiffusion",
|
||||
"midjourney",
|
||||
"dalle2",
|
||||
"aiArt",
|
||||
"PromptEngineering",
|
||||
"MediaSynthesis",
|
||||
],
|
||||
},
|
||||
"ai_video_generation": {
|
||||
"patterns": [
|
||||
"video generation",
|
||||
"text to video",
|
||||
"text-to-video",
|
||||
"sora",
|
||||
"veo 3",
|
||||
"veo3",
|
||||
"runway gen",
|
||||
"kling",
|
||||
"pika labs",
|
||||
"luma dream machine",
|
||||
"hailuo",
|
||||
],
|
||||
"peer_subs": [
|
||||
"aivideo",
|
||||
"StableDiffusion",
|
||||
"runwayml",
|
||||
"singularity",
|
||||
"MediaSynthesis",
|
||||
],
|
||||
},
|
||||
"ai_music_generation": {
|
||||
"patterns": [
|
||||
"music generation",
|
||||
"ai music",
|
||||
"suno",
|
||||
"udio",
|
||||
"riffusion",
|
||||
"stable audio",
|
||||
],
|
||||
"peer_subs": [
|
||||
"SunoAI",
|
||||
"udiomusic",
|
||||
"aimusic",
|
||||
"artificial",
|
||||
],
|
||||
},
|
||||
"ai_coding_agent": {
|
||||
"patterns": [
|
||||
"claude code",
|
||||
"cursor ide",
|
||||
"github copilot",
|
||||
"windsurf",
|
||||
"aider",
|
||||
"cline",
|
||||
"openclaw",
|
||||
"hermes agent",
|
||||
"continue.dev",
|
||||
"codeium",
|
||||
"sweep ai",
|
||||
"devin ai",
|
||||
"coding agent",
|
||||
"coding assistant",
|
||||
],
|
||||
"peer_subs": [
|
||||
"ChatGPTCoding",
|
||||
"LocalLLaMA",
|
||||
"singularity",
|
||||
"PromptEngineering",
|
||||
],
|
||||
},
|
||||
"ai_agent_framework": {
|
||||
"patterns": [
|
||||
"agent framework",
|
||||
"agentic framework",
|
||||
"langchain",
|
||||
"langgraph",
|
||||
"crewai",
|
||||
"autogen",
|
||||
"llamaindex",
|
||||
"dspy",
|
||||
"smolagents",
|
||||
],
|
||||
"peer_subs": [
|
||||
"LangChain",
|
||||
"LocalLLaMA",
|
||||
"AI_Agents",
|
||||
"MachineLearning",
|
||||
],
|
||||
},
|
||||
"ai_chat_model": {
|
||||
"patterns": [
|
||||
"gpt-5",
|
||||
"gpt-4",
|
||||
"claude opus",
|
||||
"claude sonnet",
|
||||
"claude haiku",
|
||||
"gemini pro",
|
||||
"gemini flash",
|
||||
"llama 3",
|
||||
"llama 4",
|
||||
"deepseek",
|
||||
"qwen",
|
||||
"mistral large",
|
||||
"grok",
|
||||
],
|
||||
"peer_subs": [
|
||||
"LocalLLaMA",
|
||||
"ChatGPT",
|
||||
"ClaudeAI",
|
||||
"singularity",
|
||||
"artificial",
|
||||
],
|
||||
},
|
||||
"saas_screen_recording": {
|
||||
"patterns": [
|
||||
"screen recording",
|
||||
"screen recorder",
|
||||
"loom video",
|
||||
"tella screen",
|
||||
"vidyard",
|
||||
"screen capture tool",
|
||||
],
|
||||
"peer_subs": [
|
||||
"SaaS",
|
||||
"screenrecording",
|
||||
"productivity",
|
||||
"Entrepreneur",
|
||||
],
|
||||
},
|
||||
"saas_productivity": {
|
||||
"patterns": [
|
||||
"notion app",
|
||||
"obsidian plugin",
|
||||
"obsidian app",
|
||||
"linear app",
|
||||
"asana",
|
||||
"clickup",
|
||||
"productivity app",
|
||||
],
|
||||
"peer_subs": [
|
||||
"productivity",
|
||||
"SaaS",
|
||||
"ObsidianMD",
|
||||
"Notion",
|
||||
],
|
||||
},
|
||||
"prediction_markets": {
|
||||
"patterns": [
|
||||
"polymarket",
|
||||
"kalshi",
|
||||
"prediction market",
|
||||
"event contracts",
|
||||
"manifold markets",
|
||||
],
|
||||
"peer_subs": [
|
||||
"Polymarket",
|
||||
"Kalshi",
|
||||
"predictionmarkets",
|
||||
],
|
||||
},
|
||||
"crypto_defi": {
|
||||
"patterns": [
|
||||
"defi protocol",
|
||||
"yield farming",
|
||||
"liquidity pool",
|
||||
"stablecoin",
|
||||
"ethereum layer",
|
||||
"layer 2",
|
||||
"l2 rollup",
|
||||
],
|
||||
"peer_subs": [
|
||||
"defi",
|
||||
"ethfinance",
|
||||
"CryptoCurrency",
|
||||
"ethereum",
|
||||
],
|
||||
},
|
||||
"dev_tool_cli": {
|
||||
"patterns": [
|
||||
"cli tool",
|
||||
"command line tool",
|
||||
"terminal app",
|
||||
"dev tool",
|
||||
],
|
||||
"peer_subs": [
|
||||
"commandline",
|
||||
"programming",
|
||||
"webdev",
|
||||
],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def detect_category(topic: Optional[str]) -> Optional[str]:
|
||||
"""Classify a topic into a known category by compound-term match.
|
||||
|
||||
Returns the category id (e.g. "ai_image_generation") or None if no
|
||||
category's patterns match. Matching is case-insensitive substring over
|
||||
the lowercased topic. Declaration order wins (first-match-wins), so the
|
||||
map is ordered from most-specific to least-specific.
|
||||
|
||||
A None or empty topic returns None. Classification never raises on
|
||||
normal string inputs; callers do not need to wrap in try/except for
|
||||
typical paths, though defensive callers may.
|
||||
"""
|
||||
if not topic:
|
||||
return None
|
||||
lowered = topic.lower()
|
||||
for category_id, entry in CATEGORY_PEERS.items():
|
||||
for pattern in entry["patterns"]:
|
||||
if pattern in lowered:
|
||||
return category_id
|
||||
return None
|
||||
|
||||
|
||||
def peer_subs_for(category_id: Optional[str]) -> List[str]:
|
||||
"""Return the priority-ordered peer subreddit list for a category.
|
||||
|
||||
Returns an empty list for None or unknown category ids. The returned
|
||||
list is a fresh copy; callers may safely mutate it.
|
||||
"""
|
||||
if not category_id:
|
||||
return []
|
||||
entry = CATEGORY_PEERS.get(category_id)
|
||||
if not entry:
|
||||
return []
|
||||
return list(entry["peer_subs"])
|
||||
+67
-5
@@ -11,14 +11,64 @@ import re
|
||||
import sys
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from datetime import datetime, timezone
|
||||
from typing import Optional
|
||||
|
||||
from . import dates, grounding
|
||||
from . import categories, dates, grounding
|
||||
|
||||
MAX_SUBS = 10
|
||||
|
||||
|
||||
def _log(msg: str) -> None:
|
||||
print(f"[Resolve] {msg}", file=sys.stderr)
|
||||
|
||||
|
||||
def _merge_category_peers(topic: str, subreddits: list[str]) -> tuple[list[str], Optional[str]]:
|
||||
"""Extend the WebSearch-extracted subreddit list with category peers.
|
||||
|
||||
Classifies the topic, fetches the category's peer subs, dedupes
|
||||
case-insensitively against the existing list, and appends missing
|
||||
peers in priority order. Caps the final list at MAX_SUBS, preserving
|
||||
every WebSearch-returned sub (they are the freshest signal) and
|
||||
trimming from the peer-additions end.
|
||||
|
||||
Returns a tuple of (merged_subs, matched_category_id_or_None).
|
||||
Emits a [Resolve] Matched category log line only when peers were
|
||||
actually added (not when every peer was already in the WebSearch set).
|
||||
|
||||
Classification failures degrade to "no match" — the unwidened list
|
||||
is returned and a warning is logged.
|
||||
"""
|
||||
try:
|
||||
category = categories.detect_category(topic)
|
||||
except Exception as exc:
|
||||
_log(f"Category classification failed: {exc}")
|
||||
return list(subreddits)[:MAX_SUBS], None
|
||||
|
||||
if category is None:
|
||||
return list(subreddits)[:MAX_SUBS], None
|
||||
|
||||
peers = categories.peer_subs_for(category)
|
||||
if not peers:
|
||||
return list(subreddits)[:MAX_SUBS], category
|
||||
|
||||
existing_lower = {s.lower() for s in subreddits}
|
||||
merged = list(subreddits)
|
||||
added: list[str] = []
|
||||
for peer in peers:
|
||||
if len(merged) >= MAX_SUBS:
|
||||
break
|
||||
if peer.lower() in existing_lower:
|
||||
continue
|
||||
merged.append(peer)
|
||||
existing_lower.add(peer.lower())
|
||||
added.append(peer)
|
||||
|
||||
if added:
|
||||
_log(f"Matched category={category}, adding peers: {', '.join(added)}")
|
||||
|
||||
return merged, category
|
||||
|
||||
|
||||
def _has_backend(config: dict) -> bool:
|
||||
"""Check if any web search backend is available."""
|
||||
return bool(
|
||||
@@ -134,10 +184,19 @@ def auto_resolve(topic: str, config: dict) -> dict:
|
||||
config: Dict with API keys (BRAVE_API_KEY, EXA_API_KEY, SERPER_API_KEY).
|
||||
|
||||
Returns:
|
||||
Dict with keys: subreddits, x_handle, context, searches_run.
|
||||
Returns empty result if no web search backend is available.
|
||||
Dict with keys: subreddits, x_handle, github_user, github_repos,
|
||||
context, category, searches_run. Returns empty result if no web
|
||||
search backend is available.
|
||||
"""
|
||||
empty = {"subreddits": [], "x_handle": "", "context": "", "searches_run": 0}
|
||||
empty = {
|
||||
"subreddits": [],
|
||||
"x_handle": "",
|
||||
"github_user": "",
|
||||
"github_repos": [],
|
||||
"context": "",
|
||||
"category": None,
|
||||
"searches_run": 0,
|
||||
}
|
||||
|
||||
if not _has_backend(config):
|
||||
_log("No web search backend available, skipping resolve")
|
||||
@@ -184,7 +243,9 @@ def auto_resolve(topic: str, config: dict) -> dict:
|
||||
github_repos = _extract_github_repos(results.get("github", []))
|
||||
context = _build_context_summary(results.get("news", []))
|
||||
|
||||
_log(f"Resolved {len(subreddits)} subreddits, x_handle={x_handle!r}, github_user={github_user!r}, github_repos={github_repos!r}, context_len={len(context)}")
|
||||
subreddits, category = _merge_category_peers(topic, subreddits)
|
||||
|
||||
_log(f"Resolved {len(subreddits)} subreddits, x_handle={x_handle!r}, github_user={github_user!r}, github_repos={github_repos!r}, context_len={len(context)}, category={category!r}")
|
||||
|
||||
return {
|
||||
"subreddits": subreddits,
|
||||
@@ -192,5 +253,6 @@ def auto_resolve(topic: str, config: dict) -> dict:
|
||||
"github_user": github_user,
|
||||
"github_repos": github_repos,
|
||||
"context": context,
|
||||
"category": category,
|
||||
"searches_run": searches_run,
|
||||
}
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
# Fixture: `Prompting GPT Image 2` Resolved-block regression
|
||||
|
||||
Documentation-grade fixture. Captures the pre-fix and post-fix shape of the
|
||||
Step 0.55 Resolved block for the topic `Prompting GPT Image 2`. Not parsed
|
||||
by test code — read by reviewers when evaluating regressions in
|
||||
`scripts/lib/categories.py` or the SKILL.md Step 0.55 block.
|
||||
|
||||
The live assertion lives in `tests/test_category_integration.py`. This
|
||||
markdown fixture exists so reviewers can eyeball expected behavior without
|
||||
running pytest.
|
||||
|
||||
## Failing run (2026-04-22, pre-fix)
|
||||
|
||||
User ran `/last30days Prompting GPT Image 2`. Step 0.55 WebSearch returned
|
||||
OpenAI-brand communities. The model resolved exactly those.
|
||||
|
||||
```
|
||||
Resolved:
|
||||
- X: @OpenAI (+ @sama, @openaidevs)
|
||||
- Reddit: r/OpenAI, r/ChatGPT, r/singularity, r/artificial, r/ChatGPTpromptengineering
|
||||
- TikTok: #gptimage2, #openai, #aiart
|
||||
```
|
||||
|
||||
Engine run returned thin results. User manually intervened with "make sure
|
||||
to check image generatorion reddits too" and re-ran with the image-gen
|
||||
peer subs added.
|
||||
|
||||
## Expected run (post-fix, no user intervention)
|
||||
|
||||
After Step 0.55 Section 2a (category-peer expansion) and Unit 2's engine-side
|
||||
merge in `auto_resolve`, the same topic produces:
|
||||
|
||||
```
|
||||
Resolved:
|
||||
- X: @OpenAI (+ @sama, @openaidevs)
|
||||
- Reddit: r/OpenAI, r/ChatGPT, r/singularity, r/ChatGPTpromptengineering, r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt (+ ai_image_generation peers)
|
||||
- TikTok: #gptimage2, #openai, #aiart
|
||||
```
|
||||
|
||||
The peer subs (`StableDiffusion, midjourney, dalle2, aiArt`) appear alongside
|
||||
the WebSearch-returned brand subs. The `(+ ai_image_generation peers)`
|
||||
annotation is the observable contract — its absence on a product-in-a-known-
|
||||
category topic is a Step 0.55 regression.
|
||||
|
||||
## Guards
|
||||
|
||||
- `tests/test_categories.py::DetectCategoryHappyPath::test_prompting_gpt_image_2_matches_image_generation`
|
||||
- `tests/test_resolve.py::MergeCategoryPeersHappyPath::test_image_gen_topic_appends_peers`
|
||||
- `tests/test_resolve.py::AutoResolveCategoryIntegration::test_auto_resolve_returns_category_key`
|
||||
- `tests/test_category_integration.py` — end-to-end over `auto_resolve` with
|
||||
a stubbed WebSearch that mimics the original failing response.
|
||||
|
||||
## When to update this fixture
|
||||
|
||||
- Category map changed (a peer sub was reordered, added, or removed).
|
||||
- The observable Resolved-block annotation format changed.
|
||||
- A new category was added that affects this topic.
|
||||
|
||||
Do not update casually. This file is the pre/post record of the 2026-04-22
|
||||
failure.
|
||||
@@ -0,0 +1,154 @@
|
||||
"""Unit tests for scripts/lib/categories.py — the Step 0.55 category-peer map.
|
||||
|
||||
Guards the 2026-04-22 `Prompting GPT Image 2` failure mode: the original bug
|
||||
was that Step 0.55 resolved only brand-adjacent subs (r/OpenAI, r/ChatGPT)
|
||||
and missed the category peers (r/StableDiffusion, r/midjourney, r/dalle2)
|
||||
where prompting techniques actually live.
|
||||
"""
|
||||
|
||||
import re
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts"))
|
||||
|
||||
from lib import categories
|
||||
from lib.categories import CATEGORY_PEERS, detect_category, peer_subs_for
|
||||
|
||||
|
||||
class DetectCategoryHappyPath(unittest.TestCase):
|
||||
def test_prompting_gpt_image_2_matches_image_generation(self):
|
||||
self.assertEqual(
|
||||
detect_category("Prompting GPT Image 2"),
|
||||
"ai_image_generation",
|
||||
)
|
||||
|
||||
def test_claude_code_matches_coding_agent(self):
|
||||
self.assertEqual(
|
||||
detect_category("Claude Code skills"),
|
||||
"ai_coding_agent",
|
||||
)
|
||||
|
||||
def test_suno_matches_music_generation(self):
|
||||
self.assertEqual(detect_category("Suno v4 review"), "ai_music_generation")
|
||||
|
||||
def test_polymarket_matches_prediction_markets(self):
|
||||
self.assertEqual(
|
||||
detect_category("Polymarket election odds"),
|
||||
"prediction_markets",
|
||||
)
|
||||
|
||||
def test_sora_matches_video_generation(self):
|
||||
self.assertEqual(detect_category("Sora 2 prompts"), "ai_video_generation")
|
||||
|
||||
|
||||
class PeerSubsForHappyPath(unittest.TestCase):
|
||||
def test_image_generation_peer_subs_priority_order(self):
|
||||
subs = peer_subs_for("ai_image_generation")
|
||||
self.assertIn("StableDiffusion", subs)
|
||||
self.assertIn("midjourney", subs)
|
||||
self.assertIn("dalle2", subs)
|
||||
self.assertLess(subs.index("StableDiffusion"), subs.index("midjourney"))
|
||||
self.assertLess(subs.index("midjourney"), subs.index("dalle2"))
|
||||
|
||||
def test_unknown_category_returns_empty_list(self):
|
||||
self.assertEqual(peer_subs_for("unknown_category"), [])
|
||||
|
||||
def test_none_category_returns_empty_list(self):
|
||||
self.assertEqual(peer_subs_for(None), [])
|
||||
|
||||
def test_returned_list_is_fresh_copy(self):
|
||||
first = peer_subs_for("ai_image_generation")
|
||||
first.append("MutatedSub")
|
||||
second = peer_subs_for("ai_image_generation")
|
||||
self.assertNotIn("MutatedSub", second)
|
||||
|
||||
|
||||
class DetectCategoryEdgeCases(unittest.TestCase):
|
||||
def test_case_insensitive_match(self):
|
||||
self.assertEqual(
|
||||
detect_category("STABLE DIFFUSION walkthrough"),
|
||||
"ai_image_generation",
|
||||
)
|
||||
|
||||
def test_non_category_topic_returns_none(self):
|
||||
self.assertIsNone(detect_category("Kanye West"))
|
||||
|
||||
def test_bare_image_word_does_not_trigger_image_generation(self):
|
||||
# Compound-term guard: "image" alone is not a pattern; only
|
||||
# multi-word compounds or domain-specific brand names match.
|
||||
self.assertIsNone(detect_category("image editing on my phone"))
|
||||
|
||||
def test_bare_ai_word_does_not_trigger_any_category(self):
|
||||
self.assertIsNone(detect_category("ai news today"))
|
||||
|
||||
def test_empty_topic_returns_none(self):
|
||||
self.assertIsNone(detect_category(""))
|
||||
|
||||
def test_none_topic_returns_none(self):
|
||||
self.assertIsNone(detect_category(None))
|
||||
|
||||
def test_first_match_wins_image_gen_before_chat_model(self):
|
||||
# "gpt image 2" contains "gpt image" (ai_image_generation) and the
|
||||
# substring "gpt" could resemble gpt-N chat-model patterns. The
|
||||
# narrower category wins because it is declared earlier.
|
||||
self.assertEqual(
|
||||
detect_category("gpt image 2 review"),
|
||||
"ai_image_generation",
|
||||
)
|
||||
|
||||
|
||||
class CategoryMapInvariants(unittest.TestCase):
|
||||
"""Regression guards on the map itself — catch accidental bare-word patterns."""
|
||||
|
||||
# Common nouns that would produce false positives if used as bare patterns.
|
||||
FORBIDDEN_BARE_PATTERNS = frozenset({
|
||||
"image", "video", "music", "ai", "model", "agent", "chat",
|
||||
"code", "cli", "app", "tool", "defi",
|
||||
})
|
||||
|
||||
def test_no_category_has_a_bare_common_noun_pattern(self):
|
||||
offenders = []
|
||||
for category_id, entry in CATEGORY_PEERS.items():
|
||||
for pattern in entry["patterns"]:
|
||||
if pattern.strip() in self.FORBIDDEN_BARE_PATTERNS:
|
||||
offenders.append((category_id, pattern))
|
||||
self.assertEqual(
|
||||
offenders,
|
||||
[],
|
||||
msg=(
|
||||
"Bare common-noun patterns cause false positives. "
|
||||
f"Offenders: {offenders}. Patterns must be compound "
|
||||
"(e.g. 'image generation') or domain-specific "
|
||||
"(e.g. 'midjourney')."
|
||||
),
|
||||
)
|
||||
|
||||
def test_every_category_has_at_least_one_compound_or_brand_pattern(self):
|
||||
multi_word_or_brand = re.compile(r"(\s|-|\.)|^[a-z][a-z0-9]{3,}$")
|
||||
for category_id, entry in CATEGORY_PEERS.items():
|
||||
patterns = entry["patterns"]
|
||||
self.assertTrue(patterns, f"{category_id} has no patterns")
|
||||
has_strong = any(multi_word_or_brand.search(p) for p in patterns)
|
||||
self.assertTrue(
|
||||
has_strong,
|
||||
f"{category_id} needs at least one multi-word or brand pattern",
|
||||
)
|
||||
|
||||
def test_every_category_has_at_least_two_peer_subs(self):
|
||||
for category_id, entry in CATEGORY_PEERS.items():
|
||||
self.assertGreaterEqual(
|
||||
len(entry["peer_subs"]),
|
||||
2,
|
||||
f"{category_id} should list at least 2 peer subs",
|
||||
)
|
||||
|
||||
def test_category_count_is_in_expected_range(self):
|
||||
# Sanity check: the map is intentionally small and curated.
|
||||
self.assertGreaterEqual(len(CATEGORY_PEERS), 8)
|
||||
self.assertLessEqual(len(CATEGORY_PEERS), 20)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,145 @@
|
||||
"""End-to-end regression test for the 2026-04-22 `Prompting GPT Image 2` bug.
|
||||
|
||||
Guards the failing run's Resolved-block shape end-to-end: stubs
|
||||
`grounding.web_search` to return the OpenAI-only subs that caused the
|
||||
original failure, then asserts that `auto_resolve` now returns the widened
|
||||
list and emits the expected stderr trace.
|
||||
|
||||
If this test starts failing after a `scripts/lib/categories.py` edit, either
|
||||
the fix regressed or the map intentionally dropped the `ai_image_generation`
|
||||
category — update the test deliberately.
|
||||
|
||||
Fixture reference: `tests/fixtures/prompting-gpt-image-2-resolved-block.md`.
|
||||
"""
|
||||
|
||||
import io
|
||||
import sys
|
||||
import unittest
|
||||
from contextlib import redirect_stderr
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts"))
|
||||
|
||||
from lib import resolve
|
||||
|
||||
|
||||
OPENAI_BRAND_SUBREDDIT_RESULTS = [
|
||||
{
|
||||
"title": "r/OpenAI community hub",
|
||||
"snippet": "Discussion at r/ChatGPT and r/singularity about GPT Image 2.",
|
||||
"url": "https://reddit.com/r/OpenAI/",
|
||||
},
|
||||
{
|
||||
"title": "r/ChatGPTpromptengineering prompt collection",
|
||||
"snippet": "Also see r/artificial for broader AI chatter.",
|
||||
"url": "",
|
||||
},
|
||||
]
|
||||
|
||||
EMPTY_RESULTS: list[dict] = []
|
||||
|
||||
|
||||
def _fake_websearch(label_to_items: dict[str, list[dict]]):
|
||||
def _search(query, date_range, config):
|
||||
if "subreddit" in query:
|
||||
return label_to_items.get("subreddit", EMPTY_RESULTS), {}
|
||||
if "news" in query:
|
||||
return label_to_items.get("news", EMPTY_RESULTS), {}
|
||||
if "handle" in query:
|
||||
return label_to_items.get("x_handle", EMPTY_RESULTS), {}
|
||||
if "github" in query:
|
||||
return label_to_items.get("github", EMPTY_RESULTS), {}
|
||||
return EMPTY_RESULTS, {}
|
||||
|
||||
return _search
|
||||
|
||||
|
||||
class PromptingGptImage2RegressionGuard(unittest.TestCase):
|
||||
"""The named 2026-04-22 failure mode. Resolved block must include peers."""
|
||||
|
||||
@patch("lib.resolve.grounding.web_search")
|
||||
def test_auto_resolve_widens_to_image_gen_peers(self, mock_search):
|
||||
mock_search.side_effect = _fake_websearch({
|
||||
"subreddit": OPENAI_BRAND_SUBREDDIT_RESULTS,
|
||||
})
|
||||
|
||||
result = resolve.auto_resolve(
|
||||
"Prompting GPT Image 2",
|
||||
{"BRAVE_API_KEY": "fake"},
|
||||
)
|
||||
|
||||
subs_lower = [s.lower() for s in result["subreddits"]]
|
||||
|
||||
# Original WebSearch-returned brand subs preserved
|
||||
self.assertIn("openai", subs_lower)
|
||||
self.assertIn("chatgpt", subs_lower)
|
||||
self.assertIn("singularity", subs_lower)
|
||||
|
||||
# At least three of the image-gen peers were added
|
||||
expected_peers = {"stablediffusion", "midjourney", "dalle2", "aiart", "promptengineering"}
|
||||
found_peers = expected_peers.intersection(subs_lower)
|
||||
self.assertGreaterEqual(
|
||||
len(found_peers),
|
||||
3,
|
||||
f"Expected at least 3 image-gen peer subs, found: {found_peers}. "
|
||||
f"Actual subs: {result['subreddits']}",
|
||||
)
|
||||
|
||||
self.assertEqual(result["category"], "ai_image_generation")
|
||||
|
||||
@patch("lib.resolve.grounding.web_search")
|
||||
def test_stderr_contains_category_match_log_line(self, mock_search):
|
||||
mock_search.side_effect = _fake_websearch({
|
||||
"subreddit": OPENAI_BRAND_SUBREDDIT_RESULTS,
|
||||
})
|
||||
|
||||
buf = io.StringIO()
|
||||
with redirect_stderr(buf):
|
||||
resolve.auto_resolve(
|
||||
"Prompting GPT Image 2",
|
||||
{"BRAVE_API_KEY": "fake"},
|
||||
)
|
||||
|
||||
self.assertIn("Matched category=ai_image_generation", buf.getvalue())
|
||||
|
||||
@patch("lib.resolve.grounding.web_search")
|
||||
def test_cap_enforced_end_to_end(self, mock_search):
|
||||
# Synthesize a subreddit response with 9 brand subs
|
||||
many_subs_items = [
|
||||
{"title": f"r/Brand{i}", "snippet": "", "url": ""}
|
||||
for i in range(9)
|
||||
]
|
||||
mock_search.side_effect = _fake_websearch({
|
||||
"subreddit": many_subs_items,
|
||||
})
|
||||
|
||||
result = resolve.auto_resolve(
|
||||
"Prompting GPT Image 2",
|
||||
{"BRAVE_API_KEY": "fake"},
|
||||
)
|
||||
|
||||
self.assertLessEqual(len(result["subreddits"]), resolve.MAX_SUBS)
|
||||
# The first WebSearch sub is still present (brand subs never evicted)
|
||||
self.assertIn("Brand0", result["subreddits"])
|
||||
|
||||
@patch("lib.resolve.grounding.web_search")
|
||||
def test_uncategorized_topic_does_not_inject_peers(self, mock_search):
|
||||
mock_search.side_effect = _fake_websearch({
|
||||
"subreddit": [{"title": "r/Kanye is wild", "snippet": "", "url": ""}],
|
||||
})
|
||||
|
||||
buf = io.StringIO()
|
||||
with redirect_stderr(buf):
|
||||
result = resolve.auto_resolve(
|
||||
"Kanye West latest album",
|
||||
{"BRAVE_API_KEY": "fake"},
|
||||
)
|
||||
|
||||
self.assertEqual(result["subreddits"], ["Kanye"])
|
||||
self.assertIsNone(result["category"])
|
||||
self.assertNotIn("Matched category=", buf.getvalue())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+179
-4
@@ -1,11 +1,14 @@
|
||||
import io
|
||||
import sys
|
||||
import unittest
|
||||
from contextlib import redirect_stderr
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts"))
|
||||
|
||||
from lib import resolve
|
||||
from lib.resolve import MAX_SUBS, _merge_category_peers
|
||||
|
||||
|
||||
class TestHasBackend(unittest.TestCase):
|
||||
@@ -137,8 +140,8 @@ class TestAutoResolve(unittest.TestCase):
|
||||
self.assertEqual(result["subreddits"], ["technology", "gadgets"])
|
||||
self.assertEqual(result["x_handle"], "techco")
|
||||
self.assertIn("breakthrough", result["context"])
|
||||
self.assertEqual(result["searches_run"], 3)
|
||||
self.assertEqual(mock_search.call_count, 3)
|
||||
self.assertEqual(result["searches_run"], 4)
|
||||
self.assertEqual(mock_search.call_count, 4)
|
||||
|
||||
@patch("lib.resolve.grounding.web_search")
|
||||
def test_search_failure_graceful(self, mock_search):
|
||||
@@ -167,8 +170,180 @@ class TestAutoResolve(unittest.TestCase):
|
||||
self.assertEqual(result["subreddits"], ["cooking"])
|
||||
# News search failed, so context is empty
|
||||
self.assertEqual(result["context"], "")
|
||||
# 2 out of 3 succeeded
|
||||
self.assertEqual(result["searches_run"], 2)
|
||||
# 3 out of 4 succeeded (subreddit, x_handle, github; news failed)
|
||||
self.assertEqual(result["searches_run"], 3)
|
||||
|
||||
|
||||
class MergeCategoryPeersHappyPath(unittest.TestCase):
|
||||
def test_image_gen_topic_appends_peers(self):
|
||||
merged, category = _merge_category_peers(
|
||||
"Prompting GPT Image 2",
|
||||
["OpenAI", "ChatGPT", "singularity"],
|
||||
)
|
||||
self.assertEqual(category, "ai_image_generation")
|
||||
self.assertIn("OpenAI", merged)
|
||||
self.assertIn("ChatGPT", merged)
|
||||
self.assertIn("singularity", merged)
|
||||
self.assertIn("StableDiffusion", merged)
|
||||
self.assertIn("midjourney", merged)
|
||||
self.assertIn("dalle2", merged)
|
||||
|
||||
def test_preserves_websearch_order_then_appends_peers(self):
|
||||
merged, _ = _merge_category_peers(
|
||||
"Prompting GPT Image 2",
|
||||
["OpenAI", "ChatGPT"],
|
||||
)
|
||||
self.assertEqual(merged[0], "OpenAI")
|
||||
self.assertEqual(merged[1], "ChatGPT")
|
||||
self.assertEqual(merged[2], "StableDiffusion")
|
||||
|
||||
def test_emits_stderr_log_when_peers_added(self):
|
||||
buf = io.StringIO()
|
||||
with redirect_stderr(buf):
|
||||
_merge_category_peers(
|
||||
"Prompting GPT Image 2",
|
||||
["OpenAI", "ChatGPT"],
|
||||
)
|
||||
output = buf.getvalue()
|
||||
self.assertIn("Matched category=ai_image_generation", output)
|
||||
self.assertIn("StableDiffusion", output)
|
||||
|
||||
|
||||
class MergeCategoryPeersDedupe(unittest.TestCase):
|
||||
def test_peer_already_in_websearch_not_duplicated(self):
|
||||
merged, _ = _merge_category_peers(
|
||||
"midjourney v7 prompts",
|
||||
["midjourney", "aiArt"],
|
||||
)
|
||||
self.assertEqual(
|
||||
sum(1 for s in merged if s.lower() == "midjourney"),
|
||||
1,
|
||||
)
|
||||
|
||||
def test_dedupe_is_case_insensitive(self):
|
||||
merged, _ = _merge_category_peers(
|
||||
"Prompting GPT Image 2",
|
||||
["STABLEDIFFUSION"],
|
||||
)
|
||||
lower = [s.lower() for s in merged]
|
||||
self.assertEqual(lower.count("stablediffusion"), 1)
|
||||
|
||||
def test_no_log_when_all_peers_already_present(self):
|
||||
buf = io.StringIO()
|
||||
with redirect_stderr(buf):
|
||||
_merge_category_peers(
|
||||
"Prompting GPT Image 2",
|
||||
[
|
||||
"StableDiffusion",
|
||||
"midjourney",
|
||||
"dalle2",
|
||||
"aiArt",
|
||||
"PromptEngineering",
|
||||
"MediaSynthesis",
|
||||
],
|
||||
)
|
||||
self.assertNotIn("Matched category=", buf.getvalue())
|
||||
|
||||
|
||||
class MergeCategoryPeersEdgeCases(unittest.TestCase):
|
||||
def test_topic_with_no_category_returns_unchanged(self):
|
||||
merged, category = _merge_category_peers(
|
||||
"Kanye West",
|
||||
["Kanye", "hiphopheads"],
|
||||
)
|
||||
self.assertIsNone(category)
|
||||
self.assertEqual(merged, ["Kanye", "hiphopheads"])
|
||||
|
||||
def test_empty_subreddit_list_with_category_still_adds_peers(self):
|
||||
merged, category = _merge_category_peers("Prompting GPT Image 2", [])
|
||||
self.assertEqual(category, "ai_image_generation")
|
||||
self.assertIn("StableDiffusion", merged)
|
||||
|
||||
def test_empty_topic_returns_unchanged(self):
|
||||
merged, category = _merge_category_peers("", ["foo", "bar"])
|
||||
self.assertIsNone(category)
|
||||
self.assertEqual(merged, ["foo", "bar"])
|
||||
|
||||
def test_none_topic_returns_unchanged(self):
|
||||
merged, category = _merge_category_peers(None, ["foo", "bar"])
|
||||
self.assertIsNone(category)
|
||||
self.assertEqual(merged, ["foo", "bar"])
|
||||
|
||||
def test_no_log_when_topic_has_no_category(self):
|
||||
buf = io.StringIO()
|
||||
with redirect_stderr(buf):
|
||||
_merge_category_peers("Kanye West", ["Kanye"])
|
||||
self.assertNotIn("Matched category=", buf.getvalue())
|
||||
|
||||
|
||||
class MergeCategoryPeersCap(unittest.TestCase):
|
||||
def test_cap_is_enforced_at_max_subs(self):
|
||||
websearch_subs = [f"Sub{i}" for i in range(9)]
|
||||
merged, _ = _merge_category_peers(
|
||||
"Prompting GPT Image 2",
|
||||
websearch_subs,
|
||||
)
|
||||
self.assertEqual(len(merged), MAX_SUBS)
|
||||
for s in websearch_subs:
|
||||
self.assertIn(s, merged)
|
||||
self.assertEqual(len(merged) - len(websearch_subs), 1)
|
||||
self.assertEqual(merged[9], "StableDiffusion")
|
||||
|
||||
def test_cap_preserves_highest_priority_peer_when_trimming(self):
|
||||
websearch_subs = [f"Sub{i}" for i in range(8)]
|
||||
merged, _ = _merge_category_peers(
|
||||
"Prompting GPT Image 2",
|
||||
websearch_subs,
|
||||
)
|
||||
self.assertEqual(len(merged), MAX_SUBS)
|
||||
self.assertEqual(merged[8], "StableDiffusion")
|
||||
self.assertEqual(merged[9], "midjourney")
|
||||
|
||||
|
||||
class MergeCategoryPeersClassificationFailure(unittest.TestCase):
|
||||
def test_classification_error_returns_unwidened_list_and_logs(self):
|
||||
original = resolve.categories.detect_category
|
||||
|
||||
def boom(_topic):
|
||||
raise RuntimeError("synthetic classifier failure")
|
||||
|
||||
resolve.categories.detect_category = boom
|
||||
try:
|
||||
buf = io.StringIO()
|
||||
with redirect_stderr(buf):
|
||||
merged, category = _merge_category_peers(
|
||||
"Prompting GPT Image 2",
|
||||
["OpenAI"],
|
||||
)
|
||||
self.assertEqual(merged, ["OpenAI"])
|
||||
self.assertIsNone(category)
|
||||
self.assertIn("Category classification failed", buf.getvalue())
|
||||
finally:
|
||||
resolve.categories.detect_category = original
|
||||
|
||||
|
||||
class AutoResolveCategoryIntegration(unittest.TestCase):
|
||||
@patch("lib.resolve.grounding.web_search")
|
||||
def test_auto_resolve_returns_category_key(self, mock_search):
|
||||
def side_effect(query, date_range, config):
|
||||
if "subreddit" in query:
|
||||
return [
|
||||
{"title": "r/OpenAI", "snippet": "r/ChatGPT r/singularity", "url": ""},
|
||||
], {}
|
||||
return [], {}
|
||||
|
||||
mock_search.side_effect = side_effect
|
||||
result = resolve.auto_resolve(
|
||||
"Prompting GPT Image 2",
|
||||
{"BRAVE_API_KEY": "fake"},
|
||||
)
|
||||
self.assertEqual(result["category"], "ai_image_generation")
|
||||
self.assertIn("StableDiffusion", result["subreddits"])
|
||||
self.assertIn("OpenAI", result["subreddits"])
|
||||
|
||||
def test_no_backend_returns_category_none(self):
|
||||
result = resolve.auto_resolve("test topic", {})
|
||||
self.assertIsNone(result["category"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Reference in New Issue
Block a user