Matt Van Horn
4e91f4e754
* feat(resolve): category-peer subreddit map for Step 0.55 Introduces scripts/lib/categories.py with a curated category->peer-subs map and wires scripts/lib/resolve.py auto_resolve() to merge peers into the WebSearch-extracted subreddit list. Named 2026-04-22 failure mode: a "Prompting GPT Image 2" run resolved only r/OpenAI + r/ChatGPT and missed r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt where prompting techniques actually live. Map is static, curated, ~11 categories (ai_image_generation, ai_video_generation, ai_music_generation, ai_coding_agent, ai_agent_framework, ai_chat_model, saas_screen_recording, saas_productivity, prediction_markets, crypto_defi, dev_tool_cli). First-match-wins ordering from most-specific to least-specific. Compound-term patterns only (no bare common nouns like "image", "ai"). auto_resolve now: - calls detect_category(topic) after _extract_subreddits - merges peer_subs case-insensitively, caps at MAX_SUBS (10) - preserves every WebSearch-returned sub (freshest signal) - emits [Resolve] Matched category=<id>, adding peers: <list> on stderr only when peers were actually added - returns new "category" key in the result dict for observability - wraps classifier in try/except so failures degrade to unwidened list Includes drive-by: test_full_resolve / test_partial_failure searches_run expectations bumped from 3->4 / 2->3 to match the current queries dict (subreddit + news + x_handle + github). * feat(skill): Step 0.55 category-peer expansion and self-check Adds Section 2a (category-peer expansion, MANDATORY for product topics) and the Step 0.55 self-check checkpoint that fires immediately before the Resolved block displays. Structural mirror of the engine-side categories.py map: same categories, same peer subs, same priority order. The model-side path now: - Applies category-peer expansion to the WebSearch-resolved subs on every product-in-a-known-category run. - Emits the (+ <category_id> peers) annotation on the Reddit line of the Resolved block as the observable contract. Absence on a product-in-a-known-category topic is a Step 0.55 regression. - Runs a self-check before emitting Resolved: "does the resolved list include at least 2 peer subs for the matched category? if not, widen NOW and do not run the engine yet." Mirror of the Python map lives inside Step 0.55 as a table for the model to pattern-match against; extrapolation to unlisted categories is explicitly allowed. Worked example (the exact failing query) appears below the table so reviewers can see before/after at a glance. Both changes land inside the existing Step 0.55 block. No new top-level section, no new LAW. LAWs 1-6 wording unchanged. * test: end-to-end regression for GPT Image 2 failure mode Stubs grounding.web_search to return the OpenAI-only subs that caused the 2026-04-22 failure, then asserts that auto_resolve widens to include the image-gen peers and emits the [Resolve] Matched category=ai_image_generation stderr line. Covers the cap boundary and the uncategorized-topic no-op path. Fixture tests/fixtures/prompting-gpt-image-2-resolved-block.md is documentation-grade (not parsed by tests) and shows the pre-fix vs post-fix Resolved block shape so reviewers can evaluate future categories.py edits against the original bug. --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
2.4 KiB
Fixture: Prompting GPT Image 2 Resolved-block regression
Documentation-grade fixture. Captures the pre-fix and post-fix shape of the
Step 0.55 Resolved block for the topic Prompting GPT Image 2. Not parsed
by test code — read by reviewers when evaluating regressions in
scripts/lib/categories.py or the SKILL.md Step 0.55 block.
The live assertion lives in tests/test_category_integration.py. This
markdown fixture exists so reviewers can eyeball expected behavior without
running pytest.
Failing run (2026-04-22, pre-fix)
User ran /last30days Prompting GPT Image 2. Step 0.55 WebSearch returned
OpenAI-brand communities. The model resolved exactly those.
Resolved:
- X: @OpenAI (+ @sama, @openaidevs)
- Reddit: r/OpenAI, r/ChatGPT, r/singularity, r/artificial, r/ChatGPTpromptengineering
- TikTok: #gptimage2, #openai, #aiart
Engine run returned thin results. User manually intervened with "make sure to check image generatorion reddits too" and re-ran with the image-gen peer subs added.
Expected run (post-fix, no user intervention)
After Step 0.55 Section 2a (category-peer expansion) and Unit 2's engine-side
merge in auto_resolve, the same topic produces:
Resolved:
- X: @OpenAI (+ @sama, @openaidevs)
- Reddit: r/OpenAI, r/ChatGPT, r/singularity, r/ChatGPTpromptengineering, r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt (+ ai_image_generation peers)
- TikTok: #gptimage2, #openai, #aiart
The peer subs (StableDiffusion, midjourney, dalle2, aiArt) appear alongside
the WebSearch-returned brand subs. The (+ ai_image_generation peers)
annotation is the observable contract — its absence on a product-in-a-known-
category topic is a Step 0.55 regression.
Guards
tests/test_categories.py::DetectCategoryHappyPath::test_prompting_gpt_image_2_matches_image_generationtests/test_resolve.py::MergeCategoryPeersHappyPath::test_image_gen_topic_appends_peerstests/test_resolve.py::AutoResolveCategoryIntegration::test_auto_resolve_returns_category_keytests/test_category_integration.py— end-to-end overauto_resolvewith a stubbed WebSearch that mimics the original failing response.
When to update this fixture
- Category map changed (a peer sub was reordered, added, or removed).
- The observable Resolved-block annotation format changed.
- A new category was added that affects this topic.
Do not update casually. This file is the pre/post record of the 2026-04-22 failure.