fix: Step 0.55 category-peer subreddit expansion (#305)

* feat(resolve): category-peer subreddit map for Step 0.55

Introduces scripts/lib/categories.py with a curated category->peer-subs
map and wires scripts/lib/resolve.py auto_resolve() to merge peers into
the WebSearch-extracted subreddit list. Named 2026-04-22 failure mode:
a "Prompting GPT Image 2" run resolved only r/OpenAI + r/ChatGPT and
missed r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt where
prompting techniques actually live.

Map is static, curated, ~11 categories (ai_image_generation,
ai_video_generation, ai_music_generation, ai_coding_agent,
ai_agent_framework, ai_chat_model, saas_screen_recording,
saas_productivity, prediction_markets, crypto_defi, dev_tool_cli).
First-match-wins ordering from most-specific to least-specific.
Compound-term patterns only (no bare common nouns like "image", "ai").

auto_resolve now:
- calls detect_category(topic) after _extract_subreddits
- merges peer_subs case-insensitively, caps at MAX_SUBS (10)
- preserves every WebSearch-returned sub (freshest signal)
- emits [Resolve] Matched category=<id>, adding peers: <list> on stderr
  only when peers were actually added
- returns new "category" key in the result dict for observability
- wraps classifier in try/except so failures degrade to unwidened list

Includes drive-by: test_full_resolve / test_partial_failure
searches_run expectations bumped from 3->4 / 2->3 to match the current
queries dict (subreddit + news + x_handle + github).

* feat(skill): Step 0.55 category-peer expansion and self-check

Adds Section 2a (category-peer expansion, MANDATORY for product topics)
and the Step 0.55 self-check checkpoint that fires immediately before
the Resolved block displays. Structural mirror of the engine-side
categories.py map: same categories, same peer subs, same priority
order.

The model-side path now:
- Applies category-peer expansion to the WebSearch-resolved subs on
  every product-in-a-known-category run.
- Emits the (+ <category_id> peers) annotation on the Reddit line of
  the Resolved block as the observable contract. Absence on a
  product-in-a-known-category topic is a Step 0.55 regression.
- Runs a self-check before emitting Resolved: "does the resolved list
  include at least 2 peer subs for the matched category? if not,
  widen NOW and do not run the engine yet."

Mirror of the Python map lives inside Step 0.55 as a table for the
model to pattern-match against; extrapolation to unlisted categories
is explicitly allowed. Worked example (the exact failing query)
appears below the table so reviewers can see before/after at a glance.

Both changes land inside the existing Step 0.55 block. No new
top-level section, no new LAW. LAWs 1-6 wording unchanged.

* test: end-to-end regression for GPT Image 2 failure mode

Stubs grounding.web_search to return the OpenAI-only subs that caused
the 2026-04-22 failure, then asserts that auto_resolve widens to
include the image-gen peers and emits the [Resolve] Matched
category=ai_image_generation stderr line. Covers the cap boundary
and the uncategorized-topic no-op path.

Fixture tests/fixtures/prompting-gpt-image-2-resolved-block.md is
documentation-grade (not parsed by tests) and shows the pre-fix vs
post-fix Resolved block shape so reviewers can evaluate future
categories.py edits against the original bug.

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
This commit is contained in:
Matt Van Horn
2026-04-22 14:31:39 -07:00
committed by GitHub
parent 952a876536
commit 4e91f4e754
7 changed files with 929 additions and 11 deletions
+60
View File
@@ -0,0 +1,60 @@
# Fixture: `Prompting GPT Image 2` Resolved-block regression
Documentation-grade fixture. Captures the pre-fix and post-fix shape of the
Step 0.55 Resolved block for the topic `Prompting GPT Image 2`. Not parsed
by test code — read by reviewers when evaluating regressions in
`scripts/lib/categories.py` or the SKILL.md Step 0.55 block.
The live assertion lives in `tests/test_category_integration.py`. This
markdown fixture exists so reviewers can eyeball expected behavior without
running pytest.
## Failing run (2026-04-22, pre-fix)
User ran `/last30days Prompting GPT Image 2`. Step 0.55 WebSearch returned
OpenAI-brand communities. The model resolved exactly those.
```
Resolved:
- X: @OpenAI (+ @sama, @openaidevs)
- Reddit: r/OpenAI, r/ChatGPT, r/singularity, r/artificial, r/ChatGPTpromptengineering
- TikTok: #gptimage2, #openai, #aiart
```
Engine run returned thin results. User manually intervened with "make sure
to check image generatorion reddits too" and re-ran with the image-gen
peer subs added.
## Expected run (post-fix, no user intervention)
After Step 0.55 Section 2a (category-peer expansion) and Unit 2's engine-side
merge in `auto_resolve`, the same topic produces:
```
Resolved:
- X: @OpenAI (+ @sama, @openaidevs)
- Reddit: r/OpenAI, r/ChatGPT, r/singularity, r/ChatGPTpromptengineering, r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt (+ ai_image_generation peers)
- TikTok: #gptimage2, #openai, #aiart
```
The peer subs (`StableDiffusion, midjourney, dalle2, aiArt`) appear alongside
the WebSearch-returned brand subs. The `(+ ai_image_generation peers)`
annotation is the observable contract — its absence on a product-in-a-known-
category topic is a Step 0.55 regression.
## Guards
- `tests/test_categories.py::DetectCategoryHappyPath::test_prompting_gpt_image_2_matches_image_generation`
- `tests/test_resolve.py::MergeCategoryPeersHappyPath::test_image_gen_topic_appends_peers`
- `tests/test_resolve.py::AutoResolveCategoryIntegration::test_auto_resolve_returns_category_key`
- `tests/test_category_integration.py` — end-to-end over `auto_resolve` with
a stubbed WebSearch that mimics the original failing response.
## When to update this fixture
- Category map changed (a peer sub was reordered, added, or removed).
- The observable Resolved-block annotation format changed.
- A new category was added that affects this topic.
Do not update casually. This file is the pre/post record of the 2026-04-22
failure.