fix: Simplify Reddit prompt and increase search volume

OpenAI's web_search doesn't reliably find recent Reddit content.
Strategy change: request MORE threads and let server-side date
filtering handle it.

Changes:
- Increased depth config (20-30 → 30-50 default)
- Simplified prompt to search broadly and include all matches
- Let server-side date filter remove old content
- Removed aggressive pre-filtering that was limiting results

Tradeoff: Some topics have more X coverage than Reddit due to
web_search limitations. This is acceptable.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Matt Van Horn
2026-01-25 10:35:14 -08:00
parent acd7a533c0
commit 2d43571875
+25 -25
View File
@@ -16,54 +16,54 @@ def _log_error(msg: str):
OPENAI_RESPONSES_URL = "https://api.openai.com/v1/responses"
# Depth configurations: (min, max) threads to request
# Request MORE than needed since many get filtered by date
DEPTH_CONFIG = {
"quick": (8, 12),
"default": (20, 30),
"deep": (50, 70),
"quick": (15, 25),
"default": (30, 50),
"deep": (70, 100),
}
REDDIT_SEARCH_PROMPT = """Find RECENT Reddit discussion threads about: {topic}
REDDIT_SEARCH_PROMPT = """Find Reddit discussion threads about: {topic}
STEP 1: EXTRACT THE CORE SUBJECT
Strip qualifiers and get the MAIN NOUN/PRODUCT/TOPIC:
Get the MAIN NOUN/PRODUCT/TOPIC:
- "best nano banana prompting practices""nano banana"
- "killer features of clawdbot""clawdbot"
- "top Claude Code skills""Claude Code skills"
DO NOT search for "best", "top", "tips", "practices" - just the core subject.
- "top Claude Code skills""Claude Code"
DO NOT include "best", "top", "tips", "practices", "features" in your search.
STEP 2: SEARCH FOR RECENT CONTENT (CRITICAL)
You MUST search with date filters to find recent threads:
1. "[core subject] site:reddit.com after:{from_date}"
2. "reddit [core subject] 2026" or "reddit [core subject] January 2026"
3. "[core subject] site:reddit.com/r/"
STEP 2: SEARCH BROADLY
Search for the core subject:
1. "[core subject] site:reddit.com"
2. "reddit [core subject]"
3. "[core subject] reddit"
The goal is threads from {from_date} to {to_date}. Older threads are NOT useful.
Return as many relevant threads as you find. We filter by date server-side.
STEP 3: VALIDATE DATES
Before including a thread, verify its date is AFTER {from_date}.
If you cannot determine the date, set date to null.
EXCLUDE threads you know are older than {from_date}.
STEP 3: INCLUDE ALL MATCHES
- Include ALL threads about the core subject
- Set date to "YYYY-MM-DD" if you can determine it, otherwise null
- We verify dates and filter old content server-side
- DO NOT pre-filter aggressively - include anything relevant
REQUIRED URL FORMAT: Must contain "/r/" AND "/comments/"
REJECT: developers.reddit.com, business.reddit.com, reddit.com/user/
REQUIRED: URLs must contain "/r/" AND "/comments/"
REJECT: developers.reddit.com, business.reddit.com
Find {min_items}-{max_items} RECENT discussion threads (from last 30 days).
Find {min_items}-{max_items} threads. Return MORE rather than fewer.
Return JSON:
{{
"items": [
{{
"title": "Thread title",
"url": "https://www.reddit.com/r/subreddit/comments/xyz/title/",
"url": "https://www.reddit.com/r/sub/comments/xyz/title/",
"subreddit": "subreddit_name",
"date": "YYYY-MM-DD or null",
"why_relevant": "How this relates to {topic}",
"why_relevant": "Why relevant",
"relevance": 0.85
}}
]
}}
IMPORTANT: Only return threads from the last 30 days. Old threads will be filtered out anyway."""
}}"""
def search_reddit(