fix: YouTube timeout bump to 90s + Reddit 429 fail-fast

YouTube: Add youtube_future timeout key (60/90/120s for quick/default/deep)
separate from the shared future timeout. YouTube needs more time because
it does search + parallel transcript fetching. Previously, 20 videos +
5 transcripts exceeded the 60s budget and all results were discarded.

Reddit 429: Propagate rate-limit errors instead of swallowing them.
Enrichment now uses 10s timeout / 1 retry (was 30s / 3 retries).
On first 429, cancel remaining enrichment and skip Phase 2 Reddit.
Total time wasted on 429 drops from ~75s to ~12s.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Matt Van Horn
2026-02-15 00:48:26 -08:00
parent c3640931ed
commit 20a859ecec
4 changed files with 61 additions and 15 deletions
+4 -2
View File
@@ -124,11 +124,13 @@ def post(url: str, json_data: Dict[str, Any], headers: Optional[Dict[str, str]]
return request("POST", url, headers=headers, json_data=json_data, **kwargs)
def get_reddit_json(path: str) -> Dict[str, Any]:
def get_reddit_json(path: str, timeout: int = DEFAULT_TIMEOUT, retries: int = MAX_RETRIES) -> Dict[str, Any]:
"""Fetch Reddit thread JSON.
Args:
path: Reddit path (e.g., /r/subreddit/comments/id/title)
timeout: HTTP timeout per attempt in seconds
retries: Number of retries on failure
Returns:
Parsed JSON response
@@ -149,4 +151,4 @@ def get_reddit_json(path: str) -> Dict[str, Any]:
"Accept": "application/json",
}
return get(url, headers=headers)
return get(url, headers=headers, timeout=timeout, retries=retries)