fix: YouTube timeout bump to 90s + Reddit 429 fail-fast
YouTube: Add youtube_future timeout key (60/90/120s for quick/default/deep) separate from the shared future timeout. YouTube needs more time because it does search + parallel transcript fetching. Previously, 20 videos + 5 transcripts exceeded the 60s budget and all results were discarded. Reddit 429: Propagate rate-limit errors instead of swallowing them. Enrichment now uses 10s timeout / 1 retry (was 30s / 3 retries). On first 429, cancel remaining enrichment and skip Phase 2 Reddit. Total time wasted on 429 drops from ~75s to ~12s. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
+4
-2
@@ -124,11 +124,13 @@ def post(url: str, json_data: Dict[str, Any], headers: Optional[Dict[str, str]]
|
||||
return request("POST", url, headers=headers, json_data=json_data, **kwargs)
|
||||
|
||||
|
||||
def get_reddit_json(path: str) -> Dict[str, Any]:
|
||||
def get_reddit_json(path: str, timeout: int = DEFAULT_TIMEOUT, retries: int = MAX_RETRIES) -> Dict[str, Any]:
|
||||
"""Fetch Reddit thread JSON.
|
||||
|
||||
Args:
|
||||
path: Reddit path (e.g., /r/subreddit/comments/id/title)
|
||||
timeout: HTTP timeout per attempt in seconds
|
||||
retries: Number of retries on failure
|
||||
|
||||
Returns:
|
||||
Parsed JSON response
|
||||
@@ -149,4 +151,4 @@ def get_reddit_json(path: str) -> Dict[str, Any]:
|
||||
"Accept": "application/json",
|
||||
}
|
||||
|
||||
return get(url, headers=headers)
|
||||
return get(url, headers=headers, timeout=timeout, retries=retries)
|
||||
|
||||
Reference in New Issue
Block a user