Transient DNS resolution failures (socket.gaierror, surfaced as
urllib.error.URLError with reason=gaierror) were retried with the
generic URLError handler — linear backoff (2s, 4s, 6s) and bounded by
the caller-passed `retries` parameter. For callers that pass small
retry values (e.g. lib/reddit.py::_subreddit_search uses retries=2), a
single first-attempt DNS hiccup followed by one quick retry on the
still-flaky resolver would exhaust the retry budget and wipe a whole
subreddit sweep — which the caller's broad `except Exception` then
silent-empties as `[]`.
Fix:
- Distinguish URLError-with-gaierror-reason from generic URLError via
a new `_is_dns_failure()` helper.
- For DNS failures, use exponential backoff (1s, 2s, 4s, ...) instead
of the linear default.
- For DNS failures, expand the effective retry budget to at least
MIN_DNS_RETRIES (=3) on first occurrence, so callers that passed
`retries=2` still get a meaningful retry budget for the transient
case. Non-DNS URLErrors and HTTPErrors keep the caller's value.
- DNS attempts are counted separately (`dns_attempts`) so unrelated
URLError or OSError failures within the same call don't accidentally
expand the budget further.
Reported during a community-signal pass where the Reddit subreddit
sweep silently returned zero items after a first-round transient DNS
hiccup. The fix lives at the http layer (where the retry loop is)
rather than per-source so every caller benefits.
Tests:
- Verifies a caller-passed retries=2 still gets MIN_DNS_RETRIES=3
attempts on gaierror.
- Verifies gaierror-then-success returns successfully on attempt 2.
- Verifies the exponential-backoff sleep pattern (1s, 2s) on the
retry attempts before exhaustion.
- Verifies a non-DNS URLError (ConnectionRefusedError reason) does
NOT expand the retry budget — only true DNS failures do.
All 12 http tests pass (8 baseline + 4 new). No regressions in the
broader test suite (1373 pass / 14 fail, vs 1369 pass / 14 fail on
main — the 14 failures are pre-existing and unrelated to this PR).
Twitter's edge intermittently serves an HTML anti-bot interstitial in
place of JSON when the bird-search subprocess hits a per-query rate
limit. Before this fix, that response made json.loads raise
JSONDecodeError and _run_bird_search() returned {"error": ..., "items":
[]} with the parsed exception message — silent-empty against an
orchestrator that has no way to distinguish "Twitter served HTML; retry
likely succeeds" from "no tweets matched the query."
Surfaced during a community-signal pass where a Karpathy-LLM-wiki
subquery returned zero X items, while a second identical run a few
seconds later returned full results.
Fix:
- Extract the subprocess invocation into _invoke_bird_subprocess() so
the retry loop can call it multiple times cleanly. Returns
(result, terminal_error) — terminal_error is non-None for
unrecoverable cases (subprocess timeout, spawn failure) that should
NOT be retried.
- In _run_bird_search(), wrap the json.loads parse in a retry loop
bounded by MAX_JSON_DECODE_RETRIES (=2) with JSON_DECODE_RETRY_DELAY
(=5s) between attempts.
- On non-JSON stdout, log a diagnostic that names the shape
(`looks_html`, first-80-chars stdout preview, attempt counter) so
silent-empty failures become legible in logs.
- On retry exhaustion, return an error dict whose message explicitly
names "anti-bot interstitial" as the likely cause, distinguishing
this failure from a genuine no-results case.
Subprocess timeout, spawn failure, and non-zero return-code paths are
unchanged — those are terminal and don't retry.
Tests:
- Verifies HTML-then-JSON returns success on attempt 2.
- Verifies all-HTML returns the diagnostic error dict mentioning the
anti-bot interstitial cause.
- Verifies subprocess timeout is NOT retried.
All 12 bird_x tests pass (9 baseline + 3 new).