fix(reddit): use browser-like headers to fix HTTP 403 from urllib

Reddit's public JSON endpoint returns 403 to requests carrying the
generic User-Agent and minimal header set urllib defaults to, while
matching curl requests succeed. Switch to a current-Chrome User-Agent
and add Accept-Language / Accept-Encoding / Connection headers so the
fingerprint matches a normal browser. Reddit now serves gzip when
Accept-Encoding includes it, so decompress the body before JSON parse.
Update the user-agent assertion in tests/test_reddit_public.py to match
the new browser-like string. Closes #199.

Co-authored-by: Franco Carballar <francocarballar@gmail.com>
This commit is contained in:
Trevin Chow
2026-05-17 01:29:26 -07:00
parent a4f1f94802
commit 4bae05e7fa
2 changed files with 14 additions and 3 deletions
+1 -1
View File
@@ -313,7 +313,7 @@ class TestSearchRedditPublicHighLevel:
reddit_public.search("test")
req = mock_urlopen.call_args[0][0]
assert req.get_header("User-agent") == "last30days/3.0 (research tool)"
assert "Mozilla/5.0" in req.get_header("User-agent")
class TestMissingSubreddit: