fix(reddit): use browser-like headers to fix HTTP 403 from urllib
Reddit's public JSON endpoint returns 403 to requests carrying the generic User-Agent and minimal header set urllib defaults to, while matching curl requests succeed. Switch to a current-Chrome User-Agent and add Accept-Language / Accept-Encoding / Connection headers so the fingerprint matches a normal browser. Reddit now serves gzip when Accept-Encoding includes it, so decompress the body before JSON parse. Update the user-agent assertion in tests/test_reddit_public.py to match the new browser-like string. Closes #199. Co-authored-by: Franco Carballar <francocarballar@gmail.com>
This commit is contained in:
@@ -313,7 +313,7 @@ class TestSearchRedditPublicHighLevel:
|
||||
reddit_public.search("test")
|
||||
|
||||
req = mock_urlopen.call_args[0][0]
|
||||
assert req.get_header("User-agent") == "last30days/3.0 (research tool)"
|
||||
assert "Mozilla/5.0" in req.get_header("User-agent")
|
||||
|
||||
|
||||
class TestMissingSubreddit:
|
||||
|
||||
Reference in New Issue
Block a user