refactor: drop requests dep, route all providers through lib/http urllib wrapper (#393)

Five provider modules (pinterest, threads, instagram, tiktok, youtube_yt)
and watchlist.py each carried a try/except `requests` import with parallel
urllib + requests branches. The urllib path already used the
stdlib-only wrapper at `lib/http.py` (retries, 429 handling, HTTPError).
This collapses every dual-branch into a single `http.get`/`http.post`
call and removes the `requests` dependency from `pyproject.toml`.

Also drops 4 transitive deps (urllib3, certifi, charset-normalizer, idna)
from the lockfile, leaving the skill stdlib-only at runtime.

Tests for tiktok comments and watchlist delivery were rewritten to mock
`lib.http` directly instead of the now-removed `requests` module.

Out of scope but flagged during review: the 13 surviving SC call sites
share a near-identical scaffold and would benefit from a
`http.scrapecreators_get(url, params, token, ...)` helper. Filed for a
follow-up PR rather than expanding scope here.

This commit is contained in:

Trevin Chow

2026-05-15 08:07:43 -07:00

committed by

GitHub

parent c845f483d6

commit 80a1a47eef

10 changed files with 218 additions and 596 deletions

									
										pyproject.toml
									
		+1
		-3
	
												View File
												
				@@ -4,9 +4,7 @@ version = "3.2.1"

				description = "Multi-source last-30-days research skill"

				readme = "README.md"

				requires-python = ">=3.12"

				dependencies = [

				  "requests>=2.32,<3",

				]

				dependencies = []

				[dependency-groups]

				dev = [