fix(github): reject garbage in _parse_date; consolidate date parsing

github.py _parse_date used naive string slicing (return iso_str[:10])
which accepted any 10+ character string as a "date." For input
"hello world" it returned "hello worl". Now delegates to
dates.parse_date() which validates the format and returns None for
non-dates.

Also migrated reddit.py and threads.py _parse_date to the shared
dates.parse_date(). Both previously reimplemented ISO-with-trailing-
offset handling (the .replace("Z", "+00:00") dance) and reddit.py
also had its own Unix timestamp branch. dates.parse_date() already
handles all of this, including the +0000 no-colon variant Reddit emits.

Preserved reddit.py's original falsy-check so 0 still returns None
(epoch 0 would otherwise parse as "1970-01-01", breaking an existing
test and changing long-standing behavior).

Added 4 new github tests for garbage rejection and offset variants.
All 1026 existing tests pass (15 pre-existing failures unchanged).
This commit is contained in:
Ilia Alshanetsky
2026-04-10 07:39:07 -04:00
parent 86e1d77ad7
commit 65fcf6be65
4 changed files with 43 additions and 51 deletions
+8 -20
View File
@@ -12,7 +12,6 @@ import sys
import time
from collections import Counter
from concurrent.futures import ThreadPoolExecutor, as_completed, wait as futures_wait
from datetime import datetime, timezone
from typing import Any, Dict, List, Optional, Set
try:
@@ -28,7 +27,7 @@ def _first_of(*values, default=None):
return v
return default
from . import http, log
from . import dates, http, log
SCRAPECREATORS_BASE = "https://api.scrapecreators.com/v1/reddit"
@@ -212,27 +211,16 @@ def _parse_date(value) -> Optional[str]:
Global search returns ``created_at`` as an ISO string
(e.g. "2018-05-03T01:09:17.620000+0000"); subreddit search returns
``created_utc`` as a Unix timestamp. Handle both.
``created_utc`` as a Unix timestamp. dates.parse_date() handles both,
plus edge cases like Z suffix and +0000 (no colon) offset.
Falsy inputs (None, "", 0) return None, matching the original behavior
where a Unix timestamp of 0 meant "no date" rather than epoch 0.
"""
if not value:
return None
# ISO-8601 string (contains 'T' or '-')
if isinstance(value, str) and ("T" in value or "-" in value):
try:
# Strip trailing offset variations (+0000, Z) for fromisoformat
clean = value.replace("Z", "+00:00")
if clean.endswith("+0000"):
clean = clean[:-5] + "+00:00"
dt = datetime.fromisoformat(clean)
return dt.strftime("%Y-%m-%d")
except (ValueError, TypeError):
pass
# Unix timestamp (int or float or numeric string)
try:
dt = datetime.fromtimestamp(float(value), tz=timezone.utc)
return dt.strftime("%Y-%m-%d")
except (ValueError, TypeError, OSError):
return None
dt = dates.parse_date(str(value))
return dt.strftime("%Y-%m-%d") if dt else None
def _extract_subreddit_name(value: Any) -> str: