fix: Enforce strict 30-day date filtering
Previously Reddit was returning ~60% old content (some from 2022).
This commit adds multiple layers of date enforcement:
- Reddit prompt: Explicit from_date/to_date with "fewer results > older results"
- Hard filter: filter_by_date_range() in normalize.py excludes old content
- WebSearch Date Detective: Extracts dates from URLs (/2026/01/24/) and
snippets ("January 24, 2026", "3 days ago")
- WebSearch scoring: +10 bonus for verified dates, -20 penalty for unknown
The skill now guarantees only content from the last 30 days.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
+17
-5
@@ -15,6 +15,10 @@ WEBSEARCH_WEIGHT_RELEVANCE = 0.55
|
||||
WEBSEARCH_WEIGHT_RECENCY = 0.45
|
||||
WEBSEARCH_SOURCE_PENALTY = 15 # Points deducted for lacking engagement
|
||||
|
||||
# WebSearch date confidence adjustments
|
||||
WEBSEARCH_VERIFIED_BONUS = 10 # Bonus for URL-verified recent date (high confidence)
|
||||
WEBSEARCH_NO_DATE_PENALTY = 20 # Heavy penalty for no date signals (low confidence)
|
||||
|
||||
# Default engagement score for unknown
|
||||
DEFAULT_ENGAGEMENT = 35
|
||||
UNKNOWN_ENGAGEMENT_PENALTY = 10
|
||||
@@ -223,6 +227,11 @@ def score_websearch_items(items: List[schema.WebSearchItem]) -> List[schema.WebS
|
||||
Uses reweighted formula: 55% relevance + 45% recency - 15pt source penalty.
|
||||
This ensures WebSearch items rank below comparable Reddit/X items.
|
||||
|
||||
Date confidence adjustments:
|
||||
- High confidence (URL-verified date): +10 bonus
|
||||
- Med confidence (snippet-extracted date): no change
|
||||
- Low confidence (no date signals): -20 penalty
|
||||
|
||||
Args:
|
||||
items: List of WebSearch items
|
||||
|
||||
@@ -255,11 +264,14 @@ def score_websearch_items(items: List[schema.WebSearchItem]) -> List[schema.WebS
|
||||
# Apply source penalty (WebSearch < Reddit/X for same relevance/recency)
|
||||
overall -= WEBSEARCH_SOURCE_PENALTY
|
||||
|
||||
# Apply penalty for low date confidence
|
||||
if item.date_confidence == "low":
|
||||
overall -= 10
|
||||
elif item.date_confidence == "med":
|
||||
overall -= 5
|
||||
# Apply date confidence adjustments
|
||||
# High confidence (URL-verified): reward with bonus
|
||||
# Med confidence (snippet-extracted): neutral
|
||||
# Low confidence (no date signals): heavy penalty
|
||||
if item.date_confidence == "high":
|
||||
overall += WEBSEARCH_VERIFIED_BONUS # Reward verified recent dates
|
||||
elif item.date_confidence == "low":
|
||||
overall -= WEBSEARCH_NO_DATE_PENALTY # Heavy penalty for unknown
|
||||
|
||||
item.score = max(0, min(100, int(overall)))
|
||||
|
||||
|
||||
Reference in New Issue
Block a user