feat(tiktok): add TikTok as 7th signal source via Apify
Add TikTok search, scoring, and rendering using the Apify platform (clockworks/tiktok-scraper actor). Users bring their own APIFY_API_TOKEN ($5/month free credits, no CC required). The shared apify_client_wrapper module is designed for reuse by future Facebook/Instagram sources. - New modules: tiktok.py (search + caption extraction), apify_client_wrapper.py - Schema: TikTokItem dataclass, shares field on Engagement, Report.tiktok - Pipeline: normalize → filter → score → sort → dedupe → cross-link → render - Scoring: 0.50*log1p(views) + 0.30*log1p(likes) + 0.20*log1p(comments) - SKILL.md bumped to v2.7 with TikTok stats, citations, and security docs - 26 unit tests covering relevance, normalize, score, dedupe, render, round-trip Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
+13
-1
@@ -45,7 +45,7 @@ def jaccard_similarity(set1: Set[str], set2: Set[str]) -> float:
|
||||
return intersection / union if union > 0 else 0.0
|
||||
|
||||
|
||||
AnyItem = Union[schema.RedditItem, schema.XItem, schema.YouTubeItem,
|
||||
AnyItem = Union[schema.RedditItem, schema.XItem, schema.YouTubeItem, schema.TikTokItem,
|
||||
schema.HackerNewsItem, schema.PolymarketItem, schema.WebSearchItem]
|
||||
|
||||
|
||||
@@ -57,6 +57,8 @@ def get_item_text(item: AnyItem) -> str:
|
||||
return item.title
|
||||
elif isinstance(item, schema.YouTubeItem):
|
||||
return f"{item.title} {item.channel_name}"
|
||||
elif isinstance(item, schema.TikTokItem):
|
||||
return f"{item.text} {item.author_name}"
|
||||
elif isinstance(item, schema.PolymarketItem):
|
||||
return f"{item.title} {item.question}"
|
||||
elif isinstance(item, schema.WebSearchItem):
|
||||
@@ -74,6 +76,8 @@ def _get_cross_source_text(item: AnyItem) -> str:
|
||||
"""
|
||||
if isinstance(item, schema.XItem):
|
||||
return item.text[:100]
|
||||
if isinstance(item, schema.TikTokItem):
|
||||
return item.text[:100]
|
||||
if isinstance(item, schema.HackerNewsItem):
|
||||
title = item.title
|
||||
if title.startswith("Show HN:"):
|
||||
@@ -194,6 +198,14 @@ def dedupe_youtube(
|
||||
return dedupe_items(items, threshold)
|
||||
|
||||
|
||||
def dedupe_tiktok(
|
||||
items: List[schema.TikTokItem],
|
||||
threshold: float = 0.7,
|
||||
) -> List[schema.TikTokItem]:
|
||||
"""Dedupe TikTok items."""
|
||||
return dedupe_items(items, threshold)
|
||||
|
||||
|
||||
def dedupe_hackernews(
|
||||
items: List[schema.HackerNewsItem],
|
||||
threshold: float = 0.7,
|
||||
|
||||
Reference in New Issue
Block a user