refactor(github): split search_github / parse_github_response / enrich_with_comments
search_github returned a normalized List[dict] directly while every
other adapter follows search_X -> dict envelope, parse_X_response ->
list[dict]. The github branch in pipeline._retrieve_stream was the
only one that called search_* and returned (result, {}) without a
parse step. This blocked fixture-driven testing: there was no parse
function to feed a synthetic envelope to.
Split into three:
search_github(...) -> Dict[str, Any]
HTTP fetch only. Returns {"items": [raw items], "context": {core,
from_date, to_date, count}}.
parse_github_response(response) -> List[Dict[str, Any]]
Pure function. Normalizes, date-filters, sorts by relevance.
enrich_with_comments(items, depth, token) -> List[Dict[str, Any]]
Public extraction of the old private _enrich_top_items. Resolves
the token via env / gh CLI fallback so callers don't have to.
Pipeline now does the standard 3-call dance:
response = github.search_github(...)
items = github.parse_github_response(response)
items = github.enrich_with_comments(items, depth=depth, token=token)
Keeping enrich_with_comments in parse_github_response would make parse
impure and force every fixture-driven test to either mock HTTP or
skip enrichment. Splitting it out matches the YouTube adapter's
pattern.
This commit is contained in:
@@ -1007,8 +1007,11 @@ def _retrieve_stream(
|
||||
result = polymarket.search_polymarket(subquery.search_query, from_date, to_date, depth=depth)
|
||||
return polymarket.parse_polymarket_response(result, topic=subquery.search_query), {}
|
||||
if source == "github":
|
||||
result = github.search_github(subquery.search_query, from_date, to_date, depth=depth, token=config.get("GITHUB_TOKEN"))
|
||||
return result, {}
|
||||
token = config.get("GITHUB_TOKEN")
|
||||
response = github.search_github(subquery.search_query, from_date, to_date, depth=depth, token=token)
|
||||
items = github.parse_github_response(response)
|
||||
items = github.enrich_with_comments(items, depth=depth, token=token)
|
||||
return items, {}
|
||||
if source == "pinterest":
|
||||
result = pinterest.search_pinterest(
|
||||
subquery.search_query, from_date, to_date,
|
||||
|
||||
Reference in New Issue
Block a user