The entity-grounding demotion required the full multi-word primary
entity as a contiguous substring, so on-entity items missing a trailing
search descriptor were buried: a 323-pt HN thread "Stripe is friendly
to 'friendly fraud'" scored 0 on a "Stripe payments" query. New
_entity_grounded helper keys on the brand head token; items that never
name the brand still miss it and stay demoted. reddit_keyless
_slot_priority, which had re-implemented the old check while claiming
to mirror rerank's signal, now calls the shared helper so the two
paths cannot diverge.
Comparison tables gain a narrative axis judging whether each entity's
community conversation is about what the entity pitches or about
something else (pricing, rivals, an incident). Backed by a new
mandatory Step 0.55 research item that fetches each entity's CURRENT
first-party positioning (RESOLVED_POSITIONING) instead of relying on
memory, and a narrative-check synthesis beat for single-entity company
runs. The mismatch is the signal: companies usually don't control
their own conversation.
* fix(reddit): relevance-aware comment-enrichment slot selection in keyless path
* docs(changelog): record relevance-aware enrichment fix under Unreleased
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
* chore: remove dev artifacts from installer scan surface
Hermes (and other harnesses that clone raw GitHub instead of honoring
.clawhubignore) scan files that never reach an installed skill, producing
a wall of false-positive security findings. Remove the stale SKILL-original.md
backup, internal docs/plans and docs/test-results, and release-notes.md so
the scanned tree matches what actually ships.
These were already excluded from the ClawHub bundle via .clawhubignore and
from git archives via .gitattributes export-ignore. No runtime files change.
Refs #464
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: drop dangling SKILL-original.md reference in AGENTS.md
Greptile-flagged: the deletion left a 'kept for reference only' pointer to
the removed file. Refs #465
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: remove duplicate command wrapper so plugin exposes only the skill (#461)
The plugin shipped both commands/last30days.md and the skill under the
same name, so /last30 surfaced two `last30days` entries with two
different descriptions. Remove the wrapper; the skill already carries
its own argument-hint, so the /last30days <topic> picker UX is unchanged.
Also corrects the README install note that claimed Claude Code dedupes
the slash command across install methods (it does not), and bumps
3.3.0 -> 3.3.1 across plugin.json, marketplace.json, gemini-extension.json,
and SKILL.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: bump pyproject.toml version to 3.3.1 (manifest contract)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: update uv.lock for 3.3.1 version bump
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(reddit): add live RSS + shreddit comment fixtures
Captured from reddit.com on 2026-05-29 (search.rss listing + the
/svc/shreddit/comments partial), trimmed to a representative subset plus
two synthetic edge cases (deleted author, negative score) for offline
parser tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(http): add keyless get_text helper
Browser-UA text fetch for RSS/HTML endpoints; returns None on any HTTP or
network failure so tiered callers fall through cleanly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(reddit): keyless RSS discovery (search.rss + listing feeds)
Replaces the now-403 search.json with keyless Atom feeds, normalized to the
existing reddit_public post shape. Scores are placeholder zeros, backfilled
during shreddit enrichment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(reddit): keyless shreddit comment scraper
Parses <shreddit-comment> elements from /svc/shreddit/comments/r/{sub}/t3_{id}
(score/author/created/permalink + thingId-anchored body) into top comments,
matching reddit_enrich output. Replaces the dead {thread}.json enrichment.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(reddit): tiered keyless orchestrator
Tier 0 one-shot .json (residential bonus) -> Tier 1 RSS discovery ->
Tier 2 shreddit enrichment. Returns [] never raises, so the SC backup
still engages when every keyless tier is empty.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(reddit): route free path through keyless pipeline (.json is dead)
search_reddit_public is now a thin shim over reddit_keyless, so pipeline.py
and other callers need no change. Removes the dead .json enrichment helpers;
search/_parse_posts remain as the demoted Tier 0 attempt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(reddit): request sort=top so true top comments land on page 1
Guarantees the highest-scored comments are captured even on large threads,
independent of Reddit's default comment sort. Local score re-sort remains.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(reddit): recover post upvote scores via keyless listing partials
The shreddit community-more-posts partial server-renders each post's score
and comment count (works for normal users, not IP-gated), unlike RSS or the
comments endpoint. Use it as a scored discovery source and to backfill scores
onto RSS-discovered posts (subreddits derived from results when not provided).
Ranking now uses real upvote score.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(reddit): listings backfill scores only on bare queries, not discovery
Caught running the full pipeline on a bare topic: deriving subreddits from
noisy RSS results and merging their top/hot listings flooded results with
high-upvote off-topic posts. Now derived-subreddit listings are used only to
backfill scores onto keyword-matched RSS posts; listing cards are merged as
discovery only when the caller explicitly provides subreddits (on-topic).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a pytest-discovered tests/conftest.py for the last30days scripts path and
remove duplicate per-file sys.path.insert boilerplate from tests.
Normalize affected imports to rely on the shared scripts path and remove the
now-unneeded E402 suppressions.
Greptile review (PR #438) flagged two issues:
1. search_github and enrich_with_comments both call _resolve_token,
so when GITHUB_TOKEN is absent from config and env the gh-CLI
subprocess (with its 5s timeout) fires twice per query.
2. The no-token early-return envelope `{"items": [], "error": "no token"}`
was missing the `context` key that every other failure path includes,
making the envelope shape inconsistent between the no-token and
fetch-failure cases.
Fix 1: add public github.resolve_token(token) wrapping the existing
_resolve_token. Pipeline calls it once before search and enrich, so
both downstream calls receive an already-resolved (or already-None)
token and skip the fallback chain.
Fix 2: thread core/from_date/to_date/count through the no-token
envelope's `context` key, matching the fetch-failure envelope shape.
parse_github_response was already tolerant of the missing key, but
diagnostics callers that read response["context"]["..."] now get a
consistent dict in both error paths.
Reviewer's suggested code patch for issue 1 was a no-op (it kept the
same _resolve_token(token) call inside enrich_with_comments); the
underlying intent — resolve at the boundary — is what this commit
implements.
search_github returned a normalized List[dict] directly while every
other adapter follows search_X -> dict envelope, parse_X_response ->
list[dict]. The github branch in pipeline._retrieve_stream was the
only one that called search_* and returned (result, {}) without a
parse step. This blocked fixture-driven testing: there was no parse
function to feed a synthetic envelope to.
Split into three:
search_github(...) -> Dict[str, Any]
HTTP fetch only. Returns {"items": [raw items], "context": {core,
from_date, to_date, count}}.
parse_github_response(response) -> List[Dict[str, Any]]
Pure function. Normalizes, date-filters, sorts by relevance.
enrich_with_comments(items, depth, token) -> List[Dict[str, Any]]
Public extraction of the old private _enrich_top_items. Resolves
the token via env / gh CLI fallback so callers don't have to.
Pipeline now does the standard 3-call dance:
response = github.search_github(...)
items = github.parse_github_response(response)
items = github.enrich_with_comments(items, depth=depth, token=token)
Keeping enrich_with_comments in parse_github_response would make parse
impure and force every fixture-driven test to either mock HTTP or
skip enrichment. Splitting it out matches the YouTube adapter's
pattern.
- Commands block's inline comment on `npx skills add` still said
"symlink this repo into every detected harness's skill dir" — the
exact misconception the PR set out to correct. Rewrite to describe
the frozen-copy behavior and point at the Rules section for the
full explanation.
- Structure section had SKILL.md listed twice (the original line 6
entry plus a new line 13 entry added in this PR). Fold the
SKILL-original.md context into line 6 and drop the duplicate.
AGENTS.md said "edits in the working tree propagate live to every harness"
after `npx skills add . -g -y`, but the install actually drops a real
(frozen-at-install-time) copy at ~/.agents/skills/<name>/ and per-host
symlinks point at *that copy*, not at the working tree. Clarify the
mechanism and offer two ways forward: re-run `npx skills add` to sync,
or replace the install copy with a working-tree symlink for live-edit.
Also fill in two gaps a fresh agent hits on entry:
- `uv run pytest` commands for the ~89-file test suite (no test runner
was documented before)
- Python 3.12+ / `uv` / `.venv/` convention
- Brief doc map: CONFIGURATION.md, SKILL.md vs SKILL-original.md,
CHANGELOG.md / release-notes.md, HERMES_SETUP.md
parse_x_response was returning an empty items list whenever xAI returned
a 200 OK with a malformed payload — empty output text, missing "items"
key, or invalid JSON. The pipeline saw "successful response with zero
items" and quietly handed the user a degraded report with no indication
the API had failed. Now raise http.HTTPError on each of those branches
so _retrieve_stream's caller catches it and surfaces the failure in
errors_by_source, giving the user a visible signal that X didn't work.
Closes#155.
Co-authored-by: Kaustav Mishra <km.git007@gmail.com>
The ScrapeCreators 402 (payment required / credits exhausted) status
was being swallowed by the broad except Exception handlers in
_global_search, _subreddit_search, and fetch_post_comments, returning
[] instead of propagating. That caused users with exhausted credits
to silently get zero Reddit results instead of falling through to
the OpenAI / public Reddit JSON fallback chain in _search_reddit_thread.
Add 402 to the existing 401/403 re-raise list across all three
ScrapeCreators call paths. Closes#170.
Co-authored-by: Jonathan Oppenheim <no-reply@postquantum.space>
Reddit's public JSON endpoint returns 403 to requests carrying the
generic User-Agent and minimal header set urllib defaults to, while
matching curl requests succeed. Switch to a current-Chrome User-Agent
and add Accept-Language / Accept-Encoding / Connection headers so the
fingerprint matches a normal browser. Reddit now serves gzip when
Accept-Encoding includes it, so decompress the body before JSON parse.
Update the user-agent assertion in tests/test_reddit_public.py to match
the new browser-like string. Closes#199.
Co-authored-by: Franco Carballar <francocarballar@gmail.com>
Originally added in #268 to spread load across free-tier accounts when
SCRAPECREATORS_API_KEY is set to a comma-separated list. The 7-line block
was inadvertently dropped during the v3.0.6 consolidation (d14814a) even
though the changelog still advertised the feature. Re-apply the same
random.choice rotation in get_config() so user-facing behavior matches
the documented contract. Closes#287.
Co-authored-by: Eric Oberhofer <eric@oberhofer.io>
_cleanup_children() called os.killpg unconditionally — Windows doesn't
have killpg as an attribute on os, so the call raised AttributeError
(not caught by the existing OSError-family handler) and aborted cleanup.
Guard with hasattr(os, "killpg") and fall back to os.kill(pid, SIGTERM)
on platforms without process-group APIs. Closes#226. Refs #110.
Co-authored-by: gujishh <baiaoshh@163.com>
str.lstrip("r/") treats its argument as a character set, stripping
leading r and / repeatedly. Subreddits starting with 'r' (e.g. r/robotics,
r/ruby) were silently mangled to 'obotics' / 'uby'. Replace with
str.removeprefix("r/") at all four call sites. Python 3.9+ pattern is
safe here — project requires 3.12. Closes#288.
Co-authored-by: Alex Key <alexanderkey0508@gmail.com>
REPO_DIR now derives from the script's location (with env-var override)
and the Claude binary is looked up via PATH (with CLAUDE env-var override)
instead of hardcoded to /Users/mvanhorn/.local/bin/claude. Works on any
checkout. Closes#297.
Co-authored-by: Dave Morin <dave@morin.com>
Rebased onto current main where:
- instagram.py uses unified http.get (not _requests fallback); tests now
mock http.get and assert params/timeout kwargs.
- quality_nudge tests use lib.* import path with sys.path setup.
- README/CONFIGURATION.md memory-dir lines say "defaults to" so they pass
test_no_stray_hardcoded_memory_dir_paths.
Six small additive changes that make the skill correctly understand its
configured sources, plus tests + docs.
User-visible benefits
- LAST30DAYS_STORE=1 in .env turns persistence default-on without
remembering --store on every invocation. Mirrors LAST30DAYS_DEBUG /
LAST30DAYS_SKIP_PREFLIGHT convention.
- SCRAPE_CREATORS_API_KEY (with underscore) accepted as alias for the
canonical name. Matches the spelling used in the vendor's own example
code (Adrian Horning's repo); saves the next user the same diagnostic
rabbit hole.
- Bluesky search now hits api.bsky.app (canonical AppView) instead of
public.api.bsky.app (BunnyCDN-blocked public mirror as of 2026-05-04).
BSKY_SEARCH_HOST env var lets users self-rescue future host migrations
without a code release. Pre-fix: silent 0 Bluesky posts on every run.
- App-password format validator emits a one-shot stderr warning when
BSKY_APP_PASSWORD doesn't match xxxx-xxxx-xxxx-xxxx form. Detect-don't-
gate: createSession still accepts main passwords; the warning helps
users identify a hygiene issue without breaking existing setups.
- Instagram retry on multi-token 500. SC's v2 reels endpoint wraps
Google Search and 500's frequently on multi-word queries; a hashtag-
form retry runs once before bubbling up. Documented vendor instability.
- LAST30DAYS_TRANSCRIPT_TIMEOUT env var (default 30s, was hardcoded 15s).
SC's transcript endpoint regularly takes >15s; the old default was
clipping legitimate responses.
- Silent-failure visibility: new bonus_errored field in the quality
nudge fires when SC is configured but Instagram returned 0 items.
Users see "Bonus source silent: Instagram" instead of unexplained
absence.
- YouTube degraded-ratio false-positive fixed. Captions-disabled videos
can never produce a transcript regardless of yt-dlp version; they're
now subtracted from the denominator so a single uploader-disabled
video doesn't false-trigger the "stale yt-dlp" nudge.
- urllib retry path: status_code attribute typo fix. The Instagram
500-retry was dead code on the urllib branch (getattr(e, 'status', ...)
while http.HTTPError exposes status_code).
Docs
- README.md: added /plugin install last30days step after marketplace add
in three places (the install was previously omitted in the docs).
- CONFIGURATION.md: documented LAST30DAYS_STORE env var, added
BSKY_SEARCH_HOST + app-password format section, mentioned
LAST30DAYS_TRANSCRIPT_TIMEOUT in the Instagram source row.
Test plan
- 43 new unit tests across test_bluesky.py, test_instagram_sc.py,
test_quality_nudge.py, test_youtube_yt.py
- 141 total tests passing in target suite
- Verified end-to-end: /last30days "Toronto resale condo market" with
all 11+ sources active stored 35 new + 5 updated findings, all builder-
PR-style accounts absent (organic agent voice in Instagram + TikTok
results)
Backward compatibility
All changes are strictly additive. Optional kwargs default to None.
New env vars are opt-in. Existing CLI flags untouched. Existing callers
of public functions unaffected.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Personal directories used by the /note slash command (work/) and PDF
print exports (print/) - these are local research artifacts, not
shipping content. Lives on daniel/personal to keep upstream/main and
PR branches free of personal noise.
google/gemini-flash-2.0 is not a valid OpenRouter model ID (segments reversed).
Every rerank and FunJudge call fails with HTTP 400 when REASONING_PROVIDER=openrouter
and LAST30DAYS_RERANK_MODEL is not explicitly pinned, silently falling back to
local-score heuristics.
OpenRouter error body:
{"message":"google/gemini-flash-2.0 is not a valid model ID","code":400}
Switching to google/gemini-3.1-flash-lite-preview, which matches the
GEMINI_FLASH_LITE constant already used by the native Gemini provider on
line 12 of the same file. This makes the Gemini and OpenRouter providers
consistent and avoids a future divergence.
Validated with /last30days 'Claude Opus 4.7' --quick:
- Rerank/FunJudge HTTP 400 errors: 1 per run -> 0
- 'Why: fallback-local-score' markers in output: every cluster -> 0
- LLM-generated 'Why:' reasoning lines: 0 -> 11
The version_consistency test (test_no_stray_hardcoded_memory_dir_paths)
flagged 4 lines where `~/Documents/Last30Days/` appeared without the
canonical "defaults to" phrasing or the ${LAST30DAYS_MEMORY_DIR:-...}
literal form. Tightened the wording in the CONFIGURATION.md table +
footer paragraph, and the README "Where research files are saved"
section, so each path mention is anchored at the env-var override
contract rather than as a bare hardcoded default.
Two findings from Greptile review on PR #302:
1. tests/test_cli_v3.py:302 - The test asserted run_mock.call_args_list[0]
was the main runner's invocation, but fanout.run_competitor_fanout
submits main + competitors to a ThreadPoolExecutor and iterates with
as_completed. With zero-latency mocks, thread scheduling determines
which pipeline.run call lands first, so the competitor's call could
take index [0] and flake CI. Replace [0] indexing with a predicate
match on the canonicalized github_repos kwargs.
2. skills/last30days/scripts/last30days.py:662 - When auto_resolve returns
github_repos, it has already run canonicalize_github_repos(cap=5) and
ranked by relevance. The downstream block then re-canonicalized with
cap=None, which can re-sort by topic-slug match and clobber the
auto_resolve relevance order. Guard the second canonicalization with
a repos_from_auto_resolve flag so it only fires for user-supplied
--github-repo input.
The 6-min architecture walkthrough video is the contributor's own
work — not affiliated with the project. Removing the README pointer
to avoid implying endorsement. CONFIGURATION.md prose stands.
Trevin Chow