parse_x_response was returning an empty items list whenever xAI returned
a 200 OK with a malformed payload — empty output text, missing "items"
key, or invalid JSON. The pipeline saw "successful response with zero
items" and quietly handed the user a degraded report with no indication
the API had failed. Now raise http.HTTPError on each of those branches
so _retrieve_stream's caller catches it and surfaces the failure in
errors_by_source, giving the user a visible signal that X didn't work.
Closes#155.
Co-authored-by: Kaustav Mishra <km.git007@gmail.com>
The ScrapeCreators 402 (payment required / credits exhausted) status
was being swallowed by the broad except Exception handlers in
_global_search, _subreddit_search, and fetch_post_comments, returning
[] instead of propagating. That caused users with exhausted credits
to silently get zero Reddit results instead of falling through to
the OpenAI / public Reddit JSON fallback chain in _search_reddit_thread.
Add 402 to the existing 401/403 re-raise list across all three
ScrapeCreators call paths. Closes#170.
Co-authored-by: Jonathan Oppenheim <no-reply@postquantum.space>
Reddit's public JSON endpoint returns 403 to requests carrying the
generic User-Agent and minimal header set urllib defaults to, while
matching curl requests succeed. Switch to a current-Chrome User-Agent
and add Accept-Language / Accept-Encoding / Connection headers so the
fingerprint matches a normal browser. Reddit now serves gzip when
Accept-Encoding includes it, so decompress the body before JSON parse.
Update the user-agent assertion in tests/test_reddit_public.py to match
the new browser-like string. Closes#199.
Co-authored-by: Franco Carballar <francocarballar@gmail.com>
Originally added in #268 to spread load across free-tier accounts when
SCRAPECREATORS_API_KEY is set to a comma-separated list. The 7-line block
was inadvertently dropped during the v3.0.6 consolidation (d14814a) even
though the changelog still advertised the feature. Re-apply the same
random.choice rotation in get_config() so user-facing behavior matches
the documented contract. Closes#287.
Co-authored-by: Eric Oberhofer <eric@oberhofer.io>
_cleanup_children() called os.killpg unconditionally — Windows doesn't
have killpg as an attribute on os, so the call raised AttributeError
(not caught by the existing OSError-family handler) and aborted cleanup.
Guard with hasattr(os, "killpg") and fall back to os.kill(pid, SIGTERM)
on platforms without process-group APIs. Closes#226. Refs #110.
Co-authored-by: gujishh <baiaoshh@163.com>
str.lstrip("r/") treats its argument as a character set, stripping
leading r and / repeatedly. Subreddits starting with 'r' (e.g. r/robotics,
r/ruby) were silently mangled to 'obotics' / 'uby'. Replace with
str.removeprefix("r/") at all four call sites. Python 3.9+ pattern is
safe here — project requires 3.12. Closes#288.
Co-authored-by: Alex Key <alexanderkey0508@gmail.com>
REPO_DIR now derives from the script's location (with env-var override)
and the Claude binary is looked up via PATH (with CLAUDE env-var override)
instead of hardcoded to /Users/mvanhorn/.local/bin/claude. Works on any
checkout. Closes#297.
Co-authored-by: Dave Morin <dave@morin.com>
Rebased onto current main where:
- instagram.py uses unified http.get (not _requests fallback); tests now
mock http.get and assert params/timeout kwargs.
- quality_nudge tests use lib.* import path with sys.path setup.
- README/CONFIGURATION.md memory-dir lines say "defaults to" so they pass
test_no_stray_hardcoded_memory_dir_paths.
Six small additive changes that make the skill correctly understand its
configured sources, plus tests + docs.
User-visible benefits
- LAST30DAYS_STORE=1 in .env turns persistence default-on without
remembering --store on every invocation. Mirrors LAST30DAYS_DEBUG /
LAST30DAYS_SKIP_PREFLIGHT convention.
- SCRAPE_CREATORS_API_KEY (with underscore) accepted as alias for the
canonical name. Matches the spelling used in the vendor's own example
code (Adrian Horning's repo); saves the next user the same diagnostic
rabbit hole.
- Bluesky search now hits api.bsky.app (canonical AppView) instead of
public.api.bsky.app (BunnyCDN-blocked public mirror as of 2026-05-04).
BSKY_SEARCH_HOST env var lets users self-rescue future host migrations
without a code release. Pre-fix: silent 0 Bluesky posts on every run.
- App-password format validator emits a one-shot stderr warning when
BSKY_APP_PASSWORD doesn't match xxxx-xxxx-xxxx-xxxx form. Detect-don't-
gate: createSession still accepts main passwords; the warning helps
users identify a hygiene issue without breaking existing setups.
- Instagram retry on multi-token 500. SC's v2 reels endpoint wraps
Google Search and 500's frequently on multi-word queries; a hashtag-
form retry runs once before bubbling up. Documented vendor instability.
- LAST30DAYS_TRANSCRIPT_TIMEOUT env var (default 30s, was hardcoded 15s).
SC's transcript endpoint regularly takes >15s; the old default was
clipping legitimate responses.
- Silent-failure visibility: new bonus_errored field in the quality
nudge fires when SC is configured but Instagram returned 0 items.
Users see "Bonus source silent: Instagram" instead of unexplained
absence.
- YouTube degraded-ratio false-positive fixed. Captions-disabled videos
can never produce a transcript regardless of yt-dlp version; they're
now subtracted from the denominator so a single uploader-disabled
video doesn't false-trigger the "stale yt-dlp" nudge.
- urllib retry path: status_code attribute typo fix. The Instagram
500-retry was dead code on the urllib branch (getattr(e, 'status', ...)
while http.HTTPError exposes status_code).
Docs
- README.md: added /plugin install last30days step after marketplace add
in three places (the install was previously omitted in the docs).
- CONFIGURATION.md: documented LAST30DAYS_STORE env var, added
BSKY_SEARCH_HOST + app-password format section, mentioned
LAST30DAYS_TRANSCRIPT_TIMEOUT in the Instagram source row.
Test plan
- 43 new unit tests across test_bluesky.py, test_instagram_sc.py,
test_quality_nudge.py, test_youtube_yt.py
- 141 total tests passing in target suite
- Verified end-to-end: /last30days "Toronto resale condo market" with
all 11+ sources active stored 35 new + 5 updated findings, all builder-
PR-style accounts absent (organic agent voice in Instagram + TikTok
results)
Backward compatibility
All changes are strictly additive. Optional kwargs default to None.
New env vars are opt-in. Existing CLI flags untouched. Existing callers
of public functions unaffected.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Personal directories used by the /note slash command (work/) and PDF
print exports (print/) - these are local research artifacts, not
shipping content. Lives on daniel/personal to keep upstream/main and
PR branches free of personal noise.
google/gemini-flash-2.0 is not a valid OpenRouter model ID (segments reversed).
Every rerank and FunJudge call fails with HTTP 400 when REASONING_PROVIDER=openrouter
and LAST30DAYS_RERANK_MODEL is not explicitly pinned, silently falling back to
local-score heuristics.
OpenRouter error body:
{"message":"google/gemini-flash-2.0 is not a valid model ID","code":400}
Switching to google/gemini-3.1-flash-lite-preview, which matches the
GEMINI_FLASH_LITE constant already used by the native Gemini provider on
line 12 of the same file. This makes the Gemini and OpenRouter providers
consistent and avoids a future divergence.
Validated with /last30days 'Claude Opus 4.7' --quick:
- Rerank/FunJudge HTTP 400 errors: 1 per run -> 0
- 'Why: fallback-local-score' markers in output: every cluster -> 0
- LLM-generated 'Why:' reasoning lines: 0 -> 11
The version_consistency test (test_no_stray_hardcoded_memory_dir_paths)
flagged 4 lines where `~/Documents/Last30Days/` appeared without the
canonical "defaults to" phrasing or the ${LAST30DAYS_MEMORY_DIR:-...}
literal form. Tightened the wording in the CONFIGURATION.md table +
footer paragraph, and the README "Where research files are saved"
section, so each path mention is anchored at the env-var override
contract rather than as a bare hardcoded default.
Two findings from Greptile review on PR #302:
1. tests/test_cli_v3.py:302 - The test asserted run_mock.call_args_list[0]
was the main runner's invocation, but fanout.run_competitor_fanout
submits main + competitors to a ThreadPoolExecutor and iterates with
as_completed. With zero-latency mocks, thread scheduling determines
which pipeline.run call lands first, so the competitor's call could
take index [0] and flake CI. Replace [0] indexing with a predicate
match on the canonicalized github_repos kwargs.
2. skills/last30days/scripts/last30days.py:662 - When auto_resolve returns
github_repos, it has already run canonicalize_github_repos(cap=5) and
ranked by relevance. The downstream block then re-canonicalized with
cap=None, which can re-sort by topic-slug match and clobber the
auto_resolve relevance order. Guard the second canonicalization with
a repos_from_auto_resolve flag so it only fires for user-supplied
--github-repo input.
The 6-min architecture walkthrough video is the contributor's own
work — not affiliated with the project. Removing the README pointer
to avoid implying endorsement. CONFIGURATION.md prose stands.
Adds CONFIGURATION.md at repo root - a focused configuration reference
covering save paths, the per-source API-key matrix, reasoning and
web-search backend priority, the trend-monitoring stack (--store +
watchlist.py + briefing.py), and per-client patterns.
Surfaces two things that ship in the engine but were not documented
for users:
- The project-scoped .claude/last30days.env config file (currently only
referenced in hooks/scripts/check-config.sh) which takes precedence
over the global ~/.config/last30days/.env when present. Cleanest
pattern for per-client setups - drop a file in the client folder, cd
in, run normally.
- The existing trend-monitoring scripts (--store flag, watchlist.py,
briefing.py) that the README did not surface for users.
Updates README with a brief "Configuration" section pointing to the new
file, plus a one-line "Going deeper" callout linking a 6-min community
architecture walkthrough on YouTube.
All CLI surface claims (watchlist subcommands, briefing modes, source
dedupe key, env file priority chain) fact-checked against the live
scripts/ source before commit.
Brave uses identical v10 AES-128-CBC encryption to Chrome; only the
DB path (BraveSoftware/Brave-Browser) and Keychain service name
("Brave Safe Storage") differ. Refactored chrome_cookies.py to share
a single _extract_chromium_cookies_macos helper rather than duplicating
the decryption logic.
Profile discovery tries Default/ first, then scans numbered Profile N/
directories so non-default Brave profiles are covered.
Kaustav Mishra