- Remove double quotes around $ARGUMENTS so argparse can parse flags
like --deep, --store separately instead of as part of the topic string.
Fixes#61.
- Add ~/.claude/plugins/marketplaces/last30days-skill to the path
discovery loop so marketplace installs can find scripts/.
Fixes#54.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update .claude-plugin/marketplace.json and plugin.json to v2.9.5 with
full metadata. Add plugin install instructions to README as the
recommended install method. The repo root serves as both the marketplace
and the plugin - skills/last30days/SKILL.md (symlink) is discovered
automatically.
Users can now install with:
/plugin marketplace add mvanhorn/last30days-skill
/plugin install last30days@last30days-skill
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add gemini-extension.json manifest with correct array-format settings,
symlink skills/last30days/SKILL.md to root SKILL.md for Gemini skill
discovery, add Gemini install paths to bash for-loop in both main and
open variant, and add Gemini CLI install instructions to README.
Incorporates the good parts of PR #53 (manifest, paths, README) while
avoiding duplicate SKILL.md, tool name scattering, and allowed-tools
pollution that would have created maintenance issues.
Closes#45
Co-Authored-By: Alex Ferrari <alex@thealexferrari.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PR merges on March 7 (PR #48 Xiaohongshu, upstream merge) regressed
SKILL.md by re-introducing the "Save Research to Documents" section
that v2.9.4 removed. Those branches were forked before v2.9.4 and
brought the old content back via merge resolution.
Fixes: remove save section, restore --save-dir flag on bash command,
update agent mode line, add tool-call guard to STOP instruction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Explored adding scheduled re-runs and cumulative intelligence to
last30days. Concluded that Claude Code's session-scoped scheduling
(CronCreate/loop) can't support true perpetual monitoring since
jobs die when the terminal closes. Plan documents the architecture,
what exists, and why we're waiting for persistent background agents.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Xiaohongshu search via local MCP service (opt-in, zero impact if service not running)
- Reddit public JSON fallback (works with zero API keys)
- Reddit priority: ScrapeCreators -> OpenAI -> public fallback
- Updated env.py: Reddit always available via public fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Resolve conflicts between ScrapeCreators Reddit (main) and
public Reddit fallback (PR #48). Priority: ScrapeCreators ->
OpenAI -> public Reddit fallback.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove prompt-injection false positive ("you are now" → "treat yourself as")
- Declare AUTH_TOKEN and CT0 in frontmatter optionalEnv
- Clarify X token access language (no browser session access)
- Add permissions overview block near top of file
Zero functionality changes — metadata and prose only.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add --save-dir flag to last30days.py that saves raw research output
during the existing script run. Remove entire "Save Research to
Documents" section from SKILL.md (~45 lines). No more extra tool
calls, no (No output), no multi-minute cogitation after invitation.
Tested: --mock confirms file creation and duplicate date suffixing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CRITICAL: run_in_background callbacks caused model to re-engage after
save, hallucinate fake "Human:" messages, and generate unsolicited
multi-paragraph responses. Switch to foreground cat > heredoc which
executes sub-second with no callback.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Background Bash heredoc instead of Write tool
- Suppress response text on save completion
- 📎 footer line replaces verbose confirmation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Resolves merge conflict, keeping upstream's approach:
- Background heredoc save instead of Write tool
- Adds 📎 footer line
- No more "Wrote N lines..." cluttering output
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Write tool displays "Wrote N lines..." after the invitation,
ruining the end-of-run experience. Now saves via background Bash
with a subtle 📎 footer line in the invitation text.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Write tool displays "Wrote N lines..." after the invitation,
ruining the end-of-run experience. Now saves via background Bash
with a subtle 📎 footer line in the invitation text.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sync from public repo. Every run now saves the complete briefing as a
topic-named .md file to ~/Documents/Last30Days/. Credit @devin_explores.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bump version to 2.9.1, update changelog and release notes.
Credit @devin_explores for inspiring the feature.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Every run now automatically saves the complete briefing (synthesis,
stats, follow-up suggestions) as a topic-named .md file in the user's
Documents folder. Agent mode also saves. No Python script changes -
this is purely a SKILL.md instruction addition.
Inspired by @devin_explores manually saving results to build a
personal research library.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three focused improvements based on 5 full-pipeline beta tests:
1. Elevate top Reddit comments in scoring and rendering
2. Improve subreddit discovery heuristic for ambiguous queries
3. Make ScrapeCreators the default recommended Reddit method
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- New scripts/lib/reddit.py: multi-query expansion, global search,
subreddit discovery, targeted subreddit search, comment enrichment
- 68 results in 17s vs ~15 results in 60-90s (OpenAI)
- Cost: ~$0.02/search vs $0.03-0.10 (15-50x cheaper)
- Real engagement data (score, comments, dates) from API
- No more 429 rate limits on comment enrichment
- Falls back to OpenAI if SCRAPECREATORS_API_KEY missing
- Registered as last30daysbeta for parallel local testing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- add xiaohongshu/xhs source path via xiaohongshu-mcp HTTP API\n- add Reddit public JSON fallback when OpenAI auth is unavailable\n- update diagnostics/UI rendering for new source availability states\n- harden Xiaohongshu availability probe to reduce false negatives\n- include source status reporting for Xiaohongshu
Add Instagram Reels as the 8th research source via ScrapeCreators API.
One API key (SCRAPECREATORS_API_KEY) now covers both TikTok and Instagram.
- Add scripts/lib/instagram.py: keyword search, transcript extraction,
relevance scoring, engagement metrics (views, likes, comments)
- Add InstagramItem to schema, normalization, scoring, dedup, rendering
- Add Instagram to orchestrator pipeline, watchlist, and UI spinners
- Update SKILL.md: stats template, citation priority, item format,
URL-to-name extraction rules, anti-Sources instruction
- Update README and CHANGELOG for v2.8
- Fix: Instagram/TikTok not running in --search= web-only path
- Fix: web stats line showing full URLs instead of domain names
- Replace APIFY_API_TOKEN with SCRAPECREATORS_API_KEY throughout
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all Apify references with ScrapeCreators. Key points:
- No subscription required (was $5/mo with Apify)
- 100 free credits, pay-as-you-go after
- SCRAPECREATORS_API_KEY replaces APIFY_API_TOKEN
- Backwards compatible: APIFY_API_TOKEN still works as fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root cause of empty TikTok results: Apify required monthly subscription.
ScrapeCreators is PAYG with 100 free credits and no subscription.
Key fix: ScrapeCreators nests items under aweme_info wrapper
(search_item_list[].aweme_info.{fields}), which the previous
implementation missed, causing all fields to be empty.
Changes:
- Rewrite tiktok.py to use ScrapeCreators REST API
- Add aweme_info unwrapping for correct field extraction
- Add transcript fetching via /video/transcript endpoint
- Add SCRAPECREATORS_API_KEY to env.py config
- Update last30days.py to use env.get_tiktok_token()
- Delete apify_client_wrapper.py (no longer needed)
- Update tests for new date field format (create_time)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove double quotes around $ARGUMENTS in SKILL.md so bash word-splits
the expansion, and change argparse topic from nargs="?" to nargs="*"
so multi-word topics still work. Also document --store, --include-web,
--diagnose, and --timeout flags in the Options section.
Closes#36
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add TikTok as 7th source throughout README: new V2.7 banner, real
search examples (Iran Israel: 61.6M views, Leah Halton: 152.6M views),
APIFY_API_TOKEN in installation, Apify in security table, fix stale
"six sources" references to "seven sources".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Only print debug log lines when running in an interactive terminal.
In Claude Code (non-TTY), the spinner system handles progress display,
so these raw log lines just add noise.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pass logger=None to Apify .call() to prevent the SDK from streaming
raw actor run logs (status messages, crawler stats, warnings) that
drown out the clean spinner UI in Claude Code.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove quiet=True from YouTube and TikTok spinners so they display
the same colored emoji progress lines as Reddit and X in non-TTY mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When running in Claude Code, the assistant has a built-in WebSearch tool
that's free and higher quality than Parallel AI/Brave/OpenRouter. Adding
--no-native-web to the SKILL.md invocation defers web search to the
assistant, saving API credits. OpenClaw invocations don't pass this flag,
so they continue using native web backends.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The websearch module import was dropped when the tiktok import was added,
causing the script to crash during the rendering phase after all data
was successfully collected.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add TikTok search, scoring, and rendering using the Apify platform
(clockworks/tiktok-scraper actor). Users bring their own APIFY_API_TOKEN
($5/month free credits, no CC required). The shared apify_client_wrapper
module is designed for reuse by future Facebook/Instagram sources.
- New modules: tiktok.py (search + caption extraction), apify_client_wrapper.py
- Schema: TikTokItem dataclass, shares field on Engagement, Report.tiktok
- Pipeline: normalize → filter → score → sort → dedupe → cross-link → render
- Scoring: 0.50*log1p(views) + 0.30*log1p(likes) + 0.20*log1p(comments)
- SKILL.md bumped to v2.7 with TikTok stats, citations, and security docs
- 26 unit tests covering relevance, normalize, score, dedupe, render, round-trip
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Inspired by PR #26 (wkbaran), whose early work on HN/YouTube sources helped
shape what we built in v2.5. Cherry-picks the source-filtering concept as a
clean implementation against our existing architecture.
--search=SOURCES accepts comma-separated: reddit, x, hn, youtube, polymarket, web
Example: --search reddit,hn (run only Reddit + Hacker News)
Also:
- bird_x: add noise words (trending, viral, plugin, skills) + last-chance retry
- render: show xAI tip for reddit-only mode regardless of missing_keys value
- tests: new test_bird_x.py (5 tests)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Cherry-picked from PR #24 (el-analista). Adds trending/viral/plugin/skill/tool
noise words to _extract_core_subject, and a last-chance retry that falls back
to the longest non-noise token when 2-word retry also returns 0 results.
cache.py and render.py env overrides were already on main.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
URLs in markdown links wrap badly in terminals (discovered after first
fix attempt). Change to plain names like "Newsweek, Sportskeeda, Medium"
on the Web: stats line. Update citation note to explain the reason.
Tested on Dor Brothers, Kanye West, Logan Paul - no trailing Sources:
block appeared in any of the three test runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The WebSearch tool has a system-level mandate to append a Sources:
section at the end of every response. SKILL.md's old "DO NOT output
Sources: list" instruction was too weak to override it.
Fix: redirect citations into the stats block's Web: line as inline
links. The WebSearch citation requirement is satisfied there; an
explicit note after the stats block tells the model not to append
a separate trailing section.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- .gitignore: protect docs/comparison-results/ and other private
benchmark artifacts from accidental upstream push
- docs/plans: add plan for PR #37 Codex auth finalization
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test was picking up the real OPENAI_API_KEY from the shell
environment, causing it to fail on any machine with that key set.
Added @patch.dict(os.environ, {}, clear=True) so the test runs in
a clean env and exercises the file_env path as intended.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Matt Van Horn