Add --save-dir flag to last30days.py that saves raw research output
during the existing script run. Remove entire "Save Research to
Documents" section from SKILL.md (~45 lines). No more extra tool
calls, no (No output), no multi-minute cogitation after invitation.
Tested: --mock confirms file creation and duplicate date suffixing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CRITICAL: run_in_background callbacks caused model to re-engage after
save, hallucinate fake "Human:" messages, and generate unsolicited
multi-paragraph responses. Switch to foreground cat > heredoc which
executes sub-second with no callback.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Background Bash heredoc instead of Write tool
- Suppress response text on save completion
- 📎 footer line replaces verbose confirmation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Resolves merge conflict, keeping upstream's approach:
- Background heredoc save instead of Write tool
- Adds 📎 footer line
- No more "Wrote N lines..." cluttering output
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Write tool displays "Wrote N lines..." after the invitation,
ruining the end-of-run experience. Now saves via background Bash
with a subtle 📎 footer line in the invitation text.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sync from public repo. Every run now saves the complete briefing as a
topic-named .md file to ~/Documents/Last30Days/. Credit @devin_explores.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bump version to 2.9.1, update changelog and release notes.
Credit @devin_explores for inspiring the feature.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Every run now automatically saves the complete briefing (synthesis,
stats, follow-up suggestions) as a topic-named .md file in the user's
Documents folder. Agent mode also saves. No Python script changes -
this is purely a SKILL.md instruction addition.
Inspired by @devin_explores manually saving results to build a
personal research library.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three focused improvements based on 5 full-pipeline beta tests:
1. Elevate top Reddit comments in scoring and rendering
2. Improve subreddit discovery heuristic for ambiguous queries
3. Make ScrapeCreators the default recommended Reddit method
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- New scripts/lib/reddit.py: multi-query expansion, global search,
subreddit discovery, targeted subreddit search, comment enrichment
- 68 results in 17s vs ~15 results in 60-90s (OpenAI)
- Cost: ~$0.02/search vs $0.03-0.10 (15-50x cheaper)
- Real engagement data (score, comments, dates) from API
- No more 429 rate limits on comment enrichment
- Falls back to OpenAI if SCRAPECREATORS_API_KEY missing
- Registered as last30daysbeta for parallel local testing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Instagram Reels as the 8th research source via ScrapeCreators API.
One API key (SCRAPECREATORS_API_KEY) now covers both TikTok and Instagram.
- Add scripts/lib/instagram.py: keyword search, transcript extraction,
relevance scoring, engagement metrics (views, likes, comments)
- Add InstagramItem to schema, normalization, scoring, dedup, rendering
- Add Instagram to orchestrator pipeline, watchlist, and UI spinners
- Update SKILL.md: stats template, citation priority, item format,
URL-to-name extraction rules, anti-Sources instruction
- Update README and CHANGELOG for v2.8
- Fix: Instagram/TikTok not running in --search= web-only path
- Fix: web stats line showing full URLs instead of domain names
- Replace APIFY_API_TOKEN with SCRAPECREATORS_API_KEY throughout
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all Apify references with ScrapeCreators. Key points:
- No subscription required (was $5/mo with Apify)
- 100 free credits, pay-as-you-go after
- SCRAPECREATORS_API_KEY replaces APIFY_API_TOKEN
- Backwards compatible: APIFY_API_TOKEN still works as fallback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root cause of empty TikTok results: Apify required monthly subscription.
ScrapeCreators is PAYG with 100 free credits and no subscription.
Key fix: ScrapeCreators nests items under aweme_info wrapper
(search_item_list[].aweme_info.{fields}), which the previous
implementation missed, causing all fields to be empty.
Changes:
- Rewrite tiktok.py to use ScrapeCreators REST API
- Add aweme_info unwrapping for correct field extraction
- Add transcript fetching via /video/transcript endpoint
- Add SCRAPECREATORS_API_KEY to env.py config
- Update last30days.py to use env.get_tiktok_token()
- Delete apify_client_wrapper.py (no longer needed)
- Update tests for new date field format (create_time)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove double quotes around $ARGUMENTS in SKILL.md so bash word-splits
the expansion, and change argparse topic from nargs="?" to nargs="*"
so multi-word topics still work. Also document --store, --include-web,
--diagnose, and --timeout flags in the Options section.
Closes#36
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add TikTok as 7th source throughout README: new V2.7 banner, real
search examples (Iran Israel: 61.6M views, Leah Halton: 152.6M views),
APIFY_API_TOKEN in installation, Apify in security table, fix stale
"six sources" references to "seven sources".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Only print debug log lines when running in an interactive terminal.
In Claude Code (non-TTY), the spinner system handles progress display,
so these raw log lines just add noise.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pass logger=None to Apify .call() to prevent the SDK from streaming
raw actor run logs (status messages, crawler stats, warnings) that
drown out the clean spinner UI in Claude Code.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove quiet=True from YouTube and TikTok spinners so they display
the same colored emoji progress lines as Reddit and X in non-TTY mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When running in Claude Code, the assistant has a built-in WebSearch tool
that's free and higher quality than Parallel AI/Brave/OpenRouter. Adding
--no-native-web to the SKILL.md invocation defers web search to the
assistant, saving API credits. OpenClaw invocations don't pass this flag,
so they continue using native web backends.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The websearch module import was dropped when the tiktok import was added,
causing the script to crash during the rendering phase after all data
was successfully collected.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add TikTok search, scoring, and rendering using the Apify platform
(clockworks/tiktok-scraper actor). Users bring their own APIFY_API_TOKEN
($5/month free credits, no CC required). The shared apify_client_wrapper
module is designed for reuse by future Facebook/Instagram sources.
- New modules: tiktok.py (search + caption extraction), apify_client_wrapper.py
- Schema: TikTokItem dataclass, shares field on Engagement, Report.tiktok
- Pipeline: normalize → filter → score → sort → dedupe → cross-link → render
- Scoring: 0.50*log1p(views) + 0.30*log1p(likes) + 0.20*log1p(comments)
- SKILL.md bumped to v2.7 with TikTok stats, citations, and security docs
- 26 unit tests covering relevance, normalize, score, dedupe, render, round-trip
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Inspired by PR #26 (wkbaran), whose early work on HN/YouTube sources helped
shape what we built in v2.5. Cherry-picks the source-filtering concept as a
clean implementation against our existing architecture.
--search=SOURCES accepts comma-separated: reddit, x, hn, youtube, polymarket, web
Example: --search reddit,hn (run only Reddit + Hacker News)
Also:
- bird_x: add noise words (trending, viral, plugin, skills) + last-chance retry
- render: show xAI tip for reddit-only mode regardless of missing_keys value
- tests: new test_bird_x.py (5 tests)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Cherry-picked from PR #24 (el-analista). Adds trending/viral/plugin/skill/tool
noise words to _extract_core_subject, and a last-chance retry that falls back
to the longest non-noise token when 2-word retry also returns 0 results.
cache.py and render.py env overrides were already on main.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
URLs in markdown links wrap badly in terminals (discovered after first
fix attempt). Change to plain names like "Newsweek, Sportskeeda, Medium"
on the Web: stats line. Update citation note to explain the reason.
Tested on Dor Brothers, Kanye West, Logan Paul - no trailing Sources:
block appeared in any of the three test runs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The WebSearch tool has a system-level mandate to append a Sources:
section at the end of every response. SKILL.md's old "DO NOT output
Sources: list" instruction was too weak to override it.
Fix: redirect citations into the stats block's Web: line as inline
links. The WebSearch citation requirement is satisfied there; an
explicit note after the stats block tells the model not to append
a separate trailing section.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- .gitignore: protect docs/comparison-results/ and other private
benchmark artifacts from accidental upstream push
- docs/plans: add plan for PR #37 Codex auth finalization
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The test was picking up the real OPENAI_API_KEY from the shell
environment, causing it to fail on any machine with that key set.
Added @patch.dict(os.environ, {}, clear=True) so the test runs in
a clean env and exercises the file_env path as intended.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: fix YAML error in argument-hint
* add codex auth support to responses API
* Use gpt-5.1-codex-mini as default model for Codex auth
Add CODEX_FALLBACK_MODELS chain (gpt-5.1-codex-mini → gpt-5.2) for
Codex endpoint which doesn't support standard OpenAI models like
gpt-4o-mini. Adds model fallback retry on 400 errors in the Codex
search path. Also adds test_codex_auth.py with 22 unit tests covering
JWT decoding, auth resolution, SSE parsing, and payload building.
* Pass .env credentials to Bird Node subprocesses for X auth
On platforms without browser cookie access (e.g. WSL2), Bird's
vendored Node.js module cannot read AUTH_TOKEN/CT0 from Firefox
or Chrome cookie stores. The .env config file already supports
these values, but they were only loaded into the Python config
dict — never exported to the environment of Node subprocesses.
- Add AUTH_TOKEN/CT0 to env.py config key loading
- Add set_credentials()/\_subprocess_env() to bird_x.py to inject
credentials into the env dict passed to subprocess.run/Popen
- Call set_credentials() in main() before Bird auth detection
---------
Co-authored-by: Justin Williams <jblwilliams@gmail.com>
Removes disable-model-invocation restriction so the skill can be called
by other agents via the Skill tool. Adds --agent flag for non-interactive
report output (skips intro, AskUserQuestion, wait pause, and invitation).
Fixes false security doc claiming autonomous invocation was blocked.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reorder v2.5 features: Polymarket prediction markets and HN lead as #1,
multi-signal quality-ranked relevance scoring as #2. Add Anthropic Odds
example showcasing 11 live markets from a two-word query. Add Anthropic
and OpenAI Polymarket transcripts to launch tweets.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Gamma API only searches event titles/slugs, missing markets where the
topic is an outcome (e.g., "Arizona" in NCAA Tournament Winner). This adds:
- All-word query expansion (not just first word): "Arizona Basketball" now
searches "Arizona", "Basketball" independently
- Tag-based domain expansion: extracts category tags (e.g., "NCAA") from
first-pass results and searches those as a second pass
- Neg-risk binary market synthesis: shows team names from market questions
instead of generic Yes/No outcomes
- Question shortening: extracts "Arizona" from "Will Arizona win the NCAA
Tournament?" for clean display
- Increased depth (3 pages) and result caps (15) for more coverage
Live results: "Arizona Basketball" now finds NCAA Tournament Winner (12%),
#1 Seed (88%), Big 12 Champion (69%). "Iran War" returns 15 markets (up
from 9) with no regression.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add CROSS to sync targets with sed-patched name/version/description
- Switch cp to rsync to handle identical file edge case on APFS
- CROSS SKILL.md gets last30daysCROSS skill root injected into search path
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- _compute_text_similarity() now checks outcome names with bidirectional
substring matching (0.85) and token overlap (0.7), not just event titles
- Collect outcomes from ALL active markets per event, filter to >1% price
- Reorder outcome_prices to surface topic-matching outcome before top-3 truncation
- Add SKILL.md "Prediction Markets" synthesis section with structural/long-term
market preference, domain examples, citation format, and narrative weaving
- Add Polymarket to citation priority list between HN and Web
- Update stats box template to show up to 5 market odds
- Fix render.py "vol24h" label to "volume"
- Add NCAA seed fixture event for outcome-only matching tests
- 82 polymarket tests pass (14 new)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Make the "omit 0-result sources" instruction more emphatic (CRITICAL prefix,
enumerate all formats to suppress). Change Polymarket stats line to show
top 2-3 market odds instead of just top-1.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Polymarket results now rank by text similarity, volume, liquidity, price
movement, and competitive score instead of API return position. Also fixes
pagination (DEPTH_CONFIG now controls page count, not a no-op limit param)
and caps results after re-ranking.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Search Polymarket's free Gamma API for relevant prediction markets on any
topic. Uses smart multi-query expansion to cast a wider net (e.g., "Arizona
Basketball" also searches "Arizona"), merges and dedupes by event ID, and
shows price movement context ("up 22.5% this week"). No API key required.
Also hides sources with zero results from the stats output (all sources).
54 new tests, all passing. Full pipeline integration with scoring, dedupe,
cross-source linking, and rendering.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Shoutout to @ARJ999 (first HN submission, PR #12), @wkbaran
(PR #26, referenced in planning), and @gbessoni for endorsing
HN as the right addition.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Reorganize the changelog to match the real story: smarter scoring,
cross-source linking, and handle resolution as one "dramatically
better results" narrative first, then HN, then handle resolution
details. Moves blinded comparison right after the results section
as proof.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Handle resolution was only triggering for "person/brand" topics. Products
and tools like "Nano Banana Pro" or "Seedance" can also have X accounts
(@nanobanana, @seedanceai) but were being skipped.
Changes:
- Broadened trigger: people, brands, products, tools, companies, communities
- Added parody/fan account verification guidance
- Added site:x.com to WebSearch query for better results
- Updated both SKILL.md and OpenClaw variant
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Matt Van Horn