Commit Graph
100 Commits
Author SHA1 Message Date
Trevin Chow eb2d8b55e0 Merge pull request #344 from dzivkovi/feat/config-enablement
feat: configuration enablement — env-var defaults + source resilience
2026-05-17 01:05:08 -07:00
Trevin Chow 1a8ffd4847 fix(quality_nudge): also guard Instagram silent-failure on INCLUDE_SOURCES allowlist 2026-05-17 01:03:55 -07:00
Trevin Chow 1814bb1967 fix(quality_nudge,bluesky): gate Instagram nudge on EXCLUDE_SOURCES + anchor bluesky tests at resolver 2026-05-17 01:03:55 -07:00
Trevin Chow f236cff86a chore(pr-344): adapt to rebased base — fix test imports + memory-dir doc style
Rebased onto current main where:
- instagram.py uses unified http.get (not _requests fallback); tests now
  mock http.get and assert params/timeout kwargs.
- quality_nudge tests use lib.* import path with sys.path setup.
- README/CONFIGURATION.md memory-dir lines say "defaults to" so they pass
  test_no_stray_hardcoded_memory_dir_paths.
2026-05-17 01:03:55 -07:00
Trevin Chow d9a0ac31f2 Merge pull request #323 from GAOJIAN-0106/fix/openrouter-default-model-id
fix: correct invalid OPENROUTER_DEFAULT model ID
2026-05-17 01:01:54 -07:00
Trevin Chow 3e60c0817d Merge pull request #339 from dzivkovi/docs/configuration-md
docs: add CONFIGURATION.md + README pointers
2026-05-17 01:01:37 -07:00
Trevin Chow aef6f35460 docs(providers): note why OpenRouter slug includes -preview suffix 2026-05-17 00:59:31 -07:00
Trevin Chow 71b1e8a411 Merge pull request #302 from nidhi-singh02/fix/github-repo-canonicalization
fix: Canonicalize ambiguous GitHub repo resolution for product comparisons
2026-05-17 00:57:05 -07:00
Trevin Chow 618458eb7e Merge pull request #320 from kaushikgopal/feat/brave-cookie-extraction
Add Brave browser support for X/Twitter cookie extraction
2026-05-17 00:56:44 -07:00
Trevin Chow 8cccd3e982 Merge pull request #334 from iamitp/codex/last-run-config-state
Preserve clean mode for last run state
2026-05-17 00:56:28 -07:00
Trevin Chow 17fb17222b fix(docs): use 'defaults to' wording for LAST30DAYS_MEMORY_DIR refs
The version_consistency test (test_no_stray_hardcoded_memory_dir_paths)
flagged 4 lines where `~/Documents/Last30Days/` appeared without the
canonical "defaults to" phrasing or the ${LAST30DAYS_MEMORY_DIR:-...}
literal form. Tightened the wording in the CONFIGURATION.md table +
footer paragraph, and the README "Where research files are saved"
section, so each path mention is anchored at the env-var override
contract rather than as a bare hardcoded default.
2026-05-17 00:54:49 -07:00
Trevin Chow 8ccd778366 fix(canonicalization): predicate-based call lookup + skip double-canon on auto-resolve
Two findings from Greptile review on PR #302:

1. tests/test_cli_v3.py:302 - The test asserted run_mock.call_args_list[0]
   was the main runner's invocation, but fanout.run_competitor_fanout
   submits main + competitors to a ThreadPoolExecutor and iterates with
   as_completed. With zero-latency mocks, thread scheduling determines
   which pipeline.run call lands first, so the competitor's call could
   take index [0] and flake CI. Replace [0] indexing with a predicate
   match on the canonicalized github_repos kwargs.

2. skills/last30days/scripts/last30days.py:662 - When auto_resolve returns
   github_repos, it has already run canonicalize_github_repos(cap=5) and
   ranked by relevance. The downstream block then re-canonicalized with
   cap=None, which can re-sort by topic-slug match and clobber the
   auto_resolve relevance order. Guard the second canonicalization with
   a repos_from_auto_resolve flag so it only fires for user-supplied
   --github-repo input.
2026-05-17 00:51:21 -07:00
Trevin Chow a3f173dc8a docs(readme): drop community-video link from "Going deeper" callout
The 6-min architecture walkthrough video is the contributor's own
work — not affiliated with the project. Removing the README pointer
to avoid implying endorsement. CONFIGURATION.md prose stands.
2026-05-17 00:51:14 -07:00
Trevin Chow 5a3ac8ca37 docs(config): document briefing.py show [--date DATE] subcommand 2026-05-17 00:50:36 -07:00
Trevin Chow e8eb15102f docs(agents): steer agents on maintaining CONFIGURATION.md 2026-05-17 00:50:36 -07:00
Trevin Chow 0f03a67166 Merge pull request #343 from Bortlesboat/codex/use-sandboxed-safari-cookie-path
fix: prefer sandboxed Safari cookie path
2026-05-17 00:47:32 -07:00
Trevin Chow 5a625fda9f Merge pull request #355 from dinakars777/test/cover-parallel-grounding-backend
test: cover parallel grounding backend
2026-05-17 00:47:20 -07:00
Trevin Chow b296a65515 fix(last-run): guard python3 absence + hoist datetime + use context manager 2026-05-17 00:44:30 -07:00
Trevin Chow f2737fc035 test(grounding): fix published_date → publish_date mock key mismatch 2026-05-17 00:42:07 -07:00
Trevin Chow 9ce7264d43 test(grounding): add serper>parallel priority + parallel empty-results coverage 2026-05-17 00:41:19 -07:00
Trevin Chow 8f565ee241 fix(chrome_cookies): sort Brave profiles by mtime, not alphabetically 2026-05-17 00:40:10 -07:00
Trevin Chow 5ab8c3ba76 Merge pull request #345 from dinakars777/docs/fix-stale-script-paths
docs: fix stale script paths
2026-05-17 00:38:17 -07:00
Trevin Chow 38bfb504e1 Merge pull request #349 from dinakars777/chore/sync-gemini-extension-version
chore: sync gemini extension version
2026-05-17 00:38:01 -07:00
Trevin Chow e8f23b4205 Merge pull request #340 from dzivkovi/fix/youtube-transcript-observability
fix(youtube): surface transcript-fetch ratio + add degraded nudge for stale yt-dlp
2026-05-17 00:31:37 -07:00
Trevin Chow b78ce34922 test(safari_cookies): add coverage for legacy fallback path 2026-05-17 00:31:09 -07:00
Trevin Chow 16a4fa9c39 Merge pull request #341 from flyingice/main
fix(grounding): align Parallel AI search with current API schema
2026-05-17 00:27:50 -07:00
Trevin Chow 3b75ff1537 Merge pull request #354 from dinakars777/fix/parallel-web-backend-source
fix: route parallel web backend through grounding
2026-05-17 00:26:50 -07:00
Trevin Chow 321975e144 Merge pull request #356 from dinakars777/fix/allow-threads-pinterest-search
fix: honor explicit optional source requests
2026-05-17 00:26:25 -07:00
Trevin Chow 5f9c637bbe Merge pull request #336 from davemorin/fix/319-xquik-source-capabilities
fix(planner): register xquik in SOURCE_CAPABILITIES (#319)
2026-05-17 00:26:08 -07:00
Trevin Chow 19132b0b5e Merge pull request #338 from dzivkovi/fix/windows-save-path-footer
fix(render): use forward slashes in save-path footer for Windows
2026-05-17 00:25:45 -07:00
Trevin Chow 1e4150ad78 Merge pull request #347 from dinakars777/docs/update-how-search-key-files
docs: update search key file paths
2026-05-17 00:24:55 -07:00
Trevin Chowandgreptile-apps[bot] 6a15afd8e8 Update skills/last30days/scripts/last30days.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-05-17 00:23:53 -07:00
Trevin Chow 0ab7051bc5 fix(planner): also register xquik in QUICK_SOURCE_PRIORITY 2026-05-17 00:22:04 -07:00
Trevin Chow ec0b126af6 test(pipeline): relax grounding assertion to stable source key 2026-05-17 00:21:57 -07:00
Trevin Chow d9e8a046ef docs(how-search): swap score.py->relevance.py at line 145 2026-05-17 00:21:36 -07:00
Trevin Chow 261ea5895c refactor(pipeline): remove dead threads-explicit-request branch 2026-05-17 00:20:56 -07:00
Trevin Chow 2692e0f4a2 Merge pull request #351 from dinakars777/docs/fix-changelog-skill-link-note
docs: correct changelog skill link note
2026-05-17 00:20:46 -07:00
Trevin Chow 10f35f82fe fix(grounding): guard parallel excerpts against None + cap at 500 chars 2026-05-17 00:20:27 -07:00
Trevin Chow 6b40d2c46f Merge pull request #346 from dinakars777/docs/fix-bug-report-repro-command
docs: fix bug report repro command
2026-05-17 00:20:21 -07:00
Trevin Chow 4a99c4f557 Merge pull request #337 from UncleMike1988/fix/path-quoting-spaces
Fix path-quoting in SessionStart check-config hook (handles spaces in…
2026-05-17 00:20:09 -07:00
Trevin Chow 0e353ae03f fix(render): apply as_posix to fallback branch + hoist shutil import 2026-05-17 00:19:49 -07:00
Trevin Chow 7214dd6051 Merge pull request #348 from dinakars777/docs/fix-readme-skill-link
docs: fix runtime skill spec link
2026-05-17 00:04:30 -07:00
Trevin Chow 6acf2fdbe2 Merge pull request #419 from mvanhorn/chore/remove-orphaned-spec-tasks
chore: remove orphaned SPEC.md and TASKS.md
2026-05-17 00:03:06 -07:00
Trevin Chow 07a3bdb3cf Merge pull request #364 from davemorin/fix/361-unsafe-eval-check-config
fix(hooks): replace unsafe eval with declare in check-config.sh
2026-05-17 00:02:44 -07:00
Trevin Chow f3df47c381 chore: remove orphaned SPEC.md and TASKS.md
Both files lived at the repo root as pre-plugin-layout artifacts. On
current main neither is referenced from README, SKILL.md, AGENTS.md,
CHANGELOG, or docs/ — no inbound links to break by removing. Git history
preserves the content for anyone who needs to dig it up.

Closes #352, #353. The PRs by @dinakars777 correctly flagged the drift;
deletion is the cleaner resolution than annotating them as historical.
2026-05-17 00:00:47 -07:00
Trevin Chow 46cf2328aa fix(hooks): use printf -v for bash 3.2 compat (declare -g is 4.2+)
macOS ships /bin/bash 3.2 and the script uses #!/bin/bash with
set -euo pipefail, so declare -g would abort the SessionStart hook
with "invalid option" on every Mac. printf -v writes via assignment
semantics (global from inside a function on 3.2+) — same scope
outcome, broader compatibility.
2026-05-16 23:49:44 -07:00
Trevin Chow 7506cbd542 fix(hooks): scope ENV_* to global (declare -g) so caller sees values 2026-05-16 23:49:05 -07:00
Trevin Chow aba6172032 Merge pull request #366 from davemorin/feat/324-reddit-json-fallback
feat(web): auto-enrich Reddit URLs from web search via JSON API
2026-05-16 23:41:57 -07:00
Trevin Chow d07e4698e3 Merge pull request #358 from dinakars777/fix/openclaw-poll-clock-init
fix: initialize OpenClaw poll timing once
2026-05-16 23:41:42 -07:00
Trevin Chow 4c0282dd55 Merge pull request #365 from davemorin/fix/284-version-metadata-drift
fix(version): replace hardcoded v3.0.0 with dynamic _skill_version()
2026-05-16 23:40:53 -07:00
Trevin Chow 99909fca67 Merge pull request #368 from hnshah/ren/advisory-security-workflow-252
ci: add advisory security workflow
2026-05-16 23:40:28 -07:00
Trevin Chow 36c43d50b7 test: drop side_effect padding to match collapsed time.time() call 2026-05-16 23:39:14 -07:00
Trevin Chow e2d9d705f6 review: fix selftext key path + break on RedditRateLimitError 2026-05-16 23:37:00 -07:00
Trevin Chow 9f39d10bc5 Merge pull request #373 from hnshah/ren/watchlist-sightings
feat(store): record per-run finding sightings
2026-05-16 23:24:09 -07:00
Trevin Chow 03043da407 Merge pull request #418 from tmchow/chore/greptile-config
chore: add greptile.json (triggerOnUpdates + statusCheck)
2026-05-16 23:23:48 -07:00
Trevin Chow 8bab997854 chore: add greptile.json to opt into update-triggered reviews + status check
Without this config, Greptile's documented default is `triggerOnUpdates: false`
(only the initial PR open triggers a review). Empirically Greptile has been
re-reviewing on force-push to this repo anyway, but documenting the intent
makes the behavior reliable across plan changes and any future config-source
shifts on Greptile's side.

`statusCheck: true` registers Greptile as a GitHub status check (not just a
PR comment). That gives maintainer-tooling a machine-readable heartbeat -
poll `GET /repos/.../commits/SHA/check-runs` and filter by app name to see
whether Greptile is `queued` / `in_progress` / `completed`. Without it the
only signal is "did a new Greptile comment appear" which is silently
ambiguous when Greptile re-reviews and finds nothing new.

If `statusCheck` is OSS-plan-restricted Greptile silently ignores the key,
which is fine - the rolling-summary comment with `Confidence Score: N/5`
remains the fallback signal.

Refs greptileai/skills `greploop` skill for the terminal-state pattern this
config enables.
2026-05-16 23:20:24 -07:00
Trevin Chow 791c0a57a0 review: gate web Reddit enrichment behind EXCLUDE_SOURCES
PR #366 routes Reddit URLs found in web-search results through the public
Reddit JSON API to recover thread body + top comments (the Claude Code
WebFetch tool blocks reddit.com directly). That bypass is sound and the
fixed problem is real - but the always-on shape ignores user intent on
source gating.

A user who sets EXCLUDE_SOURCES=reddit to suppress Reddit results would
still get Reddit content smuggled back in via web-search URLs that
happen to point at reddit.com threads. This contradicts the suppression
contract that EXCLUDE_SOURCES is supposed to provide (see
lib/pipeline.available_sources where the same env var gates the
top-level Reddit source).

Add a _reddit_excluded(config) check in web_search() that mirrors the
parsing pattern from lib/pipeline (comma-separated, case-insensitive,
whitespace-tolerant). When reddit is in EXCLUDE_SOURCES, skip the
enrichment pass entirely - the web results themselves still flow
through, but they're not augmented with Reddit body/comments.

Four new tests in test_grounding_v3.py cover:
- EXCLUDE_SOURCES=reddit skips enrichment
- case-insensitive parsing matches REDDIT/Reddit/whitespace-padded/csv
- Other sources in EXCLUDE_SOURCES don't trigger the gate
- Enrichment runs normally when reddit isn't excluded

19/19 grounding tests pass.
2026-05-16 22:52:06 -07:00
Trevin Chow d7b3995da1 Merge pull request #357 from dinakars777/fix/windows-env-permission-warning
fix: skip POSIX secret warning on Windows
2026-05-16 22:42:26 -07:00
Trevin Chow e217db77cc Merge pull request #369 from voidborne-d/fix/scrapecreators-100-credits
docs: correct ScrapeCreators free tier to 100 credits (closes #367)
2026-05-16 22:41:16 -07:00
Trevin Chow 2c2cfb9e7e Merge pull request #417 from tmchow/docs/eval-not-in-ci-solution
docs: capture eval-not-in-CI design decision under docs/solutions/
2026-05-16 22:31:39 -07:00
Trevin Chow 3276496f49 Merge pull request #376 from shoobee/feat/yt-dlp-ssh-routing
feat(youtube): route yt-dlp through SSH host for residential IP egress
2026-05-16 22:31:18 -07:00
Trevin Chowandhnshah 68ae74ff4f docs: capture eval-not-in-CI design decision under docs/solutions/
Closes #374 (adapted, not 1:1 merged).

@hnshah opened PR #374 proposing a docs/adr/ directory for architecture
decision records. The intent is right -- the "why is search-quality eval
manual?" reasoning drifts out of memory if it isn't written down -- but
the docs/adr/ convention doesn't fit alongside the existing
docs/solutions/ structure (compound-engineering ce-compound pattern with
frontmatter metadata, additive entries, no membership-contract test).

This commit adopts hnshah's ADR 002 content (search-quality eval is
manual by default) as a docs/solutions/architecture/ entry with the
canonical compound-style frontmatter (module, problem_type, applies_when,
related_components, tags). Drops the docs/adr/ directory pattern, the
README index, and the test_adr_docs.py contract test.

ADR 001 (multi-surface packaging) is intentionally not adopted here: it
referenced sync.sh as the deploy mechanism, but sync.sh was removed in
PR #405 in favor of `npx skills add . -g -y`. The multi-surface
packaging story is still real but has moved beyond what the original
ADR captured; a fresh "how we ship to multiple harnesses" entry would
make sense as a separate doc.

Co-authored-by: hnshah <hnshah@users.noreply.github.com>
2026-05-16 22:22:59 -07:00
Trevin Chow 27c90504c0 review: validate SSH host alias + rename LAST30DAYS_YT_SSH_HOST -> LAST30DAYS_YOUTUBE_SSH_HOST
Addresses two concerns surfaced during PR #376 review:

1. **SSH option-injection on the host value.** The original PR uses
   shlex.quote() on the remote command and added a `--` option terminator
   in front of the host, but neither one stops a hostile env var like
   `LAST30DAYS_YT_SSH_HOST=-oProxyCommand=...` from being read in the
   first place. Tighten `_ytdlp_ssh_host()` to validate the host against
   `^[a-zA-Z0-9._-]+$` (plain hostname/SSH-config-alias shape: letters,
   digits, dot, underscore, hyphen). Any value that doesn't match logs a
   warning to stderr and returns None, so the wrap function falls back to
   local execution. The `--` terminator stays as defense-in-depth for the
   case where a valid host happens to start with `-`, but the regex closes
   the door on the env var reaching ssh at all.

2. **Env var naming consistency.** Existing skill-internal config knobs
   spell out their domain: `LAST30DAYS_X_BACKEND`, `LAST30DAYS_X_MODEL`,
   `LAST30DAYS_PLANNER_MODEL`, `LAST30DAYS_RERANK_MODEL`, etc. The module
   is `youtube_yt.py`, the source key is `youtube`, the function family
   is `is_youtube_*()` — `YT` was the odd abbreviation out. Rename to
   `LAST30DAYS_YOUTUBE_SSH_HOST` so the variable matches the user mental
   model ("route YouTube fetches via residential IP") and the codebase's
   spelled-out convention.

Adds three new tests:
- test_host_alias_with_dash_prefix_is_rejected (validator rejects `-o...`)
- test_host_alias_with_shell_metacharacters_is_rejected (rejects spaces, ;, $, `, &)
- test_host_alias_validator_accepts_realistic_aliases (allows FQDNs, IPs, bare aliases)

The existing test_wrap_cmd_uses_option_terminator is rewritten to use a
valid host value (since an invalid one is now filtered upstream) and
continues to assert the `--` terminator placement as defense-in-depth.

44/44 youtube_yt tests pass (40 prior + 4 net new validator tests).
2026-05-16 22:19:35 -07:00
Trevin Chow 0e2059661a Merge pull request #378 from j-sperling/chore/gemini-3.1-flash-lite-ga
chore: migrate to gemini-3.1-flash-lite GA model
2026-05-16 22:12:20 -07:00
Trevin Chow b1c5f8db82 Merge pull request #382 from lustrousgorilla/bugfix/reddit-gaierror-retry
fix(http): expand retry budget + exponential backoff on DNS resolution failure
2026-05-16 22:11:56 -07:00
Trevin Chow 89c5cb9d5d Merge pull request #416 from tmchow/worktree-inherited-discovering-pebble
fix(ci): run full pytest suite, repair 13 rotted tests
2026-05-16 22:11:34 -07:00
Trevin Chow 5d4f9ef2c5 fix(store): use UTC for all date arithmetic against SQLite columns
Greptile flagged that _cli_query's --since parsing uses datetime.now()
(local time) while first_seen is stored via SQLite's datetime('now') (UTC).
The same bug exists in three other call sites that compare against either
first_seen or run_date (both UTC):

- get_daily_cost: "today" defaults to local date, returns wrong day's cost
  near UTC midnight
- get_stats: "7 days ago" cutoff for runs_7d / successful_7d
- get_trending: "N days ago" cutoff for finding activity ranking
- _cli_query: "N days ago" cutoff for --since flag (Greptile's flag)

All four now use datetime.now(timezone.utc). Same root cause and same fix
as the test_get_new_findings_filters_by_date repair in the previous commit.
2026-05-16 21:40:25 -07:00
Trevin Chow eb2d7a0f37 fix(ci): run full pytest suite, repair 13 rotted tests
CI was running only test_plugin_contract.py and test_version_consistency.py
(2 of 84 test files), masking 13 rotted tests across 4 clusters. The suite is
fully offline-safe (1402 tests in ~7s without network), so the narrow scope
wasn't gating integration flakiness; it was just stale. validate.yml now runs
`uv run pytest` against the full suite.

Engine fix: store.findings_from_report is rerank-first. ranked_candidates is
the primary persistence path; hackernews/polymarket are unconditionally
supplemented from items_by_source because they rank poorly but matter for
watchlists. When ranked_candidates was empty (rerank failed or skipped),
reddit, x, and every other source were silently dropped. The supplement loop
now falls back to all sources only when ranked_candidates is empty; the normal
path is unchanged.

Test repairs:
- test_store.py (6) + test_watchlist_commands.py (2): cascade from the engine fix
- test_get_new_findings_filters_by_date (latent): local-time vs SQLite UTC
  flake — switched to datetime.now(timezone.utc)
- TestPollDeviceAuth (3): mock_time.time side_effect lists too short after
  impl added a last_reminder call — padded timeout test, pinned others to
  return_value=0 (loops terminate via urlopen, not the clock)
- test_bare_run_emits_web_promo: engine reads ~/.config/last30days/.env, so
  a contributor's saved EXA/PARALLEL key made grounding "available" and
  suppressed the web promo. Also missing X made the "x" promo preempt "web".
  Set LAST30DAYS_CONFIG_DIR="", subprocess cwd=tmpdir, XAI_API_KEY stub.
2026-05-16 21:34:22 -07:00
Trevin Chow 719cdef2fb fix(http): contain DNS retry-budget widening to DNS path only
PR #382 introduced an `effective_retries` widening on the first gaierror,
but the widening leaked: every non-DNS error path (HTTPError, non-DNS
URLError, OSError) was gated on `effective_retries - 1` and so inherited
the expanded bound. A caller passing `retries=2` who hit DNS-then-non-DNS
got 3 attempts instead of 2 — contrary to the PR description and the
fail-fast intent of small retry budgets.

Fix:
- Gate every non-DNS sleep/retry decision on the caller's original
  `retries`, not the widened `effective_retries`.
- Add an explicit `break` in each non-DNS branch when the original
  budget is exhausted, so the (possibly widened) outer loop bound
  can't pull us into an extra attempt.

Adds two regression tests covering the DNS-then-non-DNS-URLError and
DNS-then-OSError sequences flagged in Greptile review on PR #382.
2026-05-16 21:13:17 -07:00
Trevin Chow ac04b56acc Merge pull request #383 from lustrousgorilla/bugfix/bird-x-json-decode-retry
fix(bird_x): retry subprocess on non-JSON stdout (HTML anti-bot interstitial)
2026-05-16 21:08:35 -07:00
Trevin Chow 42bfc6c76c Merge pull request #415 from tmchow/fix/sc-source-gating-consistency
fix(sources): align SC source gating between code and docs
2026-05-16 21:08:10 -07:00
Trevin Chow 9f08bb68b5 fix(sources): align SC source gating between code and docs
Two related drifts surfaced while reviewing PR #399 (EXCLUDE_SOURCES) —
docs claimed several SC-backed sources required INCLUDE_SOURCES opt-in
that the code didn't actually enforce, and threads was inconsistently
gated relative to its same-key siblings.

This commit picks the "code as source of truth + EXCLUDE_SOURCES as
suppression knob" model and aligns docs to match. It also promotes
threads to the same auto-on tier as tiktok and instagram, since all
three share the SC key and per-call cost shape — there was no real
product reason for threads being opt-in while the other two weren't.

The resulting source-gating model is three-tier and intentional:

  • **Auto-on if backing infra present** (suppress via EXCLUDE_SOURCES):
    reddit, HN, polymarket, X, youtube, github, bluesky, truthsocial,
    grounding, **tiktok, instagram, threads**

  • **INCLUDE_SOURCES persistent opt-in** (cost/billing reasons):
    perplexity (different paid API — OpenRouter),
    tiktok_comments / youtube_comments (N× extra SC calls per video)

  • **--search per-query opt-in** (relevance reasons):
    pinterest (visual pins, narrow utility),
    xiaohongshu (Chinese-market specific)

Changes:

- env.py: `is_threads_available()` drops the INCLUDE_SOURCES check,
  now mirrors tiktok/instagram (SC key → True). Docstring updated.
- tests/test_env_v3.py: new `ThreadsAvailabilityTests` class locks in
  the new contract and includes a regression guard ("INCLUDE_SOURCES
  should not be needed").
- SKILL.md: lines 333-338 rewritten so the model's "Build
  ACTIVE_SOURCES_LIST" checklist reflects what the engine actually
  runs. Drops false INCLUDE_SOURCES requirement for
  tiktok/instagram/threads; corrects pinterest to mention --search;
  adds missing INCLUDE_SOURCES=perplexity requirement.
- README: same alignment for the user-facing "Everything else in v3"
  section.

Note on EXCLUDE_SOURCES references in the new docs: the suppression
flag is wired up in PR #399. SKILL.md and README mention EXCLUDE_SOURCES
as the opt-out path; that prose is forward-looking until #399 lands.
The behavior changes in this PR (threads auto-on) are self-contained
and don't require #399 to function — but for users who want to suppress
the newly-auto-on threads source, #399 needs to land first.
2026-05-16 20:17:31 -07:00
Trevin Chow 602de1ebda Merge pull request #388 from bradferguson/fix/sc-youtube-and-hn-tokenization
fix(sources): unblock SC YouTube + multi-token HN searches
2026-05-16 19:41:12 -07:00
Trevin Chow bf3a82a87e Merge pull request #389 from kuyua9/fix/save-comparison-html-kuyua9
fix: save comparison HTML artifacts
2026-05-16 19:40:59 -07:00
Trevin Chow c010feb8f8 Merge pull request #399 from spiky02plateau/feat/exclude-sources-banner-and-pipeline
feat: honor EXCLUDE_SOURCES env var in source count + pipeline filter
2026-05-16 19:39:51 -07:00
Trevin Chow 4d4ac97ffb refactor: hoist comparison-html gate into a single condition (Greptile DRY)
Greptile flagged that `entity_reports and args.emit == "html"` appeared in
two places — once when computing the footer display path, again when calling
save_output. The else-branches differ between the two callsites (the display
needs `report.topic` as fallback; the save call needs `None` so save_output
falls back to the report's own topic), so collapsing into one shared
expression would be wrong, but hoisting just the condition into a single
`is_comparison_html` bool eliminates the risk of drift while keeping the
two callsites' fallback semantics distinct.
2026-05-16 19:32:49 -07:00
Trevin Chow 85255be350 Merge pull request #414 from mvanhorn/dependabot/uv/pytest-9.0.3
chore(deps-dev): bump pytest from 9.0.2 to 9.0.3
2026-05-16 19:31:46 -07:00
Trevin Chow 1aa120a420 Merge pull request #407 from DamienStevens/feat/macos-keychain-source
feat(env): macOS Keychain credential source
2026-05-16 19:31:34 -07:00
Trevin Chow 306d8c2d73 fix(env): wire EXCLUDE_SOURCES through get_config + SKILL.md integration
The original PR added EXCLUDE_SOURCES filtering to pipeline.available_sources()
and to the check-config.sh banner, but env.py::get_config() builds its config
dict from a hardcoded keys list that didn't include EXCLUDE_SOURCES. The
result: setting EXCLUDE_SOURCES in the environment silently no-op'd through
the Python pipeline. Only the bash hook (which reads shell env directly)
worked. The PR's unit tests didn't catch this because they construct config
dicts directly, bypassing get_config().

Changes:
- Add ('EXCLUDE_SOURCES', '') to env.py's keys list so the env var actually
  propagates into config.
- Add an end-to-end regression test that goes through get_config() rather
  than constructing config dicts directly.
- Document EXCLUDE_SOURCES in SKILL.md's source-list checklist so the model
  invoking the skill knows to subtract excluded sources before displaying
  the active-sources line. (Per AGENTS.md: engine flags without SKILL.md
  prose are incomplete — the agent invoking the skill won't know the flag
  exists.)
2026-05-16 19:30:27 -07:00
Trevin Chow d0dcf751f1 fix(keychain): single source of truth for key list + robust USER fallback
Addresses Greptile review on PR #407:

- P1: setup-keychain.sh ALL_KEYS was missing GOOGLE_GENAI_API_KEY and
  XIAOHONGSHU_API_BASE relative to _load_keychain's inline list, so users
  manually storing those keys would not see them in --list and the
  interactive prompt would never offer to set them.

  Hoist the canonical key list into lib/env.py::KEYCHAIN_KEYS, have
  get_config() pass it through, and add a parity test that parses
  ALL_KEYS out of setup-keychain.sh and asserts equality. Drift is now
  caught at CI time instead of after a user reports a missing key.

- P2: os.environ.get("USER", "") silently returned "" under sudo, in
  Docker without --env USER, or in CI runners that strip USER. The
  resulting `security find-generic-password -a ""` call would never
  match items stored by setup-keychain.sh, so all lookups silently
  returned nothing. Fall back to pwd.getpwuid(os.getuid()).pw_name when
  USER is absent.

The P2 process-listing comment ("secret visible briefly via ps because
security has no stdin path for -w") has no clean fix — the README
already documents the manual `security add-generic-password` invocation
as an alternative for users with strict secret hygiene.
2026-05-16 19:25:05 -07:00
Trevin Chow 14d8f62e02 Merge pull request #413 from tmchow/docs/compound-release-cascade-pattern
docs: compound learning on release-time consistency-test cascade failures
2026-05-16 19:12:31 -07:00
Trevin Chow 0fd532d249 docs: compound learning on release-time consistency-test cascade failures
Documents the cascade pattern surfaced during this session's install-modernization
arc: a `test_sync_cache_path_uses_skill_version` test asserted that a hardcoded
version pin in `sync.sh` matched the version frontmatter in SKILL.md. When a
release bumped SKILL.md, every open PR's CI failed simultaneously on the
unrelated stale-pin assertion. Affected at least 5 PRs across the 2026-05-13
to 2026-05-15 window (#400, #390, #392, and two others) plus required hotfix
PR #397 to unblock the queue.

The permanent fix shipped in PR #405 (deleted sync.sh + the test). This doc
captures the design lesson so the pattern doesn't reappear: don't write
consistency tests that read two files and assert one matches a value derived
from the other. Either derive at runtime from a single source of truth, or
self-skip / merge-base-scope the test so deletion is a non-event.

Created via /ce-compound. Includes:

- docs/solutions/workflow-issues/release-consistency-test-cascade-2026-05-16.md
  (the new learning — first entry under docs/solutions/)
- CONCEPTS.md (new — 4 entries: Skill, Engine, Harness, Beta channel,
  capturing project-specific vocabulary that surfaced across the session)
- AGENTS.md (added one-line Structure entries surfacing docs/solutions/ and
  CONCEPTS.md so fresh agents discover them)
- docs/plans/2026-04-22-{002,003,005,006}-*-plan.md (added deprecation banner
  to each, pointing readers at PR #405 and the new docs/solutions entry —
  these 4 historical plans still reference the deleted sync.sh inline)

Also: closed PR #379 (j-sperling's workaround for the same cascade,
superseded by PR #405's permanent fix).
2026-05-16 19:08:02 -07:00
Trevin Chow 8867a007ea Merge pull request #392 from Gujiassh/fix/openclaw-scrapecreators-optional-env
fix(openclaw): make ScrapeCreators key optional
2026-05-16 18:50:07 -07:00
Trevin Chow 01b5f3dc1e Merge pull request #363 from thinkun/pr/claim-contributor-entry
Claim contributor entry — @thinkun
2026-05-16 18:46:35 -07:00
Trevin Chow 2e39ee8ce4 Merge pull request #412 from tmchow/refactor/skill-meta-version-helper
refactor: consolidate SKILL.md version regex into lib/skill_meta.py
2026-05-16 18:42:49 -07:00
Trevin Chow 37033164da Merge pull request #410 from tmchow/docs/agents-orientation-multi-harness
docs: reframe as multi-harness Agent Skills package, flip CLAUDE.md ↔ AGENTS.md
2026-05-16 18:42:36 -07:00
Trevin Chowandgreptile-apps[bot] 9fe4b8f130 Update AGENTS.md
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-05-16 18:42:14 -07:00
Trevin Chow 73dc6b9996 refactor: consolidate SKILL.md version regex into lib/skill_meta.py
The same `^version:\s*"([^"]+)"\s*$` regex (or a slight variant) was
duplicated across three files: render.py inline, test_plugin_contract.py
local helper, test_version_consistency.py local helper. A future change
to the SKILL.md frontmatter version format would have needed to update
three places without any compile-time pressure to keep them in sync.

New skills/last30days/scripts/lib/skill_meta.py provides:
- `_VERSION_RE` private compiled pattern (accepts double-quoted,
  single-quoted, or unquoted YAML version scalars per the widening
  landed in 997708a)
- `read_skill_version(skill_md_path: Path) -> str | None` helper that
  catches OSError + UnicodeDecodeError and returns None on miss

Callers updated:
- render.py::_skill_version now calls skill_meta.read_skill_version
  inside the SKILL.md fallback loop, returning `read_skill_version(...) or "?"`.
  Semantically equivalent to the old break-after-first-SKILL.md logic.
- test_plugin_contract.py and test_version_consistency.py import the
  helper instead of defining the regex inline. Both files use the
  established sys.path.insert pattern.

Added tests/test_skill_meta.py with 6 direct unit tests covering the
helper's full contract: missing file, undecodable bytes, no-version-line,
and all three quoting styles (double, single, unquoted). Previously the
helper was only exercised transitively through render._skill_version().

Added test_skill_md_uses_double_quoted_version to
test_version_consistency.py — the old per-test regex incidentally
asserted "this repo's SKILL.md uses double-quotes" by being strict;
the shared helper accepts all three styles, so the assertion is now
explicit instead of implicit.

Code-reviewed by ce-code-review (8 reviewers); safe_auto fixes applied
inline (rename to _VERSION_RE, group or-chain instead of generator,
docstring tightened, dropped unnecessary `from __future__ import
annotations`, tightened signature to Path-only).

Conftest.py refactor for the sys.path.insert duplication across ~20 test
files filed as issue #411 — out of scope for this PR (touches many
files, separate concern).

Test results: 23 passed in the affected test set (16 prior + 6 new
test_skill_meta tests + 1 new double-quote assertion). Full suite shows
same 13 pre-existing failures as main; zero new failures.
2026-05-16 18:17:33 -07:00
Trevin Chow 1fd763e09f docs: flip CLAUDE.md ↔ AGENTS.md — AGENTS.md becomes canonical, CLAUDE.md points at it
Mirrors the multi-harness reframing of the project itself. CLAUDE.md is
Claude-Code-specific by name; AGENTS.md is the multi-harness convention
that Codex, Cursor, Gemini CLI, GitHub Copilot, and most other Agent
Skills hosts also read. The canonical content belongs in the file
multi-harness tooling expects.

git mv preserves history — the Orientation section and everything else
that was in CLAUDE.md is now tracked under AGENTS.md, with full blame
continuity. The new CLAUDE.md is a one-line `@AGENTS.md` reference so
Claude Code continues to load the content (it follows @ references).
2026-05-16 17:03:11 -07:00
Trevin Chow e0f6ef845a docs(claude.md): add Orientation section, reframe as multi-harness Agent Skills package
Closes the spirit of #335 (closed in favor of this fresh PR after the
sync.sh thread became obsolete via PR #405).

Two changes:

1. Top-of-file description reframed from "Claude Code skill" to
   "Agent Skills package... installable across Claude Code (most common
   host), Codex, Cursor, GitHub Copilot, Gemini CLI, and 50+ other
   Agent Skills hosts". The skill works across every major agent host
   after the install-modernization work in PR #400/#404/#405/#409.
   Calling it "Claude Code skill" undersells the surface and biases
   contributors toward Claude-Code-specific assumptions.

2. New ## Orientation section (4 bullets) framing the project for
   contributors who would otherwise read the python3 invocation in
   ## Commands and form a CLI-first mental model. Names the trap
   explicitly with one concrete invalid-syntax example
   (`/last30days OpenClaw --emit=html | pbcopy` — slash commands don't
   pass shell mechanics through). Bullets adapted from #335 with
   multi-harness framing replacing the Claude-Code-only framing.

No code changes. No SKILL.md changes. CLAUDE.md only. AGENTS.md
inherits via @CLAUDE.md.
2026-05-16 17:01:04 -07:00
Trevin Chow c918e18465 Merge pull request #409 from tmchow/refactor/skill-dir-relative-resolver
refactor(skill): replace SKILL_ROOT resolver with SKILL_DIR substitution
2026-05-16 16:55:32 -07:00
Trevin Chow 6fe0aca7ee refactor(skill): replace SKILL_ROOT resolver with SKILL_DIR substitution
The Step 1 and comparison-mode resolver loops walked a hardcoded list
of install paths trying to find scripts/last30days.py. Two problems:

1. The list was never exhaustive — it covered ~/.codex/skills, but not
   ~/.claude/skills, ~/.cursor/skills, ~/.gemini/skills, ~/.copilot/skills,
   ~/.hermes/skills/research, etc. PR #406 was about to fix that by
   enumerating more paths, but enumeration is the wrong shape.
2. The resolver could pick a different install than the SKILL.md the
   model loaded from. Spec-vs-engine divergence is subtle and confusing
   when it triggers.

The model already knows the SKILL.md path it loaded (from its Read tool
result). Templating that into the bash block is strictly better than
guessing across an enumerated list:

- Works for every harness without enumeration (Hermes, Cursor, anything
  new) because we just use wherever the harness loaded SKILL.md from
- Aligns spec with engine — the engine runs from the same install the
  spec was read from
- Deletes ~80 lines of bash across Step 1 + comparison-mode + the
  prose preamble describing the resolver

Mechanics:

- SKILL_DIR placeholder in both bash blocks — model substitutes the
  absolute path of the directory containing the SKILL.md it just Read
- One-line validation `[ ! -f "$SKILL_DIR/scripts/last30days.py" ]`
  catches bad templating with a clear error
- All references to $SKILL_ROOT replaced with $SKILL_DIR (badge
  VERSION lookup, prose description in the LAW-7 preamble area)
- STEP 0 unchanged — different concern (marketplaces stale-clone hop)

Version 3.2.3 -> 3.2.4 (behavior change: install paths the resolver
list never enumerated now work; install paths it did enumerate work
the same way they used to but via the SKILL_DIR template).
2026-05-16 16:26:45 -07:00
Trevin Chow 4a30923892 Merge pull request #405 from tmchow/docs/readme-multi-harness-install
docs+refactor: modernize install story everywhere, delete sync.sh
2026-05-15 23:46:57 -07:00
Trevin Chow 9fb19eae63 refactor: delete sync.sh, dev workflow moves to npx skills add . -g -y + native installers
Every job sync.sh did has a better replacement:

- Per-harness skill dirs (~/.claude/skills, ~/.codex/skills, ~/.agents/skills):
  `npx skills add . -g -y` writes to every detected harness's home dir and
  uses symlinks by default. Edits propagate live — no re-deploy step.
- Hermes (~/.hermes/skills/research/last30days):
  `hermes skills install mvanhorn/last30days-skill --force` pulls from
  GitHub and handles the deploy itself. The script wrapping was redundant.
- OpenClaw variant: `clawhub install last30days-official` is what users
  already run per the README; the maintainer doesn't need a separate
  variant-deploy step in the public repo's scripts.
- Claude marketplace cache (~/.claude/plugins/cache/...): this was a
  "test against the official install path" hack we shouldn't have been
  recommending. With PR #400's resolver collapse, STEP 0 no longer
  enforces the cache as the only valid SKILL.md location. Just install
  the skill normally via `npx skills` or the marketplace.

Cleanup:

- DELETE skills/last30days/scripts/sync.sh
- tests/test_version_consistency.py — drop test_sync_cache_path_uses_skill_version
- CLAUDE.md — replace the sync.sh command + rule with `npx skills add . -g -y`
- HERMES_SETUP.md — Installation now uses `hermes skills install --force`;
  developer-alternative section shows the symlink pattern for live editing
- render.py — _skill_version docstring no longer attributes the
  ".claude-plugin absent" case to sync.sh; explains it via per-harness
  install paths in general
- .github/PULL_REQUEST_TEMPLATE.md — drop the "Ran bash scripts/sync.sh"
  checklist item

CHANGELOG and historical docs (release notes, plan files) keep their
existing sync.sh mentions as accurate history.
2026-05-15 23:42:31 -07:00
Trevin Chow d1cc29d338 docs(readme): add -g (global) flag to every npx skills example
`npx skills add` defaults to project-local install (`./.skills/`,
committed with the repo). For a research-the-world skill like this one,
that's almost never what users want — they want it available across all
projects, not scoped to whichever directory they happened to run the
install from.

Adding `-g` (global) to every npx skills example in the README:
- Top-of-file install snippet
- Install table row
- Claude Code subsection's "alternative via npx skills" example
- Codex/Cursor/etc. subsection's default, per-harness, update, list,
  and remove commands

Brief one-liner explains what `-g` does and notes that dropping it
gives a project-local install for users who want team consistency on
a specific codebase.
2026-05-15 23:20:33 -07:00
Trevin Chow ded52062e6 docs(readme): drop Gemini CLI native-extension install path
The native `gemini extensions install` path was a workaround for the
v0.9.0 installer bug (still unresolved per upstream issue #11452).
Now that `npx skills add -a gemini-cli` covers Gemini cleanly with the
same install/update story as every other supported harness, the native
path is just one more confusing option to maintain. Users on Gemini get
the same recommendation everyone else does.

Removes the dedicated "Gemini CLI (native extension)" subsection and
the separate table row. Gemini CLI is now surfaced once, in the npx
skills section, alongside Codex, Cursor, Copilot, and the rest.
2026-05-15 23:03:51 -07:00
Trevin Chow 164d7ae6ed docs(readme): surface gemini-cli (and copilot, windsurf, 50+ others) in npx skills coverage
npx skills supports 50+ harnesses via the -a flag, including gemini-cli,
github-copilot, windsurf, cline, continue, roo, aider-desk, opencode,
goose, and more — not just the few I'd listed initially. Updating to
reflect that breadth.

- Top-of-file snippet now reads "Codex, Cursor, Copilot, Gemini CLI, or
  any of 50+ Agent Skills hosts" (was: "Codex, Cursor, Copilot, or any
  Agent Skills host" + Gemini listed separately in the table footer).
- Install table: same expansion; Gemini CLI native-extension row relabeled
  to clarify it's the native path (not the only Gemini option).
- npx skills subsection: lists the most common harness flags and links
  to the upstream vercel-labs/skills repo for the full list.
- Gemini CLI native subsection: now leads with "the npx skills path
  above is simpler" and frames the native install as the alternative
  for users with an existing Gemini extensions workflow or who hit
  the v0.9.0 installer bug.
2026-05-15 23:02:57 -07:00
Trevin Chow f1ce7533e6 docs(readme): recommend Claude Code plugin, add npx skills install for Codex/Cursor/Copilot
The skill is now installable across every major agent harness after the
SKILL.md path-resolver work landed in PR #400 + #404. README didn't yet
reflect that — the install table only listed Claude Code, OpenClaw, and
Gemini CLI, and the top-of-file install snippets featured Hermes (an
internal dev workflow, not a public install method).

Restructured the install section:

- Top-of-file snippets: just Claude Code (recommended, auto-updates) and
  the universal `npx skills add` one-liner. Dropped Hermes from the
  prominent spot (internal-only); pointed everything else to the Install
  section below.
- Install table: added a third column for update commands, since every
  harness now has a distinct update path worth surfacing. Added the
  `npx skills` row covering Codex/Cursor/Copilot/any Agent Skills host.
- Claude Code subsection: explains why it's recommended (marketplace
  handles versioned cache + auto-refresh) and notes that the agent-skills
  install also works on Claude Code if preferred (`-a claude-code`).
- New "Codex, Cursor, Copilot, and other Agent Skills hosts" subsection:
  shows the default install, per-harness `-a` targeting, and the update
  commands (`npx skills update last30days` for one skill, bare
  `npx skills update` for all).
- Manual (developer) subsection: switched from a clone-into-skills-dir
  recipe to a clone + symlink recipe. Symlink keeps the install in sync
  with the working tree as you edit, no re-copy on each change.

No code changes. No version bump (docs-only).
2026-05-15 22:58:54 -07:00
Trevin Chow 0b939bf703 Merge pull request #404 from tmchow/fix/json-plan-shell-quoting
fix(skill): write --plan / --competitors-plan to tmpfile (closes #403)
2026-05-15 22:52:49 -07:00
Trevin Chow 9f95efb215 fix(skill): use portable trailing-XXXXXX mktemp form for plan tmpfiles
Greptile's review flagged mktemp -t as non-portable between BSD and GNU.
The suggested replacement (mktemp "$TMPDIR/...XXXXXX.json") is correct
about dropping -t but still puts X's in the middle of the template name
(XXXXXX.json), which BSD mktemp does not substitute — only X's at the
end of the basename are replaced on BSD. Verified on macOS:

  mktemp "$TMPDIR/last30days-test.XXXXXX.json"
  → /var/folders/.../last30days-test.XXXXXX.json  (X's left literal)

The fully portable form uses trailing X's and drops the .json suffix
(engine reads by path, not extension):

  mktemp "$TMPDIR/last30days-test.XXXXXX"
  → /var/folders/.../last30days-test.DXAHzR     (X's substituted)

Verified on bash and zsh, BSD/macOS. GNU/Linux is already fine since
GNU substitutes X's wherever they appear in the basename.

Applied to both --competitors-plan (comparison-mode block) and --plan
(Step 1 block) tmpfile writes.
2026-05-15 22:50:56 -07:00