85255be350f348692c5d010ae7a340ceda25a0f4
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Trevin Chow
|
73dc6b9996 |
refactor: consolidate SKILL.md version regex into lib/skill_meta.py
The same `^version:\s*"([^"]+)"\s*$` regex (or a slight variant) was
duplicated across three files: render.py inline, test_plugin_contract.py
local helper, test_version_consistency.py local helper. A future change
to the SKILL.md frontmatter version format would have needed to update
three places without any compile-time pressure to keep them in sync.
New skills/last30days/scripts/lib/skill_meta.py provides:
- `_VERSION_RE` private compiled pattern (accepts double-quoted,
single-quoted, or unquoted YAML version scalars per the widening
landed in
|
||
|
Trevin Chow
|
9fb19eae63 |
refactor: delete sync.sh, dev workflow moves to npx skills add . -g -y + native installers
Every job sync.sh did has a better replacement: - Per-harness skill dirs (~/.claude/skills, ~/.codex/skills, ~/.agents/skills): `npx skills add . -g -y` writes to every detected harness's home dir and uses symlinks by default. Edits propagate live — no re-deploy step. - Hermes (~/.hermes/skills/research/last30days): `hermes skills install mvanhorn/last30days-skill --force` pulls from GitHub and handles the deploy itself. The script wrapping was redundant. - OpenClaw variant: `clawhub install last30days-official` is what users already run per the README; the maintainer doesn't need a separate variant-deploy step in the public repo's scripts. - Claude marketplace cache (~/.claude/plugins/cache/...): this was a "test against the official install path" hack we shouldn't have been recommending. With PR #400's resolver collapse, STEP 0 no longer enforces the cache as the only valid SKILL.md location. Just install the skill normally via `npx skills` or the marketplace. Cleanup: - DELETE skills/last30days/scripts/sync.sh - tests/test_version_consistency.py — drop test_sync_cache_path_uses_skill_version - CLAUDE.md — replace the sync.sh command + rule with `npx skills add . -g -y` - HERMES_SETUP.md — Installation now uses `hermes skills install --force`; developer-alternative section shows the symlink pattern for live editing - render.py — _skill_version docstring no longer attributes the ".claude-plugin absent" case to sync.sh; explains it via per-harness install paths in general - .github/PULL_REQUEST_TEMPLATE.md — drop the "Ran bash scripts/sync.sh" checklist item CHANGELOG and historical docs (release notes, plan files) keep their existing sync.sh mentions as accurate history. |
||
|
Trevin Chow
|
2f277dfc66 |
fix(skill): address greptile P1+P2 review feedback on PR #400
Two real bugs flagged in the automated review of PR #400; both small. 1. render.py::_skill_version manifest with no "version" key `json.loads(manifest.read_text()).get("version", "?")` returned "?" immediately on a valid JSON manifest that lacked the "version" key, never falling through to the SKILL.md frontmatter fallback. Contradicted the docstring's "Returns '?' only if both sources are missing" contract. Same shape if version is present but empty string ("" produces the broken badge `🌐 last30days v · synced ...`). Fix: pull the version out of the parsed dict, then `continue` to the next ancestor if it's None or empty. Falls through to the SKILL.md walk only after exhausting every ancestor. 2. SKILL.md STEP 0 re-read target hardcoded to nested cache layout STEP 0 told the model to re-read from `$CLAUDE_CACHE_LATEST/skills/last30days/SKILL.md` — the new nested layout. But Step 1's resolver explicitly handles both shapes (nested `{cache}/{version}/skills/last30days/` and flat `{cache}/{version}/`), noting "Both shapes ship in the wild." On an install where the highest-versioned cache happens to be the older flat shape, STEP 0's re-read target wouldn't exist; the model would silently stay on the stale marketplaces/ copy STEP 0 was supposed to move it away from — the exact failure mode this guard was added to prevent. Fix: extend the STEP 0 bash to resolve $CLAUDE_CACHE_SKILL_MD by probing both layouts, then have the model hop to that resolved path instead of constructing the path from a hardcoded suffix. Two new tests in tests/test_skill_version.py cover the missing-key and empty-string cases for fix 1. Fix 2 is exercised via the bash probe at verify time (the STEP 0 prose-contract test isn't unit-testable from Python, but the dual-layout bash is verified to resolve to the correct SKILL.md on both shapes). Stale finding skipped: greptile also flagged a missing try/except on the SKILL.md read_text() call, but that was already addressed during the ce-code-review safe_auto pass earlier in this PR — current code wraps it in `try/except (OSError, UnicodeDecodeError)`, strictly more defensive than the suggested fix. |
||
|
Trevin Chow
|
997708ad48 |
refactor(skill): apply ce-code-review fixes — bump to 3.2.2, fallback tests, comparison resolver
12 fixes from the multi-agent code review on PR #400: Version 3.2.1 -> 3.2.2 across all manifests (SKILL.md frontmatter + body header, pyproject.toml, .claude-plugin/{plugin,marketplace}.json, sync.sh cache path). The PR ships observable behavior changes (STEP 0 logic flip, resolver order change, badge fallback) that should not silently appear under the same version number — the new fallback reads SKILL.md version directly so the badge would otherwise be misleading. render.py::_skill_version: - `import re` moved to module top - _VERSION_RE extracted as a module-level compiled pattern that accepts double-quoted, single-quoted, and unquoted YAML version scalars - `break` -> `continue` on corrupt manifest, so a corrupt inner manifest no longer shadows a valid outer one - Wrap SKILL.md read_text() in try/except for UnicodeDecodeError to keep badge emission from crashing on mis-encoded SKILL.md - Docstring clarifies precedence; inline comment marks the fallback boundary between the manifest walk and the SKILL.md walk tests/test_skill_version.py (new): 7 unit tests for the fallback paths (manifest absent, manifest corrupt, corrupt-inner + valid-outer, both absent, SKILL.md without version, single-quoted, unquoted). tests/test_plugin_contract.py: tombstone test asserting .codex-plugin/ stays removed (was the only CI guard against accidental reintroduction). SKILL.md: - STEP 0 bash echoes CLAUDE_CACHE_LATEST so the model can see the resolved value when deciding whether to hop - "Both shapes ship in the wild" comment now names the two cache layouts (nested {cache}/{version}/skills/last30days/ vs flat {cache}/{version}/) - Comparison-mode bash invocation gets its own inline SKILL_ROOT resolver (latent gap: the contract tells the model to skip Step 1 on comparison queries, so SKILL_ROOT was previously unset there) CHANGELOG.md: [Unreleased] entries for the resolver rewrite and the breaking removal of Codex native-plugin support. All 9 reviewer personas surfaced findings; 3 cross-reviewer corroboration clusters were promoted (import re, "both shapes" comment, missing fallback tests). Maintainability follow-up flagged: regex now duplicated across render.py and 2 test files; could consolidate via shared lib/skill_meta.py helper in a future PR. |
||
|
Trevin Chow
|
c913e1cf89 |
refactor(skill): SKILL.md-relative path resolver, drop Codex native plugin
STEP 0 (CANONICAL PATH SELF-CHECK) used to force any SKILL.md load that wasn't
under $HOME/.claude/plugins/cache/last30days-skill/last30days/{version}/ to
re-Read from there. That guard is Claude-Code-specific (defends against the
marketplaces/ stale-clone bug) and broke under non-Claude installers like
`npx skills add`, ~/.codex/skills/, and ~/.agents/skills/.
The new STEP 0 narrows the check to its actual target: fire only when the
loaded SKILL.md path contains /.claude/plugins/marketplaces/. Every other
install path is trusted. The 2026-04-22 incident workaround is preserved
without breaking other harnesses.
Step 1 SKILL_ROOT resolver collapses the Codex-first / Claude-fallback /
CWD-fallback chain into a single precedence walk: Claude plugin cache
(versioned) first, then ~/.codex/skills, ~/.agents/skills, repo checkout,
./.skills/last30days (npx skills install dir), CWD, and GEMINI_EXTENSION_DIR.
Also drops Codex native plugin support: .codex-plugin/plugin.json is deleted,
the badge VERSION jq fallback in line 108 stops looking at it, and render.py's
_skill_version no longer scans for it. Codex users install via `npx skills add`
or the per-harness skill dir going forward.
render.py::_skill_version gains a SKILL.md frontmatter fallback so the badge
no longer emits `v?` on install dirs that sync.sh populates (which don't
include .claude-plugin/plugin.json).
|
||
|
Matt Van Horn
|
dc934ddb6a |
feat(digg): rename to 'Digg' and bump per-cluster post limits (#372)
* feat(digg): bump POSTS_PER_CLUSTER to 5 and render limit to 3
Match the per-item enrichment cap and inline-display cap used by the
other sources (Reddit, HN, YouTube, TikTok, GitHub all use 5 fetched /
3 displayed). At the previous 3/2 caps the engine routinely truncated
cluster context — a recent run on cli-printing-press lost the Jason
Calacanis quote tweet entirely because the display cut off after Garry
Tan's first two posts.
* feat(digg): rename 'Digg AI 1000' to 'Digg' in user-facing strings
Drop the 'AI 1000' suffix from the footer line, source label, inline
quote attribution ('via Digg'), why_relevant, container, mock title,
SKILL.md source list, and README sources table. Internal code comments
and docstrings still reference the upstream Digg AI 1000 product.
Bumps version to 3.2.1 and adds a CHANGELOG entry covering this rename
and the POSTS_PER_CLUSTER / render-limit bumps from the prior commit.
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
|
||
|
Matt Van Horn
|
c04bd67922 |
feat: add Digg AI 1000 as an opt-in source (#370)
* feat(digg): add Digg AI 1000 source module with cluster search and post enrichment - search_digg shells out to digg-pp-cli with --since 30d --agent - parse_digg_response normalizes clusters to last30days dict shape - enrich_with_top_posts attaches top-ranked X posts to top-K clusters - shutil.which gate plus subproc.run_with_timeout discipline matches bird_x.py / youtube_yt.py patterns 25 unit tests cover parse, age window, relevance, binary-missing fallback, timeout recovery, and partial enrichment failures. * feat(digg): wire Digg source into pipeline, normalize, signals, and render pipeline.py: - Import digg, add to MOCK_AVAILABLE_SOURCES, gate via shutil.which - Dispatch case calls search_digg + parse_digg_response, runs enrich_with_top_posts at default/deep depth - Mock fixture includes one enriched cluster + one bare cluster normalize.py: - _normalize_digg maps cluster dicts to SourceItem with container='Digg AI 1000' and metadata.posts pass-through signals.py: - SOURCE_QUALITY['digg'] = 0.85 (top tier alongside YouTube, reflecting Digg's curatorial layer) - ENGAGEMENT_WEIGHTS['digg'] balances postCount, uniqueAuthors, and the rank_score derived from Digg's curatorial position render.py: - SOURCE_LABELS['digg'] = 'Digg AI 1000' - _FOOTER_SOURCES adds '⛏️ Digg AI 1000' line after GitHub - ENGAGEMENT_DISPLAY mirrors footer keys - New _digg_posts_for + _format_digg_quote helpers emit inline '@handle via Digg AI 1000' quotes for clusters with attached X posts; both compact and full-dump renderers call them * feat(digg): polish per-item engagement display and progress label - ENGAGEMENT_DISPLAY for digg uses 'posts' / 'auth' to match the codebase abbreviation convention (HN: 'pts'/'cmt', X: 'rt'/'re') - Footer item word changes from 'story' to 'cluster' to dodge the pre-existing naive plural in _footer_line_for_source ('storys') and to match Digg's actual data model - ui.py SOURCE_COMPLETION_META adds digg with correct 'cluster'/ 'clusters' plural so 'Research complete' shows 'Digg: N clusters' * feat(digg): document Digg AI 1000 source in skill, README, and changelog - planner.py SOURCE_CAPABILITIES adds digg with discussion/social/link capabilities so the planner offers it through the standard fanout - SKILL.md ACTIVE_SOURCES_LIST gate includes 'which digg-pp-cli' check and the source list / available-sources line names digg as opt-in - README.md Sources table adds the Digg AI 1000 row with the activation gate so first-time readers see what they get - CHANGELOG.md Unreleased section calls out the source addition * fix(digg): enrich post-dedupe so brief survivors carry inline quotes Pipeline dispatch was attaching X posts to the top-3 items returned by search, but dedupe later picked different survivors when multiple clusters compared similar (common for trending topics). The brief ended up showing clusters with no posts attached even though enrichment ran successfully on positions 0-2. Move enrichment to _finalize_items_by_source. The new digg.enrich_source_items helper reads metadata['clusterUrlId'] and writes metadata['posts'] in place on the SourceItems that actually survive dedupe. Verified live on 'openclaw': 2 surviving clusters, both now carry real X-post quotes from @sama and @jeremyphoward attributed 'via Digg AI 1000'. Adds 3 unit tests covering survivor enrichment, non-digg skip, and clusterUrlId fallback to item_id. * test(digg): relax live off-topic test to check shape, not emptiness Digg's live search uses fuzzy/popularity fallback, so an impossible token can still return some loosely-related clusters. The contract the pipeline depends on is shape (results is always a list); token-overlap relevance handles the noise downstream. --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> |
||
|
Trevin Chow
|
b1773be8f3 |
feat(emit): --emit=html for shareable self-contained briefs (#332)
Adds a one-command shareable HTML mode to /last30days. The skill detects
HTML intent (explicit --emit=html / --emit:html / --html flag in
$ARGUMENTS, or natural-language asks like "give me a shareable brief",
"for Slack", "export as HTML"), runs the normal research + chat synthesis
flow, then saves a self-contained HTML file to
~/Documents/Last30Days/{topic}-brief.html. The synthesis appears in chat
as usual; the HTML is an additional artifact for sharing.
User experience:
/last30days OpenClaw --emit=html
/last30days OpenClaw, give me an HTML brief for Slack
Synthesis prints to chat. Last line of the response: "📎 Shareable brief
saved to ~/Documents/Last30Days/openclaw-brief.html". Open it, drag it
into a message, browser-print to PDF, email it.
Architecture:
- SKILL.md gets a small detection block (triggers + early exit +
MUST/MUST NOT rules + rationale) that points to a reference file.
- references/save-html-brief.md owns the implementation: capture the
synthesis verbatim into a temp file via heredoc, invoke the engine
with --emit=html --synthesis-file, save to disk, append the
confirmation line to chat.
- lib/render.py exposes render_for_html(report, synthesis_md=None) and
render_for_html_comparison(...) -- clean markdown for HTML
conversion. Omits debug file header, model-facing safety note, and
data quality warnings (those stay in engine stderr; recipients can't
act on them in a shared artifact).
- lib/html_render.py is a new module: ~200-line CSS template (dark
mode default, prefers-color-scheme switch, print stylesheet, mobile
breakpoint), stdlib-regex markdown-to-HTML converter, marker-based
META + engine-footer wrapping, PROSE_LABELS registry promoting plain
-text labels to <h2>, colophon builder.
- last30days.py adds --emit=html argparse choice and --synthesis-file
PATH flag (engine still callable directly without the skill in the
loop).
Design:
- Voice-led research brief, not corporate report. Inter + JetBrains
Mono via Google Fonts with full system fallbacks (no FOIT, works
offline). Brand purple #a855f7 (#7c3aed in light mode). Type ramp:
body 17px/400/muted, bold lead-in 17px/600/fg, h2 + .prose-label
20px/600/fg, monospace badge/meta/footer/colophon at 13-13.5px.
- 720px max-width, generous whitespace, no card layouts or shadows.
- Print stylesheet: light theme, A4 margins, [href]::after URL
footnotes, page-break-inside:avoid on the engine footer.
Templated (locked) shell:
- HTML5 boilerplate, Google Fonts <link> with preconnect, all CSS
inline.
- .badge / .meta / .engine-footer / .colophon containers.
Flexible (role-based):
- <h2> rendering covers BOTH plain ## headers (comparison mode per
LAW 4 exception) AND promoted prose labels via PROSE_LABELS
registry. Adding a new SKILL.md prose label is a one-line tuple
addition; no CSS or template changes.
- Marker-based engine boundaries (<!-- META: ... -->,
<!-- PASS-THROUGH FOOTER -->) survive the markdown converter and
get promoted post-conversion. Robust to engine output format
changes.
- Generic markdown-to-HTML for body content; future SKILL.md additions
(new sections, tables, blockquotes) render correctly without code
changes.
Tests: 30 new tests in tests/test_html_render.py covering snapshots
(rich/thin/comparison), CLI parsing, --synthesis-file end-to-end, prose
label promotion, warning exclusion from artifact, parseability via
html.parser, no-script self-containment.
No SKILL.md voice contract changes, no LAWs 1-8 changes, no new pip
dependencies, no JavaScript anywhere.
|
||
|
Ilia Alshanetsky
|
2acbf8a869 |
perf: batch store_findings, dedup source_items in O(1), remove dead code (#206)
1. N+1 queries in store.store_findings()
The old loop ran one SELECT per finding to check existence, then one
INSERT or UPDATE. 100 findings cost 200 serial SQLite roundtrips.
Now: one batch SELECT with WHERE source_url IN (...) builds a lookup
dict, then executemany() handles all inserts and updates. Query count
stays constant regardless of batch size. Benchmark on 500 findings:
~30ms to ~20ms; gap widens on slower storage.
2. O(n^2) source_items dedup in fusion.weighted_rrf()
Merging an item into an existing candidate ran any(existing.source ==
... for existing in candidate.source_items), linearly scanning a list
that grew with each merge. At 40 candidates with 20 source_items each,
fusion went quadratic. Now tracks (source, item_id) tuples in a
per-candidate set for O(1) lookup. The source_items list itself is
unchanged since other code iterates it.
3. Dead code removal
- providers.GeminiClient.ground_search() and .url_context_json(): zero
callers. Deleted.
- render._top_comment_excerpt(): zero callers. Deleted.
- env.is_reddit_available(): one-line wrapper around get_reddit_source.
Callers can check get_reddit_source(config) is not None directly.
|
||
|
Claire Novotny
|
72495c1c14 | Restructure as Codex plugin |