diff --git a/CHANGELOG.md b/CHANGELOG.md index 617b087..76102d5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [Unreleased] + +### Added + +- Add `--emit=html` for shareable, print-friendly HTML research briefs. + ## [3.1.1] - 2026-04-24 ### Fixed diff --git a/README.md b/README.md index dc45cd9..1f02555 100644 --- a/README.md +++ b/README.md @@ -96,6 +96,28 @@ The synthesis ranks by what real people actually engaged with. Social relevancy, ## What v3 Changed +### Shareable HTML briefs + +Ask for an HTML brief and the skill saves a self-contained, dark-mode, print-friendly file you can drop into Slack, email, or Notion. No raw markdown leaks. Inline CSS, system-font fallbacks behind Inter and JetBrains Mono. No JavaScript. Works offline. + +``` +/last30days OpenClaw --emit=html +``` + +or just ask in plain language: + +``` +/last30days OpenClaw, give me a shareable HTML brief +/last30days Cursor IDE for slack +/last30days Anthropic earnings export as html +``` + +The skill emits the synthesis in chat as usual AND saves a brief to `${LAST30DAYS_MEMORY_DIR}/{topic}-brief.html` (defaults to `~/Documents/Last30Days/`). The chat response ends with the file path so you can `open` it or drag it into a message. + +What's in the file: badge, inline metadata line, the model's synthesis verbatim with all citations, the engine footer (โœ… All agents reported back! tree), and a colophon noting the topic + how to re-run. Data quality warnings (degraded run, thin evidence, etc.) stay in the engine's stderr logs; they never leak into the shareable artifact. + +For direct CLI use without the model in the loop, the engine also accepts `--synthesis-file PATH` to convert any markdown synthesis to HTML. + ### Intelligent search: the killer feature The v3 engine doesn't just search for your topic. It figures out *where* to search before the search begins. Type "OpenClaw" and the engine resolves @steipete (Peter Steinberger, the creator), r/openclaw, r/ClaudeCode, and the right YouTube channels and TikTok hashtags - all via a new Python pre-research brain built by [@j-sperling](https://github.com/j-sperling). The old engine searched keywords. The new engine understands your topic first, then searches the right people and communities. diff --git a/skills/last30days/SKILL.md b/skills/last30days/SKILL.md index e534415..a806dd4 100644 --- a/skills/last30days/SKILL.md +++ b/skills/last30days/SKILL.md @@ -1507,6 +1507,33 @@ Close with `I have all the links to the {N} {source list} I pulled from. Just as --- +## SHAREABLE HTML BRIEF (when the user asked for one) + +**This section fires if EITHER trigger is true:** + +- `$ARGUMENTS` contains `--emit=html`, `--emit:html`, or `--html` as a flag +- The user's natural-language request asks for an HTML brief, shareable doc, or file for sharing (Slack, email, Notion, "export as HTML", etc). Use your judgment for phrasing variants. + +**If neither trigger fires, skip this entire section and proceed to WAIT FOR USER'S RESPONSE.** No HTML save flow, no reference read needed. + +**When triggered, you MUST:** + +- Read `references/save-html-brief.md` BEFORE proceeding to WAIT FOR USER'S RESPONSE +- Follow that file's instructions exactly - it is the canonical source for the save flow +- Append the confirmation line (`๐Ÿ“Ž Shareable brief saved to `) to your already-emitted chat response + +**You MUST NOT:** + +- Improvise the HTML save flow from memory or from instructions you've seen before +- Skip the reference read because the steps "look familiar" +- Save to a different path than the reference specifies +- Add data quality warnings, debug headers, or safety notes to the saved HTML +- Re-research the topic for the HTML render - the engine cache covers the second invocation + +**Why the directive is forceful:** the reference file is the only source of truth for the save flow. Skipping it produces broken artifacts - wrong path conventions, missing synthesis content, leaked engine debug output, or warnings that don't belong in shareable docs. + +--- + ## WAIT FOR USER'S RESPONSE **STOP and wait** for the user to respond. Do NOT call any tools after displaying the invitation. Do NOT append a `Sources:` section (see override above - WebSearch's mandate does not apply here). The research script already saved raw data to `LAST30DAYS_MEMORY_DIR` (defaults to `~/Documents/Last30Days`) via `--save-dir`. diff --git a/skills/last30days/references/save-html-brief.md b/skills/last30days/references/save-html-brief.md new file mode 100644 index 0000000..813593d --- /dev/null +++ b/skills/last30days/references/save-html-brief.md @@ -0,0 +1,90 @@ +# Save shareable HTML brief + +This reference file is loaded by the main `SKILL.md` when the user asked for an HTML brief (either explicitly via `--emit=html` / `--emit:html` / `--html`, or in natural language - "give me a shareable HTML brief", "for Slack", "for Notion", "export as HTML", etc.). The detection happens in `SKILL.md` so that the common no-HTML path stays short; the implementation lives here. + +The contract: the synthesis still appears in chat as the primary output. The HTML is an additional artifact saved to disk for sharing. Both happen in the same turn. + +## When to fire this flow + +- After you have already emitted the full chat response: badge, "What I learned:" (or comparison title), bold-lead-in paragraphs with citations, KEY PATTERNS list, engine footer pass-through, invitation block. +- BEFORE the WAIT FOR USER'S RESPONSE pause. +- ONLY if the user asked. Do NOT save HTML when the user didn't ask for it. + +## How to fire it + +```bash +# 1. Write your synthesis prose VERBATIM to a temp file. The synthesis is the +# "What I learned:" prose label, the bold-lead-in paragraphs with their +# inline citations as you wrote them in chat, and the "KEY PATTERNS from +# the research:" numbered list. Do NOT include the badge or the engine +# footer in the temp file - the engine adds those when it renders the HTML. +# Use the EXACT text you just wrote in chat. Do not paraphrase, do not +# summarize, do not reorder. The HTML must read identically to the chat +# response in voice and citations. +SYNTHESIS_FILE="/tmp/last30days-synthesis-${CLAUDE_SESSION_ID}.md" +cat > "$SYNTHESIS_FILE" <<'SYNTHESIS_EOF' +What I learned: + +**{First headline}** - {body with [name](url) inline citations} + +**{Second headline}** - {body} + +**{Third headline}** - {body} + +KEY PATTERNS from the research: +1. {pattern} - per [@handle](url) +2. {pattern} - per [r/sub](url) +3. {pattern} - per [@handle](url) +SYNTHESIS_EOF + +# 2. Convert the synthesis to a self-contained HTML file via the engine. +# The engine reuses the cache from your earlier engine run (same topic +# + plan), so this second invocation is typically <1s on cache hit. +SLUG=$(echo "$TOPIC" | tr '[:upper:]' '[:lower:]' | tr -cs 'a-z0-9' '-' | sed 's/^-//;s/-$//') +HTML_PATH="${LAST30DAYS_MEMORY_DIR}/${SLUG}-brief.html" +"${LAST30DAYS_PYTHON}" "${SKILL_ROOT}/scripts/last30days.py" "${TOPIC}" \ + --emit=html \ + --synthesis-file "$SYNTHESIS_FILE" \ + > "$HTML_PATH" + +# 3. Append ONE line to your already-emitted chat response, after the +# invitation block. Use a paperclip emoji as a visible signal that an +# artifact was produced: +echo "๐Ÿ“Ž Shareable brief saved to $HTML_PATH" +``` + +## What ends up in the HTML file + +The engine's `--emit=html` renderer combines: + +- The badge (`๐ŸŒ last30days vX.Y.Z ยท synced YYYY-MM-DD`) at the top +- A single inline metadata line (`{date range} ยท {active sources}`) below the badge +- Your synthesis verbatim, with prose labels promoted to `

` and bold lead-ins preserved +- All `[name](url)` citations rendered as `` tags +- The engine footer (`โœ… All agents reported back!` tree) preserved verbatim in monospace +- A colophon with the topic and a re-run hint + +The renderer strips engine-internal noise that doesn't belong in a shareable artifact: the `# last30days vX.Y.Z: TOPIC` debug file header, the model-facing `> Safety note:` blockquote, and the `I'm now an expert on X` invitation block. Data quality warnings (degraded run, thin evidence, etc.) stay in the engine's stderr logs - they never leak into the share-ready file. + +## Comparison mode + +Same flow when the topic is `X vs Y` (or `X vs Y vs Z`). The engine routes through `render_for_html_comparison` internally; you don't need to do anything special. The synthesis temp file should still contain the comparison-shaped synthesis you wrote in chat (`## Quick Verdict`, `## {Entity}` per entity, `## Head-to-Head` table, `## The Bottom Line`, `## The emerging stack` per LAW 4 comparison exception). + +## Follow-up turn + +If the user runs `/last30days OpenClaw` normally, sees the synthesis in chat, and THEN says "save that as HTML" or "give me a shareable version" in a follow-up turn, do the same save flow on the synthesis you wrote in the previous turn. Do not re-research; the synthesis is already in the conversation history. Just write it to the temp file and call the engine with `--emit=html --synthesis-file`. + +## What NOT to do + +- Do NOT save HTML if the user didn't ask. The sparse mode (no synthesis) produces a thin file; not useful as a shareable. +- Do NOT add content to the temp file beyond your synthesis prose. The badge / footer / colophon come from the engine. +- Do NOT change the file path convention. `${LAST30DAYS_MEMORY_DIR}/${SLUG}-brief.html` is the canonical location. +- Do NOT silently overwrite an existing file without telling the user. If `$HTML_PATH` already exists from a prior run, the engine will pick a date-suffixed name (`{slug}-brief-YYYY-MM-DD.html`) automatically; just print whichever path the redirect produced. +- Do NOT include the data quality warning text in the temp file or in your final chat line. Warnings are an engine-stderr concern, not an artifact concern. + +## Edge cases + +- **Topic with shell-special characters** (quotes, ampersands): the temp filename uses a slugified version, but the engine receives the raw topic. The `cat <<'SYNTHESIS_EOF'` quoted heredoc form handles arbitrary content without expansion. Your synthesis text can include any character. +- **Very long synthesis**: no upper bound. The engine handles long markdown bodies. Just paste verbatim. +- **Synthesis with images or non-ASCII**: emoji and Unicode pass through. Image tags pass through as raw HTML; the renderer doesn't transform them. If you didn't include images in chat, don't add them here. +- **No `${LAST30DAYS_MEMORY_DIR}` set**: defaults to `~/Documents/Last30Days/` per the SKILL.md `Configuration` section. diff --git a/skills/last30days/scripts/last30days.py b/skills/last30days/scripts/last30days.py index b99245a..d8cb7cf 100644 --- a/skills/last30days/scripts/last30days.py +++ b/skills/last30days/scripts/last30days.py @@ -41,7 +41,7 @@ if os.name == "nt": SCRIPT_DIR = Path(__file__).parent.resolve() sys.path.insert(0, str(SCRIPT_DIR)) -from lib import env, pipeline, render, schema, ui +from lib import env, html_render, pipeline, render, schema, ui _child_pids: set[int] = set() _child_pids_lock = threading.Lock() @@ -91,30 +91,46 @@ def slugify(value: str) -> str: return slug or "last30days" -def save_output(report: schema.Report, emit: str, save_dir: str, suffix: str = "") -> Path: +def save_output( + report: schema.Report, + emit: str, + save_dir: str, + suffix: str = "", + synthesis_md: str | None = None, +) -> Path: from datetime import datetime path = Path(save_dir).expanduser().resolve() path.mkdir(parents=True, exist_ok=True) slug = slugify(report.topic) - extension = "json" if emit == "json" else "md" + extension = "json" if emit == "json" else "html" if emit == "html" else "md" + raw_label = "raw-html" if emit == "html" else "raw" suffix_part = f"-{suffix}" if suffix else "" - out_path = path / f"{slug}-raw{suffix_part}.{extension}" + out_path = path / f"{slug}-{raw_label}{suffix_part}.{extension}" if out_path.exists(): - out_path = path / f"{slug}-raw{suffix_part}-{datetime.now().strftime('%Y-%m-%d')}.{extension}" - # Always save the FULL dump to disk (all items, all sources, transcripts). - # Claude sees compact clusters via --emit=compact on stdout. - # The saved file is the complete debug artifact. - if emit == "json": - content = emit_output(report, emit) + out_path = path / f"{slug}-{raw_label}{suffix_part}-{datetime.now().strftime('%Y-%m-%d')}.{extension}" + # Markdown saves keep the complete debug artifact. JSON and HTML preserve + # their requested wire format so file extensions match their content. + if emit in {"json", "html"}: + content = emit_output(report, emit, synthesis_md=synthesis_md) else: content = render.render_full(report) out_path.write_text(content, encoding="utf-8") return out_path -def emit_output(report: schema.Report, emit: str, fun_level: str = "medium", save_path: str | None = None) -> str: +def emit_output( + report: schema.Report, + emit: str, + fun_level: str = "medium", + save_path: str | None = None, + synthesis_md: str | None = None, +) -> str: if emit == "json": return json.dumps(schema.to_dict(report), indent=2, sort_keys=True) + if emit == "html": + return html_render.render_html( + report, fun_level=fun_level, save_path=save_path, synthesis_md=synthesis_md, + ) if emit in {"compact", "md"}: return render.render_compact(report, fun_level=fun_level, save_path=save_path) if emit == "context": @@ -127,6 +143,7 @@ def emit_comparison_output( emit: str, fun_level: str = "medium", save_path: str | None = None, + synthesis_md: str | None = None, ) -> str: if emit == "json": payload = { @@ -138,6 +155,13 @@ def emit_comparison_output( ], } return json.dumps(payload, indent=2, sort_keys=True) + if emit == "html": + return html_render.render_html_comparison( + entity_reports, + fun_level=fun_level, + save_path=save_path, + synthesis_md=synthesis_md, + ) if emit in {"compact", "md"}: return render.render_comparison_multi( entity_reports, fun_level=fun_level, save_path=save_path, @@ -156,9 +180,10 @@ def compute_save_path_display(save_dir: str, topic: str, suffix: str, emit: str) from pathlib import Path as _Path path = _Path(save_dir).expanduser().resolve() slug = slugify(topic) - extension = "json" if emit == "json" else "md" + extension = "json" if emit == "json" else "html" if emit == "html" else "md" + raw_label = "raw-html" if emit == "html" else "raw" suffix_part = f"-{suffix}" if suffix else "" - raw = path / f"{slug}-raw{suffix_part}.{extension}" + raw = path / f"{slug}-{raw_label}{suffix_part}.{extension}" try: home = _Path.home().resolve() relative = raw.relative_to(home) @@ -167,6 +192,14 @@ def compute_save_path_display(save_dir: str, topic: str, suffix: str, emit: str) return str(raw) +def read_synthesis_file(path: str) -> str: + try: + return Path(path).expanduser().read_text(encoding="utf-8") + except OSError as exc: + sys.stderr.write(f"[last30days] Cannot read --synthesis-file: {exc}\n") + raise SystemExit(2) + + def persist_report(report: schema.Report) -> dict[str, int]: import store @@ -193,7 +226,7 @@ def persist_report(report: schema.Report) -> dict[str, int]: def build_parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser(description="Research a topic across live social, market, and grounded web sources.") parser.add_argument("topic", nargs="*", help="Research topic") - parser.add_argument("--emit", default="compact", choices=["compact", "json", "context", "md"]) + parser.add_argument("--emit", default="compact", choices=["compact", "json", "context", "md", "html"]) parser.add_argument("--search", help="Comma-separated source list") parser.add_argument("--quick", action="store_true", help="Lower-latency retrieval profile") parser.add_argument("--deep", action="store_true", help="Higher-recall retrieval profile") @@ -201,6 +234,7 @@ def build_parser() -> argparse.ArgumentParser: parser.add_argument("--mock", action="store_true", help="Use mock retrieval fixtures") parser.add_argument("--diagnose", action="store_true", help="Print provider and source availability") parser.add_argument("--save-dir", help="Optional directory for saving the rendered output") + parser.add_argument("--synthesis-file", help="Markdown synthesis to embed in --emit=html output") parser.add_argument("--store", action="store_true", help="Persist ranked findings to the SQLite research store") parser.add_argument("--x-handle", help="X handle for targeted supplemental search") parser.add_argument("--x-related", help="Comma-separated related X handles (searched with lower weight)") @@ -537,6 +571,13 @@ def main() -> int: parser.print_usage(sys.stderr) return 2 + synthesis_md = None + if args.synthesis_file: + if args.emit == "html": + synthesis_md = read_synthesis_file(args.synthesis_file) + else: + sys.stderr.write("[last30days] Warning: --synthesis-file is only used with --emit=html; ignoring.\n") + if not os.environ.get("LAST30DAYS_SKIP_PREFLIGHT"): from lib import preflight refuse_msg = preflight.check_class_1_trap(topic) @@ -854,15 +895,29 @@ def main() -> int: if entity_reports: rendered = emit_comparison_output( - entity_reports, args.emit, fun_level=fun_level, save_path=footer_save_path, + entity_reports, + args.emit, + fun_level=fun_level, + save_path=footer_save_path, + synthesis_md=synthesis_md, ) else: rendered = emit_output( - report, args.emit, fun_level=fun_level, save_path=footer_save_path, + report, + args.emit, + fun_level=fun_level, + save_path=footer_save_path, + synthesis_md=synthesis_md, ) if args.save_dir: # Save the main topic's raw file (single-entity or comparison main). - save_path = save_output(report, args.emit, args.save_dir, suffix=args.save_suffix or "") + save_path = save_output( + report, + args.emit, + args.save_dir, + suffix=args.save_suffix or "", + synthesis_md=synthesis_md, + ) sys.stderr.write(f"[last30days] Saved output to {save_path}\n") # Competitor / vs-mode: also save a per-entity raw file for each peer. # Matches historical vs-mode behavior (N passes โ†’ N save files). @@ -871,6 +926,7 @@ def main() -> int: peer_path = save_output( entity_report, args.emit, args.save_dir, suffix=args.save_suffix or "", + synthesis_md=synthesis_md, ) sys.stderr.write(f"[last30days] Saved output to {peer_path}\n") sys.stderr.flush() diff --git a/skills/last30days/scripts/lib/html_render.py b/skills/last30days/scripts/lib/html_render.py new file mode 100644 index 0000000..f80b7cb --- /dev/null +++ b/skills/last30days/scripts/lib/html_render.py @@ -0,0 +1,674 @@ +"""HTML rendering for shareable last30days reports.""" + +from __future__ import annotations + +import html +import re +from datetime import date + +from . import render, schema + + +PROSE_LABELS = [ + ("What I learned:", "What I learned"), + ("KEY PATTERNS from the research:", "Key patterns from the research"), +] + +INVITATION_PATTERN = re.compile(r"^---\nI'm now an expert.*?Just ask\.$", re.MULTILINE | re.DOTALL) +EVIDENCE_BLOCK_PATTERN = re.compile(r"", re.DOTALL) +PASS_THROUGH_FOOTER_PATTERN = re.compile(r"\n(.*?)", re.DOTALL) +CANONICAL_BOUNDARY_PATTERN = re.compile(r"\n?---\n# END OF last30days CANONICAL OUTPUT.*$", re.DOTALL) +# render_for_html emits metadata as so it survives the +# markdown converter (which escapes raw HTML inside paragraphs). Promoted to +# a styled
after conversion. +META_MARKER_PATTERN = re.compile(r"") + +CSS = """ +:root { + --bg: #0e0e10; + --bg-elev: #18181b; + --fg: #fafafa; + --fg-muted: #a1a1aa; + --fg-subtle: #71717a; + --accent: #a855f7; + --accent-soft: #c4b5fd; + --border: #27272a; + --code-bg: #1a1a1d; + --max-w: 720px; +} + +@media (prefers-color-scheme: light) { + :root { + --bg: #ffffff; + --bg-elev: #fafafa; + --fg: #18181b; + --fg-muted: #52525b; + --fg-subtle: #71717a; + --accent: #7c3aed; + --accent-soft: #6d28d9; + --border: #e4e4e7; + --code-bg: #f4f4f5; + } +} + +* { box-sizing: border-box; } + +html, body { + margin: 0; + padding: 0; + background: var(--bg); + color: var(--fg); + font-family: 'Inter', -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, system-ui, sans-serif; + font-size: 17px; + line-height: 1.65; + -webkit-font-smoothing: antialiased; + -moz-osx-font-smoothing: grayscale; + text-rendering: optimizeLegibility; +} + +body { + max-width: var(--max-w); + margin: 0 auto; + padding: 4rem 1.5rem 6rem; +} + +.badge { + display: inline-block; + padding: 0.4rem 0.85rem; + margin-bottom: 2.5rem; + background: var(--bg-elev); + border: 1px solid var(--border); + border-radius: 999px; + font-family: 'JetBrains Mono', ui-monospace, 'SF Mono', 'Cascadia Code', Menlo, Consolas, monospace; + font-size: 13px; + font-weight: 500; + color: var(--fg-muted); + letter-spacing: 0; +} + +.badge .accent { color: var(--accent); } + +.meta { + margin: -1.5rem 0 2.5rem; + color: var(--fg-subtle); + font-family: 'JetBrains Mono', ui-monospace, 'SF Mono', 'Cascadia Code', Menlo, Consolas, monospace; + font-size: 13px; + letter-spacing: 0.01em; +} + +h1 { + margin: 0 0 1.5rem; + color: var(--fg); + font-size: 30px; + font-weight: 700; + line-height: 1.2; + letter-spacing: 0; +} + +h2, +.prose-label { + margin: 2.75rem 0 1.25rem; + color: var(--fg); + font-size: 20px; + font-weight: 600; + line-height: 1.35; + letter-spacing: 0; +} + +.badge + h2, +.badge + .prose-label { margin-top: 0.5rem; } + +h3 { + margin: 2rem 0 0.85rem; + color: var(--fg); + font-size: 17px; + font-weight: 600; + line-height: 1.4; + letter-spacing: 0; +} + +p { + margin: 0 0 1.4rem; + color: var(--fg-muted); +} + +p strong, +li strong, +td strong { + color: var(--fg); + font-weight: 600; +} + +a { + color: var(--accent); + text-decoration: none; + border-bottom: 1px solid transparent; + transition: border-color 0.15s ease; +} + +a:hover { border-bottom-color: var(--accent); } + +ul, +ol { + margin: 0 0 1.6rem; + padding-left: 1.5rem; + color: var(--fg-muted); +} + +li { + margin: 0.6rem 0; + padding-left: 0.4rem; +} + +li::marker { + color: var(--accent); + font-weight: 600; +} + +blockquote { + margin: 1.5rem 0; + padding-left: 1rem; + border-left: 3px solid var(--accent); + color: var(--fg-muted); +} + +hr { + margin: 2.5rem 0; + border: 0; + border-top: 1px solid var(--border); +} + +code { + font-family: 'JetBrains Mono', ui-monospace, 'SF Mono', 'Cascadia Code', Menlo, Consolas, monospace; + font-size: 0.92em; + background: var(--code-bg); + padding: 0.15rem 0.4rem; + border-radius: 4px; + color: var(--accent-soft); +} + +pre { + margin: 1.4rem 0; + background: var(--code-bg); + border: 1px solid var(--border); + border-radius: 8px; + padding: 1rem 1.25rem; + overflow-x: auto; + font-size: 14px; + line-height: 1.6; +} + +pre code { + background: none; + padding: 0; + color: var(--fg); +} + +table { + width: 100%; + border-collapse: collapse; + margin: 1.5rem 0; + font-size: 15px; +} + +th, +td { + text-align: left; + padding: 0.75rem 1rem; + border-bottom: 1px solid var(--border); + vertical-align: top; +} + +th { + color: var(--fg-muted); + font-weight: 600; + font-size: 13px; + letter-spacing: 0; + text-transform: uppercase; +} + +td { color: var(--fg-muted); } +td:first-child { color: var(--fg); font-weight: 500; } + +.engine-footer { + margin: 3rem 0 2.5rem; + padding: 1.25rem 1.5rem; + background: var(--bg-elev); + border: 1px solid var(--border); + border-radius: 8px; + color: var(--fg-muted); +} + +.engine-footer pre { + margin: 0; + padding: 0; + background: transparent; + border: 0; + border-radius: 0; + font-family: 'JetBrains Mono', ui-monospace, 'SF Mono', 'Cascadia Code', Menlo, Consolas, monospace; + font-size: 13.5px; + font-weight: 400; + line-height: 1.75; + color: inherit; + white-space: pre-wrap; + word-break: break-word; +} + +.colophon { + margin-top: 4rem; + padding-top: 2rem; + border-top: 1px solid var(--border); + color: var(--fg-subtle); + font-size: 13px; + font-family: 'JetBrains Mono', ui-monospace, 'SF Mono', 'Cascadia Code', Menlo, Consolas, monospace; + line-height: 1.7; +} + +.colophon .rerun { + display: inline-block; + padding: 0.15rem 0.5rem; + margin-left: 0.25rem; + background: var(--code-bg); + border-radius: 4px; + color: var(--accent-soft); + font-size: 0.95em; +} + +@media print { + :root { + --bg: #ffffff; + --bg-elev: #f5f5f5; + --fg: #000000; + --fg-muted: #1f2937; + --fg-subtle: #4b5563; + --accent: #6d28d9; + --accent-soft: #6d28d9; + --border: #d4d4d8; + --code-bg: #f4f4f5; + } + + @page { size: A4; margin: 1.5cm 2cm; } + + body { + max-width: none; + padding: 0; + font-size: 11pt; + } + + a { + color: inherit; + border-bottom: 0; + text-decoration: underline; + } + + a[href]::after { + content: " (" attr(href) ")"; + font-size: 0.85em; + color: var(--fg-subtle); + } + + .engine-footer { page-break-inside: avoid; } +} + +@media (max-width: 600px) { + body { + padding: 2.5rem 1.25rem 4rem; + font-size: 16px; + } + + h1 { font-size: 25px; } + .badge { font-size: 12px; } + th, td { padding: 0.65rem 0.5rem; } +} +""".strip() + +HTML_TEMPLATE = """ + + + + +last30days ยท __TITLE__ + + + + + + +__BODY__ +__COLOPHON__ + + +""" + + +def render_html( + report: schema.Report, + *, + fun_level: str = "medium", + save_path: str | None = None, + synthesis_md: str | None = None, +) -> str: + _ = fun_level + md = render.render_for_html(report, synthesis_md=synthesis_md, save_path=save_path) + md = _strip_evidence_block(md) + md = _strip_invitation(md) + md = _strip_canonical_boundary(md) + md = _promote_prose_labels(md) + body = _markdown_to_html(md) + body = _wrap_engine_footer(body) + body = _promote_meta_marker(body) + colophon = _build_colophon(report) + return _wrap_in_template(body, colophon, report.topic) + + +def render_html_comparison( + entity_reports: list[tuple[str, schema.Report]], + *, + fun_level: str = "medium", + save_path: str | None = None, + synthesis_md: str | None = None, +) -> str: + _ = fun_level + md = render.render_for_html_comparison( + entity_reports, synthesis_md=synthesis_md, save_path=save_path, + ) + md = _strip_evidence_block(md) + md = _strip_invitation(md) + md = _strip_canonical_boundary(md) + md = _promote_prose_labels(md) + body = _markdown_to_html(md) + body = _wrap_engine_footer(body) + body = _promote_meta_marker(body) + topic = " vs ".join(label for label, _ in entity_reports) + colophon = _build_colophon(entity_reports[0][1], topic=topic) + return _wrap_in_template(body, colophon, topic) + + +def _strip_evidence_block(md: str) -> str: + return EVIDENCE_BLOCK_PATTERN.sub("", md) + + +def _strip_invitation(md: str) -> str: + return INVITATION_PATTERN.sub("", md) + + +def _strip_canonical_boundary(md: str) -> str: + return CANONICAL_BOUNDARY_PATTERN.sub("", md) + + +def _promote_prose_labels(md: str) -> str: + for source, normalized in PROSE_LABELS: + md = re.sub( + rf"^{re.escape(source)}$", + f"## {normalized}", + md, + flags=re.MULTILINE, + ) + return md + + +def _markdown_to_html(md: str) -> str: + md, footers = _protect_engine_footers(md) + global _ENGINE_FOOTER_STORE + _ENGINE_FOOTER_STORE = footers + # Strip HTML comments EXCEPT preserved markers used for post-processing + # (META is promoted to
after markdown conversion). + md = re.sub(r"", "", md, flags=re.DOTALL) + lines = md.splitlines() + out: list[str] = [] + paragraph: list[str] = [] + list_type: str | None = None + in_code = False + code_lines: list[str] = [] + index = 0 + + def flush_paragraph() -> None: + nonlocal paragraph + if paragraph: + text = " ".join(part.strip() for part in paragraph).strip() + if text: + out.append(f"

{_inline_markdown(text)}

") + paragraph = [] + + def close_list() -> None: + nonlocal list_type + if list_type: + out.append(f"") + list_type = None + + while index < len(lines): + line = lines[index] + stripped = line.strip() + + if in_code: + if stripped.startswith("```"): + out.append(f"
{html.escape(chr(10).join(code_lines))}
") + code_lines = [] + in_code = False + else: + code_lines.append(line) + index += 1 + continue + + if stripped.startswith("```"): + flush_paragraph() + close_list() + in_code = True + code_lines = [] + index += 1 + continue + + if stripped in footers: + flush_paragraph() + close_list() + out.append(stripped) + index += 1 + continue + + if not stripped: + flush_paragraph() + close_list() + index += 1 + continue + + if stripped == "---": + flush_paragraph() + close_list() + out.append("
") + index += 1 + continue + + if index + 1 < len(lines) and _is_table_row(stripped) and _is_table_separator(lines[index + 1].strip()): + flush_paragraph() + close_list() + table_lines = [stripped] + index += 2 + while index < len(lines) and _is_table_row(lines[index].strip()): + table_lines.append(lines[index].strip()) + index += 1 + out.append(_render_table(table_lines)) + continue + + heading = re.match(r"^(#{1,4})\s+(.+)$", stripped) + if heading: + flush_paragraph() + close_list() + level = min(len(heading.group(1)), 3) + out.append(f"{_inline_markdown(heading.group(2))}") + index += 1 + continue + + if stripped.startswith(">"): + flush_paragraph() + close_list() + quote_lines = [] + while index < len(lines) and lines[index].strip().startswith(">"): + quote_lines.append(lines[index].strip().lstrip(">").strip()) + index += 1 + out.append(f"
{_inline_markdown(' '.join(quote_lines))}
") + continue + + unordered = re.match(r"^[-*]\s+(.+)$", stripped) + ordered = re.match(r"^\d+[.)]\s+(.+)$", stripped) + if unordered or ordered: + flush_paragraph() + next_type = "ul" if unordered else "ol" + if list_type != next_type: + close_list() + out.append(f"<{next_type}>") + list_type = next_type + item = unordered.group(1) if unordered else ordered.group(1) + out.append(f"
  • {_inline_markdown(item)}
  • ") + index += 1 + continue + + if stripped.startswith("๐ŸŒ last30days"): + flush_paragraph() + close_list() + badge_text = _inline_markdown(stripped.removeprefix("๐ŸŒ").strip()) + out.append(f'
    ๐ŸŒ {badge_text}
    ') + index += 1 + continue + + paragraph.append(line) + index += 1 + + if in_code: + out.append(f"
    {html.escape(chr(10).join(code_lines))}
    ") + flush_paragraph() + close_list() + return "\n".join(out).strip() + + +def _protect_engine_footers(md: str) -> tuple[str, dict[str, str]]: + footers: dict[str, str] = {} + + def replace(match: re.Match[str]) -> str: + token = f"__LAST30DAYS_ENGINE_FOOTER_{len(footers)}__" + footers[token] = match.group(1).strip("\n") + return f"\n{token}\n" + + return PASS_THROUGH_FOOTER_PATTERN.sub(replace, md), footers + + +def _wrap_engine_footer(body: str) -> str: + def replace(match: re.Match[str]) -> str: + footer = html.escape(_ENGINE_FOOTER_STORE.get(match.group(0), ""), quote=False) + return f'' + + return re.sub( + r"__LAST30DAYS_ENGINE_FOOTER_\d+__", + replace, + body, + ) + + +def _promote_meta_marker(body: str) -> str: + """Promote ```` markers into a styled ``
    ``. + + The marker is preserved through the comment-strip pass (see + _markdown_to_html exemption) but the markdown converter wraps it in + ``

    `` and HTML-escapes the angle brackets. After conversion the body + contains shapes like: +

    <!-- META: TEXT -->

    +

    (when not escaped) + Both collapse to ``
    TEXT
    ``. + """ + def replace(match: re.Match[str]) -> str: + text = match.group(1).strip() + return f'
    {text}
    ' + + # Escaped form (most common after markdown conversion) + body = re.sub( + r"

    \s*<!--\s*META:\s*(.*?)\s*-->\s*

    ", + replace, + body, + ) + body = re.sub(r"<!--\s*META:\s*(.*?)\s*-->", replace, body) + # Unescaped form (paranoid fallback) + body = re.sub(r"

    \s*\s*

    ", replace, body) + body = re.sub(r"", replace, body) + return body + + +_ENGINE_FOOTER_STORE: dict[str, str] = {} + + +def _inline_markdown(text: str) -> str: + escaped = html.escape(text, quote=True) + code_tokens: dict[str, str] = {} + + def code_replace(match: re.Match[str]) -> str: + token = f"__CODE_{len(code_tokens)}__" + code_tokens[token] = f"{match.group(1)}" + return token + + escaped = re.sub(r"`([^`]+)`", code_replace, escaped) + escaped = re.sub(r"\*\*([^*]+)\*\*", r"\1", escaped) + escaped = re.sub( + r"\[([^\]]+)\]\(([^)\s]+)\)", + r'
    \1', + escaped, + ) + for token, value in code_tokens.items(): + escaped = escaped.replace(token, value) + return escaped + + +def _is_table_row(line: str) -> bool: + return "|" in line and len(_split_table_cells(line)) >= 2 + + +def _is_table_separator(line: str) -> bool: + cells = _split_table_cells(line) + return bool(cells) and all(re.fullmatch(r":?-{3,}:?", cell.strip()) for cell in cells) + + +def _split_table_cells(line: str) -> list[str]: + return [cell.strip() for cell in line.strip().strip("|").split("|")] + + +def _render_table(rows: list[str]) -> str: + header = _split_table_cells(rows[0]) + body_rows = [_split_table_cells(row) for row in rows[1:]] + out = ["", "", ""] + out.extend(f"" for cell in header) + out.extend(["", "", ""]) + for row in body_rows: + out.append("") + out.extend(f"" for cell in row) + out.append("") + out.extend(["", "
    {_inline_markdown(cell)}
    {_inline_markdown(cell)}
    "]) + return "\n".join(out) + + +def _build_colophon(report: schema.Report, *, topic: str | None = None) -> str: + display_topic = topic or report.topic + generated = _generated_date(report) + version = render._skill_version() + escaped_topic = html.escape(display_topic) + rerun = html.escape(f"/last30days {display_topic}") + return ( + '
    \n' + f" Generated {generated} by /last30days v{html.escape(version)} ยท topic: {escaped_topic}
    \n" + f' Re-run for fresh data: {rerun}\n' + "
    " + ) + + +def _generated_date(report: schema.Report) -> str: + if report.generated_at: + return report.generated_at[:10] + return date.today().strftime("%Y-%m-%d") + + +def _wrap_in_template(body: str, colophon: str, title: str) -> str: + return ( + HTML_TEMPLATE + .replace("__TITLE__", html.escape(title)) + .replace("__CSS__", CSS) + .replace("__BODY__", body) + .replace("__COLOPHON__", colophon) + ) diff --git a/skills/last30days/scripts/lib/render.py b/skills/last30days/scripts/lib/render.py index 3614979..daaf8de 100644 --- a/skills/last30days/scripts/lib/render.py +++ b/skills/last30days/scripts/lib/render.py @@ -171,6 +171,168 @@ def render_compact(report: schema.Report, cluster_limit: int = 8, fun_level: str return "\n".join(lines).strip() + "\n" +def render_for_html( + report: schema.Report, + synthesis_md: str | None = None, + *, + save_path: str | None = None, +) -> str: + """Render markdown intended for shareable HTML conversion. + + This output keeps the public badge, compact source/date metadata, an + optional one-line data quality note, optional synthesized brief markdown, + and the engine footer. It deliberately omits the debug file header, + model-facing safety note, and evidence scratchpad emitted by + render_compact(). + + When synthesis_md is None, the body is intentionally sparse: badge, + metadata, optional data quality note, and engine footer only. + """ + lines = [ + *_render_badge(), + *_render_html_metadata(report), + ] + if synthesis_md: + lines.extend(["", synthesis_md.strip()]) + # Data quality warnings are NOT rendered into the HTML artifact. The HTML + # is meant to be shared (Slack, email, Notion); recipients haven't asked + # for technical commentary about how the run was produced. Generators see + # the same warnings via collect_html_warnings() routed to stderr by the + # CLI, so they can fix quality issues before sharing. + _append_html_footer(lines, report, save_path) + return "\n".join(lines).strip() + "\n" + + +def render_for_html_comparison( + entity_reports: list[tuple[str, schema.Report]], + synthesis_md: str | None = None, + *, + save_path: str | None = None, +) -> str: + """Render comparison markdown intended for shareable HTML conversion. + + Same semantics as render_for_html(), but metadata and data quality notes + are aggregated across the compared entities. + """ + if not entity_reports: + raise ValueError("render_for_html_comparison requires at least one report") + + entities = [label for label, _ in entity_reports] + main_report = entity_reports[0][1] + meta = ( + f"" + ) + lines = [ + *_render_badge(), + meta, + ] + if synthesis_md: + lines.extend(["", synthesis_md.strip()]) + # Comparison data quality notes also go to stderr, not into the artifact. + _append_html_footer(lines, main_report, save_path) + return "\n".join(lines).strip() + "\n" + + +def collect_html_warnings(report: schema.Report) -> list[str]: + """Collect data quality warnings for stderr output (NOT for the HTML artifact). + + Returns a list of human-readable warning strings. Empty list if the run + was clean. Used by the CLI to emit diagnostics to stderr after writing + the HTML to stdout/file. + """ + notes: list[str] = [] + if _render_degraded_run_warning(report): + notes.append("Run was missing pre-flight resolution. Re-run with `--plan` for richer results.") + elif _render_pre_research_warning(report): + notes.append("Pre-research was skipped, so results may be thinner than a resolved run.") + freshness_warning = _assess_data_freshness(report) + if freshness_warning: + notes.append(freshness_warning) + notes.extend(report.warnings) + return _dedupe_notes(notes) + + +def collect_html_warnings_comparison( + entity_reports: list[tuple[str, schema.Report]], +) -> list[str]: + """Collect comparison-mode warnings, prefixed by entity label.""" + notes: list[str] = [] + for label, report in entity_reports: + for w in collect_html_warnings(report): + notes.append(f"{label}: {w}") + return notes + + +def _render_html_metadata(report: schema.Report) -> list[str]: + """Inline metadata as an HTML comment marker. + + html_render.py post-processes ```` markers into a + ``
    `` after markdown conversion, so the metadata escapes + the markdown converter's HTML-escaping pass cleanly. Same pattern as the + PASS_THROUGH_FOOTER marker used for the engine tree. + """ + non_empty = [s for s, items in sorted(report.items_by_source.items()) if items] + if non_empty: + sources = ", ".join(_source_label(s) for s in non_empty) + else: + sources = "no active sources" + return [ + f"", + ] + + +def _render_html_data_quality_note(report: schema.Report) -> str | None: + notes: list[str] = [] + degraded_warning = _render_degraded_run_warning(report) + if degraded_warning: + notes.append("This run was missing pre-flight resolution. Re-run with `--plan` for richer results.") + pre_research_warning = _render_pre_research_warning(report) + if pre_research_warning and not degraded_warning: + notes.append("Pre-research was skipped, so results may be thinner than a resolved run.") + freshness_warning = _assess_data_freshness(report) + if freshness_warning: + notes.append(freshness_warning) + notes.extend(report.warnings) + if not notes: + return None + return f"> **Data quality note:** {' '.join(_dedupe_notes(notes))}" + + +def _render_html_comparison_data_quality_note( + entity_reports: list[tuple[str, schema.Report]], +) -> str | None: + notes: list[str] = [] + for label, report in entity_reports: + note = _render_html_data_quality_note(report) + if note: + clean = note.removeprefix("> **Data quality note:** ").strip() + notes.append(f"{label}: {clean}") + if not notes: + return None + return f"> **Data quality note:** {' '.join(_dedupe_notes(notes))}" + + +def _dedupe_notes(notes: list[str]) -> list[str]: + out: list[str] = [] + seen: set[str] = set() + for note in notes: + normalized = " ".join(str(note).split()) + if not normalized or normalized in seen: + continue + seen.add(normalized) + out.append(normalized) + return out + + +def _append_html_footer(lines: list[str], report: schema.Report, save_path: str | None) -> None: + footer = _render_emoji_footer(report, save_path) + lines.append("") + lines.append("") + lines.extend(footer) + lines.append("") + + def _render_canonical_boundary() -> list[str]: """Emit the explicit END-OF-CANONICAL-OUTPUT boundary. diff --git a/tests/test_html_render.py b/tests/test_html_render.py new file mode 100644 index 0000000..780884d --- /dev/null +++ b/tests/test_html_render.py @@ -0,0 +1,282 @@ +# ruff: noqa: E402 +"""Tests for the HTML emit renderer.""" + +from __future__ import annotations + +import sys +import tempfile +import unittest +from html.parser import HTMLParser +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parents[1] +sys.path.insert(0, str(REPO_ROOT / "skills" / "last30days" / "scripts")) + +import last30days as cli +from lib import html_render, schema + + +def _report(topic: str, cluster_titles: list[str]) -> schema.Report: + items: list[schema.SourceItem] = [] + candidates: list[schema.Candidate] = [] + clusters: list[schema.Cluster] = [] + + for index, title in enumerate(cluster_titles, start=1): + item = schema.SourceItem( + item_id=f"item-{index}", + source="grounding", + title=title, + body=f"Body for {title}", + url=f"https://example.test/{index}", + container="example.test", + published_at="2026-04-20", + date_confidence="high", + engagement={"views": index * 100}, + snippet=f"Snippet for {title}", + ) + candidate = schema.Candidate( + candidate_id=f"candidate-{index}", + item_id=item.item_id, + source="grounding", + title=title, + url=item.url, + snippet=item.snippet, + subquery_labels=["primary"], + native_ranks={"primary:grounding": index}, + local_relevance=0.9, + freshness=80, + engagement=50, + source_quality=1.0, + rrf_score=0.5, + final_score=90 - index, + sources=["grounding"], + source_items=[item], + ) + cluster = schema.Cluster( + cluster_id=f"cluster-{index}", + title=title, + candidate_ids=[candidate.candidate_id], + representative_ids=[candidate.candidate_id], + sources=["grounding"], + score=90 - index, + ) + items.append(item) + candidates.append(candidate) + clusters.append(cluster) + + return schema.Report( + topic=topic, + range_from="2026-03-30", + range_to="2026-04-29", + generated_at="2026-04-29T12:00:00+00:00", + provider_runtime=schema.ProviderRuntime( + reasoning_provider="local", + planner_model="mock-planner", + rerank_model="mock-rerank", + ), + query_plan=schema.QueryPlan( + intent="research", + freshness_mode="balanced_recent", + cluster_mode="story", + raw_topic=topic, + subqueries=[ + schema.SubQuery( + label="primary", + search_query=topic, + ranking_query=topic, + sources=["grounding"], + ) + ], + source_weights={"grounding": 1.0}, + ), + clusters=clusters, + ranked_candidates=candidates, + items_by_source={"grounding": items}, + errors_by_source={}, + artifacts={"pre_research_flags_present": True}, + ) + + +def _assert_parses(test_case: unittest.TestCase, html: str) -> None: + parser = HTMLParser() + parser.feed(html) + parser.close() + test_case.assertIn("", html) + + +class HtmlRenderSnapshotTests(unittest.TestCase): + def test_rich_cluster_fixture_snapshot(self): + rendered = html_render.render_html( + _report("AI agent frameworks", ["OpenClaw ships containers", "Skills marketplace grows"]) + ) + snapshot_markers = [ + "", + "last30days ยท AI agent frameworks", + '
    ๐ŸŒ last30days v', + '
    2026-03-30 to 2026-04-29', + '', + '
    2026-03-30 to 2026-04-29', + '/last30days OpenClaw vs Hermes', + ] + for marker in snapshot_markers: + self.assertIn(marker, rendered) + + +class HtmlRenderBehaviorTests(unittest.TestCase): + def test_prose_label_promotion(self): + md = html_render._promote_prose_labels("What I learned:") + rendered = html_render._markdown_to_html(md) + self.assertIn("

    What I learned

    ", rendered) + self.assertNotIn("What I learned:", rendered) + + def test_invitation_strip(self): + md = "---\nI'm now an expert on OpenClaw. Some things you could ask:\n\nJust ask." + self.assertNotIn("I'm now an expert", html_render._strip_invitation(md)) + + def test_evidence_block_strip(self): + md = "keep\n\nsecret\n" + stripped = html_render._strip_evidence_block(md) + self.assertIn("keep", stripped) + self.assertNotIn("EVIDENCE FOR SYNTHESIS", stripped) + self.assertNotIn("secret", stripped) + + def test_engine_footer_wrapping_preserves_tree(self): + md = ( + "\n" + "โœ… All agents reported back!\n" + "โ”œโ”€ ๐Ÿ”ต X: 2 posts\n" + "โ””โ”€ ๐ŸŒ Web: 1 result\n" + "" + ) + body = html_render._wrap_engine_footer(html_render._markdown_to_html(md)) + self.assertIn('