Initial import of NousResearch/hermes-agent
Deploy Site / deploy-vercel (push) Has been cancelled
Deploy Site / deploy-docs (push) Has been cancelled
Docker / shell lint / Lint Dockerfile (hadolint) (push) Has been cancelled
Docker / shell lint / Lint docker/ shell scripts (shellcheck) (push) Has been cancelled
Docker Build and Publish / build-amd64 (push) Has been cancelled
Docker Build and Publish / build-arm64 (push) Has been cancelled
Lint (ruff + ty) / ruff + ty diff (push) Has been cancelled
Lint (ruff + ty) / ruff enforcement (blocking) (push) Has been cancelled
Lint (ruff + ty) / Windows footguns (blocking) (push) Has been cancelled
Nix Lockfile Fix / auto-fix-main (push) Has been cancelled
Nix Lockfile Fix / fix (push) Has been cancelled
Nix / nix (macos-latest) (push) Has been cancelled
Nix / nix (ubuntu-latest) (push) Has been cancelled
OSV-Scanner / Scan lockfiles (push) Has been cancelled
Build Skills Index / build-index (push) Has been cancelled
Tests / test (1) (push) Has been cancelled
Tests / test (2) (push) Has been cancelled
Tests / test (3) (push) Has been cancelled
Tests / test (4) (push) Has been cancelled
Tests / test (5) (push) Has been cancelled
Tests / test (6) (push) Has been cancelled
Tests / e2e (push) Has been cancelled
uv.lock check / uv lock --check (push) Has been cancelled
Docker Build and Publish / merge (push) Has been cancelled
Build Skills Index / trigger-deploy (push) Has been cancelled
Tests / save-durations (push) Has been cancelled
Deploy Site / deploy-vercel (push) Has been cancelled
Deploy Site / deploy-docs (push) Has been cancelled
Docker / shell lint / Lint Dockerfile (hadolint) (push) Has been cancelled
Docker / shell lint / Lint docker/ shell scripts (shellcheck) (push) Has been cancelled
Docker Build and Publish / build-amd64 (push) Has been cancelled
Docker Build and Publish / build-arm64 (push) Has been cancelled
Lint (ruff + ty) / ruff + ty diff (push) Has been cancelled
Lint (ruff + ty) / ruff enforcement (blocking) (push) Has been cancelled
Lint (ruff + ty) / Windows footguns (blocking) (push) Has been cancelled
Nix Lockfile Fix / auto-fix-main (push) Has been cancelled
Nix Lockfile Fix / fix (push) Has been cancelled
Nix / nix (macos-latest) (push) Has been cancelled
Nix / nix (ubuntu-latest) (push) Has been cancelled
OSV-Scanner / Scan lockfiles (push) Has been cancelled
Build Skills Index / build-index (push) Has been cancelled
Tests / test (1) (push) Has been cancelled
Tests / test (2) (push) Has been cancelled
Tests / test (3) (push) Has been cancelled
Tests / test (4) (push) Has been cancelled
Tests / test (5) (push) Has been cancelled
Tests / test (6) (push) Has been cancelled
Tests / e2e (push) Has been cancelled
uv.lock check / uv lock --check (push) Has been cancelled
Docker Build and Publish / merge (push) Has been cancelled
Build Skills Index / trigger-deploy (push) Has been cancelled
Tests / save-durations (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,374 @@
|
||||
# Adding a New Messaging Platform
|
||||
|
||||
There are two ways to add a platform to the Hermes gateway:
|
||||
|
||||
## Plugin Path (Recommended for Community/Third-Party)
|
||||
|
||||
Create a plugin directory in `~/.hermes/plugins/` (or under `plugins/platforms/`
|
||||
for bundled plugins) with a `plugin.yaml` and `adapter.py`. The adapter
|
||||
inherits from `BasePlatformAdapter` and registers via
|
||||
`ctx.register_platform()` in the `register(ctx)` entry point. This requires
|
||||
**zero changes to core Hermes code**.
|
||||
|
||||
The plugin system automatically handles: adapter creation, config parsing,
|
||||
user authorization, cron delivery, send_message routing, system prompt hints,
|
||||
status display, gateway setup, and more.
|
||||
|
||||
**Optional hooks cover the edges most adapters need:**
|
||||
|
||||
- `env_enablement_fn: () -> Optional[dict]` — seeds `PlatformConfig.extra`
|
||||
(and an optional `home_channel` dict) from env vars BEFORE the adapter is
|
||||
constructed. Without this, env-only setups don't surface in
|
||||
`hermes gateway status` or `get_connected_platforms()` until the SDK
|
||||
instantiates.
|
||||
- `apply_yaml_config_fn: (yaml_cfg, platform_cfg) -> Optional[dict]` —
|
||||
translate this platform's `config.yaml` keys into env vars and/or seed
|
||||
`PlatformConfig.extra` directly. Lets a plugin own its YAML schema
|
||||
instead of growing core `gateway/config.py` boilerplate per platform.
|
||||
Mutating `os.environ` is allowed (use `not os.getenv(...)` guards to
|
||||
preserve env > YAML precedence); the returned dict is merged into
|
||||
`PlatformConfig.extra`. Called during `load_gateway_config()` after
|
||||
the generic shared-key loop and before `_apply_env_overrides()`.
|
||||
- `cron_deliver_env_var: str` — name of the `*_HOME_CHANNEL` env var. When
|
||||
set, `deliver=<name>` cron jobs route to this var without editing
|
||||
`cron/scheduler.py`'s hardcoded sets.
|
||||
- `standalone_sender_fn: async (...) -> dict`: out-of-process delivery
|
||||
for cron jobs that run separately from the gateway. Without this, a
|
||||
`deliver=<name>` job fires correctly but the actual send returns
|
||||
`No live adapter for platform '<name>'`. Pair with `cron_deliver_env_var`
|
||||
for end-to-end cron support. See the docsite for the signature.
|
||||
- `plugin.yaml` `requires_env` / `optional_env` rich-dict entries —
|
||||
auto-populate `OPTIONAL_ENV_VARS` in `hermes_cli/config.py` so the setup
|
||||
wizard surfaces proper descriptions, prompts, password flags, and URLs.
|
||||
|
||||
**Subclassing for platform-specific UX.** When a platform has a hard
|
||||
time-window constraint that the base adapter can't anticipate (LINE's
|
||||
60s single-use reply token, WhatsApp's 24h session window, etc.), an
|
||||
adapter can override `_keep_typing` to layer a mid-flight bubble at a
|
||||
threshold without expanding the kwarg surface. Always
|
||||
`await super()._keep_typing(...)` so the typing heartbeat keeps running,
|
||||
and tear down your side task in `finally`. See `plugins/platforms/line/`
|
||||
for the full pattern (Template Buttons postback at 45s, `RequestCache`
|
||||
state machine, `interrupt_session_activity` override for `/stop`
|
||||
orphans) and the developer-guide page for the prose walkthrough.
|
||||
|
||||
See `plugins/platforms/irc/`, `plugins/platforms/teams/`, and
|
||||
`plugins/platforms/google_chat/` for complete working examples, and
|
||||
`website/docs/developer-guide/adding-platform-adapters.md` for the full
|
||||
plugin guide with code examples and hook documentation.
|
||||
|
||||
---
|
||||
|
||||
## Built-in Path (Core Contributors Only)
|
||||
|
||||
Checklist for integrating a platform directly into the Hermes core.
|
||||
Use this as a reference when building a built-in adapter — every item here
|
||||
is a real integration point. Missing any of them will cause broken
|
||||
functionality, missing features, or inconsistent behavior.
|
||||
|
||||
---
|
||||
|
||||
## 1. Core Adapter (`gateway/platforms/<platform>.py`)
|
||||
|
||||
The adapter is a subclass of `BasePlatformAdapter` from `gateway/platforms/base.py`.
|
||||
|
||||
### Required methods
|
||||
|
||||
| Method | Purpose |
|
||||
|--------|---------|
|
||||
| `__init__(self, config)` | Parse config, init state. Call `super().__init__(config, Platform.YOUR_PLATFORM)` |
|
||||
| `connect() -> bool` | Connect to the platform, start listeners. Return True on success |
|
||||
| `disconnect()` | Stop listeners, close connections, cancel tasks |
|
||||
| `send(chat_id, text, ...) -> SendResult` | Send a text message |
|
||||
| `send_typing(chat_id)` | Send typing indicator |
|
||||
| `send_image(chat_id, image_url, caption) -> SendResult` | Send an image |
|
||||
| `get_chat_info(chat_id) -> dict` | Return `{name, type, chat_id}` for a chat |
|
||||
|
||||
### Optional methods (have default stubs in base)
|
||||
|
||||
| Method | Purpose |
|
||||
|--------|---------|
|
||||
| `send_document(chat_id, path, caption)` | Send a file attachment |
|
||||
| `send_voice(chat_id, path)` | Send a voice message |
|
||||
| `send_video(chat_id, path, caption)` | Send a video |
|
||||
| `send_animation(chat_id, path, caption)` | Send a GIF/animation |
|
||||
| `send_image_file(chat_id, path, caption)` | Send image from local file |
|
||||
|
||||
### Required function
|
||||
|
||||
```python
|
||||
def check_<platform>_requirements() -> bool:
|
||||
"""Check if this platform's dependencies are available."""
|
||||
```
|
||||
|
||||
### Key patterns to follow
|
||||
|
||||
- Use `self.build_source(...)` to construct `SessionSource` objects
|
||||
- Call `self.handle_message(event)` to dispatch inbound messages to the gateway
|
||||
- Use `MessageEvent`, `MessageType`, `SendResult` from base
|
||||
- Use `cache_image_from_bytes`, `cache_audio_from_bytes`, `cache_document_from_bytes` for attachments
|
||||
- Filter self-messages (prevent reply loops)
|
||||
- Filter sync/echo messages if the platform has them
|
||||
- Redact sensitive identifiers (phone numbers, tokens) in all log output
|
||||
- Implement reconnection with exponential backoff + jitter for streaming connections
|
||||
- Set `MAX_MESSAGE_LENGTH` if the platform has message size limits
|
||||
|
||||
---
|
||||
|
||||
## 2. Platform Enum (`gateway/config.py`)
|
||||
|
||||
Add the platform to the `Platform` enum:
|
||||
|
||||
```python
|
||||
class Platform(Enum):
|
||||
...
|
||||
YOUR_PLATFORM = "your_platform"
|
||||
```
|
||||
|
||||
Add env var loading in `_apply_env_overrides()`:
|
||||
|
||||
```python
|
||||
# Your Platform
|
||||
your_token = os.getenv("YOUR_PLATFORM_TOKEN")
|
||||
if your_token:
|
||||
if Platform.YOUR_PLATFORM not in config.platforms:
|
||||
config.platforms[Platform.YOUR_PLATFORM] = PlatformConfig()
|
||||
config.platforms[Platform.YOUR_PLATFORM].enabled = True
|
||||
config.platforms[Platform.YOUR_PLATFORM].token = your_token
|
||||
```
|
||||
|
||||
Update `get_connected_platforms()` if your platform doesn't use token/api_key
|
||||
(e.g., WhatsApp uses `enabled` flag, Signal uses `extra` dict).
|
||||
|
||||
---
|
||||
|
||||
## 3. Adapter Factory (`gateway/run.py`)
|
||||
|
||||
Add to `_create_adapter()`:
|
||||
|
||||
```python
|
||||
elif platform == Platform.YOUR_PLATFORM:
|
||||
from gateway.platforms.your_platform import YourAdapter, check_your_requirements
|
||||
if not check_your_requirements():
|
||||
logger.warning("Your Platform: dependencies not met")
|
||||
return None
|
||||
return YourAdapter(config)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Authorization Maps (`gateway/run.py`)
|
||||
|
||||
Add to BOTH dicts in `_is_user_authorized()`:
|
||||
|
||||
```python
|
||||
platform_env_map = {
|
||||
...
|
||||
Platform.YOUR_PLATFORM: "YOUR_PLATFORM_ALLOWED_USERS",
|
||||
}
|
||||
platform_allow_all_map = {
|
||||
...
|
||||
Platform.YOUR_PLATFORM: "YOUR_PLATFORM_ALLOW_ALL_USERS",
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Session Source (`gateway/session.py`)
|
||||
|
||||
If your platform needs extra identity fields (e.g., Signal's UUID alongside
|
||||
phone number), add them to the `SessionSource` dataclass with `Optional` defaults,
|
||||
and update `to_dict()`, `from_dict()`, and `build_source()` in base.py.
|
||||
|
||||
---
|
||||
|
||||
## 6. System Prompt Hints (`agent/prompt_builder.py`)
|
||||
|
||||
Add a `PLATFORM_HINTS` entry so the agent knows what platform it's on:
|
||||
|
||||
```python
|
||||
PLATFORM_HINTS = {
|
||||
...
|
||||
"your_platform": (
|
||||
"You are on Your Platform. "
|
||||
"Describe formatting capabilities, media support, etc."
|
||||
),
|
||||
}
|
||||
```
|
||||
|
||||
Without this, the agent won't know it's on your platform and may use
|
||||
inappropriate formatting (e.g., markdown on platforms that don't render it).
|
||||
|
||||
---
|
||||
|
||||
## 7. Toolset (`toolsets.py`)
|
||||
|
||||
Add a named toolset for your platform:
|
||||
|
||||
```python
|
||||
"hermes-your-platform": {
|
||||
"description": "Your Platform bot toolset",
|
||||
"tools": _HERMES_CORE_TOOLS,
|
||||
"includes": []
|
||||
},
|
||||
```
|
||||
|
||||
And add it to the `hermes-gateway` composite:
|
||||
|
||||
```python
|
||||
"hermes-gateway": {
|
||||
"includes": [..., "hermes-your-platform"]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Cron Delivery (`cron/scheduler.py`)
|
||||
|
||||
Add to `platform_map` in `_deliver_result()`:
|
||||
|
||||
```python
|
||||
platform_map = {
|
||||
...
|
||||
"your_platform": Platform.YOUR_PLATFORM,
|
||||
}
|
||||
```
|
||||
|
||||
Without this, `cronjob(action="create", deliver="your_platform", ...)` silently fails.
|
||||
|
||||
---
|
||||
|
||||
## 9. Send Message Tool (`tools/send_message_tool.py`)
|
||||
|
||||
Add to `platform_map` in `send_message_tool()`:
|
||||
|
||||
```python
|
||||
platform_map = {
|
||||
...
|
||||
"your_platform": Platform.YOUR_PLATFORM,
|
||||
}
|
||||
```
|
||||
|
||||
Add routing in `_send_to_platform()`:
|
||||
|
||||
```python
|
||||
elif platform == Platform.YOUR_PLATFORM:
|
||||
return await _send_your_platform(pconfig, chat_id, message)
|
||||
```
|
||||
|
||||
Implement `_send_your_platform()` — a standalone async function that sends
|
||||
a single message without requiring the full adapter (for use by cron jobs
|
||||
and the send_message tool outside the gateway process).
|
||||
|
||||
Update the tool schema `target` description to include your platform example.
|
||||
|
||||
---
|
||||
|
||||
## 10. Cronjob Tool Schema (`tools/cronjob_tools.py`)
|
||||
|
||||
Update the `deliver` parameter description and docstring to mention your
|
||||
platform as a delivery option.
|
||||
|
||||
---
|
||||
|
||||
## 11. Channel Directory (`gateway/channel_directory.py`)
|
||||
|
||||
If your platform can't enumerate chats (most can't), add it to the
|
||||
session-based discovery list:
|
||||
|
||||
```python
|
||||
for plat_name in ("telegram", "whatsapp", "signal", "your_platform"):
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12. Status Display (`hermes_cli/status.py`)
|
||||
|
||||
Add to the `platforms` dict in the Messaging Platforms section:
|
||||
|
||||
```python
|
||||
platforms = {
|
||||
...
|
||||
"Your Platform": ("YOUR_PLATFORM_TOKEN", "YOUR_PLATFORM_HOME_CHANNEL"),
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 13. Gateway Setup Wizard (`hermes_cli/gateway.py`)
|
||||
|
||||
Add to the `_PLATFORMS` list:
|
||||
|
||||
```python
|
||||
{
|
||||
"key": "your_platform",
|
||||
"label": "Your Platform",
|
||||
"emoji": "📱",
|
||||
"token_var": "YOUR_PLATFORM_TOKEN",
|
||||
"setup_instructions": [...],
|
||||
"vars": [...],
|
||||
}
|
||||
```
|
||||
|
||||
If your platform needs custom setup logic (connectivity testing, QR codes,
|
||||
policy choices), add a `_setup_your_platform()` function and route to it
|
||||
in the platform selection switch.
|
||||
|
||||
Update `_platform_status()` if your platform's "configured" check differs
|
||||
from the standard `bool(get_env_value(token_var))`.
|
||||
|
||||
---
|
||||
|
||||
## 14. Phone/ID Redaction (`agent/redact.py`)
|
||||
|
||||
If your platform uses sensitive identifiers (phone numbers, etc.), add a
|
||||
regex pattern and redaction function to `agent/redact.py`. This ensures
|
||||
identifiers are masked in ALL log output, not just your adapter's logs.
|
||||
|
||||
---
|
||||
|
||||
## 15. Documentation
|
||||
|
||||
| File | What to update |
|
||||
|------|---------------|
|
||||
| `README.md` | Platform list in feature table + documentation table |
|
||||
| `AGENTS.md` | Gateway description + env var config section |
|
||||
| `website/docs/user-guide/messaging/<platform>.md` | **NEW** — Full setup guide (see existing platform docs for template) |
|
||||
| `website/docs/user-guide/messaging/index.md` | Architecture diagram, toolset table, security examples, Next Steps links |
|
||||
| `website/docs/reference/environment-variables.md` | All env vars for the platform |
|
||||
|
||||
---
|
||||
|
||||
## 16. Tests (`tests/gateway/test_<platform>.py`)
|
||||
|
||||
Recommended test coverage:
|
||||
|
||||
- Platform enum exists with correct value
|
||||
- Config loading from env vars via `_apply_env_overrides`
|
||||
- Adapter init (config parsing, allowlist handling, default values)
|
||||
- Helper functions (redaction, parsing, file type detection)
|
||||
- Session source round-trip (to_dict → from_dict)
|
||||
- Authorization integration (platform in allowlist maps)
|
||||
- Send message tool routing (platform in platform_map)
|
||||
|
||||
Optional but valuable:
|
||||
- Async tests for message handling flow (mock the platform API)
|
||||
- SSE/WebSocket reconnection logic
|
||||
- Attachment processing
|
||||
- Group message filtering
|
||||
|
||||
---
|
||||
|
||||
## Quick Verification
|
||||
|
||||
After implementing everything, verify with:
|
||||
|
||||
```bash
|
||||
# All tests pass
|
||||
python -m pytest tests/ -q
|
||||
|
||||
# Grep for your platform name to find any missed integration points
|
||||
grep -r "telegram\|discord\|whatsapp\|slack" gateway/ tools/ agent/ cron/ hermes_cli/ toolsets.py \
|
||||
--include="*.py" -l | sort -u
|
||||
# Check each file in the output — if it mentions other platforms but not yours, you missed it
|
||||
```
|
||||
@@ -0,0 +1,45 @@
|
||||
"""
|
||||
Platform adapters for messaging integrations.
|
||||
|
||||
Each adapter handles:
|
||||
- Receiving messages from a platform
|
||||
- Sending messages/responses back
|
||||
- Platform-specific authentication
|
||||
- Message formatting and media handling
|
||||
"""
|
||||
|
||||
from .base import BasePlatformAdapter, MessageEvent, SendResult
|
||||
|
||||
# QQAdapter and YuanbaoAdapter were previously imported eagerly here, but
|
||||
# nothing in the codebase consumes ``from gateway.platforms import
|
||||
# QQAdapter`` (every real call site uses the long-form path
|
||||
# ``from gateway.platforms.qqbot import QQAdapter``). The eager imports
|
||||
# pulled in qqbot's chunked-upload + keyboards + onboard machinery and
|
||||
# yuanbao's websocket stack — about 48 ms wall and ~8 MB RSS on every
|
||||
# CLI invocation, even ones that never touch a gateway adapter.
|
||||
#
|
||||
# Use PEP 562 module ``__getattr__`` to keep the public re-export working
|
||||
# while deferring the actual import to first attribute access. This is
|
||||
# 100% backward-compatible for any external code that still imports the
|
||||
# adapters from the package root.
|
||||
__all__ = [
|
||||
"BasePlatformAdapter",
|
||||
"MessageEvent",
|
||||
"SendResult",
|
||||
"QQAdapter",
|
||||
"YuanbaoAdapter",
|
||||
]
|
||||
|
||||
|
||||
def __getattr__(name):
|
||||
if name == "QQAdapter":
|
||||
from .qqbot import QQAdapter # noqa: F401
|
||||
return QQAdapter
|
||||
if name == "YuanbaoAdapter":
|
||||
from .yuanbao import YuanbaoAdapter # noqa: F401
|
||||
return YuanbaoAdapter
|
||||
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
|
||||
|
||||
|
||||
def __dir__():
|
||||
return sorted(__all__)
|
||||
@@ -0,0 +1,84 @@
|
||||
"""Shared HTTP client factory for long-lived platform adapters.
|
||||
|
||||
Gateway messaging platforms (QQ Bot, Feishu, WeCom, DingTalk, Signal,
|
||||
BlueBubbles, WeCom-callback) keep a persistent ``httpx.AsyncClient``
|
||||
alive for the adapter's lifetime. That amortises TLS/connection setup
|
||||
across many API calls, but it also means the process's file-descriptor
|
||||
pressure is sensitive to how aggressively the pool recycles idle keep-
|
||||
alive connections.
|
||||
|
||||
httpx's default ``keepalive_expiry`` is 5 seconds. On macOS behind
|
||||
Cloudflare Warp (and other transparent proxies), peer-initiated FIN can
|
||||
sit in ``CLOSE_WAIT`` longer than that before the local socket actually
|
||||
drains — which, multiplied across 7 long-lived adapters plus the LLM
|
||||
client and MCP clients, walks straight into the default 256 fd limit.
|
||||
See #18451.
|
||||
|
||||
``platform_httpx_limits()`` returns a tighter ``httpx.Limits`` the
|
||||
adapter factories use instead of the httpx default. The values chosen:
|
||||
|
||||
* ``max_keepalive_connections=10`` — plenty for any single adapter;
|
||||
platform APIs rarely parallelise beyond this.
|
||||
* ``keepalive_expiry=2.0`` — close idle sockets aggressively so a
|
||||
proxy's lingering CLOSE_WAIT window can't starve the process.
|
||||
|
||||
Override via ``HERMES_GATEWAY_HTTPX_KEEPALIVE_EXPIRY`` /
|
||||
``HERMES_GATEWAY_HTTPX_MAX_KEEPALIVE`` env vars when tuning under load.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
try:
|
||||
import httpx
|
||||
except ImportError: # pragma: no cover — optional dep
|
||||
httpx = None # type: ignore[assignment]
|
||||
|
||||
|
||||
_DEFAULT_KEEPALIVE_EXPIRY_S = 2.0
|
||||
_DEFAULT_MAX_KEEPALIVE = 10
|
||||
|
||||
|
||||
def platform_httpx_limits() -> "httpx.Limits | None":
|
||||
"""Return ``httpx.Limits`` tuned for persistent platform-adapter clients.
|
||||
|
||||
Returns ``None`` when httpx isn't importable, so callers can fall
|
||||
back to httpx's built-in default without a hard dependency on this
|
||||
helper being reachable.
|
||||
"""
|
||||
if httpx is None:
|
||||
return None
|
||||
|
||||
def _env_float(name: str, default: float) -> float:
|
||||
raw = os.environ.get(name, "").strip()
|
||||
if not raw:
|
||||
return default
|
||||
try:
|
||||
val = float(raw)
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
return val if val > 0 else default
|
||||
|
||||
def _env_int(name: str, default: int) -> int:
|
||||
raw = os.environ.get(name, "").strip()
|
||||
if not raw:
|
||||
return default
|
||||
try:
|
||||
val = int(raw)
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
return val if val > 0 else default
|
||||
|
||||
keepalive_expiry = _env_float(
|
||||
"HERMES_GATEWAY_HTTPX_KEEPALIVE_EXPIRY", _DEFAULT_KEEPALIVE_EXPIRY_S
|
||||
)
|
||||
max_keepalive = _env_int(
|
||||
"HERMES_GATEWAY_HTTPX_MAX_KEEPALIVE", _DEFAULT_MAX_KEEPALIVE
|
||||
)
|
||||
|
||||
return httpx.Limits(
|
||||
max_keepalive_connections=max_keepalive,
|
||||
# Leave max_connections at httpx default (100) — plenty of headroom.
|
||||
keepalive_expiry=keepalive_expiry,
|
||||
)
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,949 @@
|
||||
"""BlueBubbles iMessage platform adapter.
|
||||
|
||||
Uses the local BlueBubbles macOS server for outbound REST sends and inbound
|
||||
webhooks. Supports text messaging, media attachments (images, voice, video,
|
||||
documents), tapback reactions, typing indicators, and read receipts.
|
||||
|
||||
Architecture based on PR #5869 (benjaminsehl) with inbound attachment
|
||||
downloading from PR #4588 (YuhangLin).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, List, Optional
|
||||
from urllib.parse import quote
|
||||
|
||||
import httpx
|
||||
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
from gateway.platforms.base import (
|
||||
BasePlatformAdapter,
|
||||
MessageEvent,
|
||||
MessageType,
|
||||
SendResult,
|
||||
cache_image_from_bytes,
|
||||
cache_audio_from_bytes,
|
||||
cache_document_from_bytes,
|
||||
)
|
||||
from gateway.platforms.helpers import strip_markdown
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
DEFAULT_WEBHOOK_HOST = "127.0.0.1"
|
||||
DEFAULT_WEBHOOK_PORT = 8645
|
||||
DEFAULT_WEBHOOK_PATH = "/bluebubbles-webhook"
|
||||
MAX_TEXT_LENGTH = 4000
|
||||
|
||||
# Tapback reaction codes (BlueBubbles associatedMessageType values)
|
||||
_TAPBACK_ADDED = {
|
||||
2000: "love", 2001: "like", 2002: "dislike",
|
||||
2003: "laugh", 2004: "emphasize", 2005: "question",
|
||||
}
|
||||
_TAPBACK_REMOVED = {
|
||||
3000: "love", 3001: "like", 3002: "dislike",
|
||||
3003: "laugh", 3004: "emphasize", 3005: "question",
|
||||
}
|
||||
|
||||
# Webhook event types that carry user messages
|
||||
_MESSAGE_EVENTS = {"new-message", "message", "updated-message"}
|
||||
|
||||
# Log redaction patterns
|
||||
_PHONE_RE = re.compile(r"\+?\d{7,15}")
|
||||
_EMAIL_RE = re.compile(r"[\w.+-]+@[\w-]+\.[\w.]+")
|
||||
|
||||
|
||||
def _redact(text: str) -> str:
|
||||
"""Redact phone numbers and emails from log output."""
|
||||
text = _PHONE_RE.sub("[REDACTED]", text)
|
||||
text = _EMAIL_RE.sub("[REDACTED]", text)
|
||||
return text
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def check_bluebubbles_requirements() -> bool:
|
||||
try:
|
||||
import aiohttp # noqa: F401
|
||||
import httpx # noqa: F401
|
||||
except ImportError:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _normalize_server_url(raw: str) -> str:
|
||||
value = (raw or "").strip()
|
||||
if not value:
|
||||
return ""
|
||||
if not re.match(r"^https?://", value, flags=re.I):
|
||||
value = f"http://{value}"
|
||||
return value.rstrip("/")
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Adapter
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class BlueBubblesAdapter(BasePlatformAdapter):
|
||||
platform = Platform.BLUEBUBBLES
|
||||
SUPPORTS_MESSAGE_EDITING = False
|
||||
MAX_MESSAGE_LENGTH = MAX_TEXT_LENGTH
|
||||
|
||||
def __init__(self, config: PlatformConfig):
|
||||
super().__init__(config, Platform.BLUEBUBBLES)
|
||||
extra = config.extra or {}
|
||||
self.server_url = _normalize_server_url(
|
||||
extra.get("server_url") or os.getenv("BLUEBUBBLES_SERVER_URL", "")
|
||||
)
|
||||
self.password = extra.get("password") or os.getenv("BLUEBUBBLES_PASSWORD", "")
|
||||
self.webhook_host = (
|
||||
extra.get("webhook_host")
|
||||
or os.getenv("BLUEBUBBLES_WEBHOOK_HOST", DEFAULT_WEBHOOK_HOST)
|
||||
)
|
||||
self.webhook_port = int(
|
||||
extra.get("webhook_port")
|
||||
or os.getenv("BLUEBUBBLES_WEBHOOK_PORT", str(DEFAULT_WEBHOOK_PORT))
|
||||
)
|
||||
self.webhook_path = (
|
||||
extra.get("webhook_path")
|
||||
or os.getenv("BLUEBUBBLES_WEBHOOK_PATH", DEFAULT_WEBHOOK_PATH)
|
||||
)
|
||||
if not str(self.webhook_path).startswith("/"):
|
||||
self.webhook_path = f"/{self.webhook_path}"
|
||||
self.send_read_receipts = bool(extra.get("send_read_receipts", True))
|
||||
self.client: Optional[httpx.AsyncClient] = None
|
||||
self._runner = None
|
||||
self._private_api_enabled: Optional[bool] = None
|
||||
self._helper_connected: bool = False
|
||||
self._guid_cache: Dict[str, str] = {}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# API helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _api_url(self, path: str) -> str:
|
||||
sep = "&" if "?" in path else "?"
|
||||
return f"{self.server_url}{path}{sep}password={quote(self.password, safe='')}"
|
||||
|
||||
async def _api_get(self, path: str) -> Dict[str, Any]:
|
||||
assert self.client is not None
|
||||
res = await self.client.get(self._api_url(path))
|
||||
res.raise_for_status()
|
||||
return res.json()
|
||||
|
||||
async def _api_post(self, path: str, payload: Dict[str, Any]) -> Dict[str, Any]:
|
||||
assert self.client is not None
|
||||
res = await self.client.post(self._api_url(path), json=payload)
|
||||
res.raise_for_status()
|
||||
return res.json()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Lifecycle
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def connect(self) -> bool:
|
||||
if not self.server_url or not self.password:
|
||||
logger.error(
|
||||
"[bluebubbles] BLUEBUBBLES_SERVER_URL and BLUEBUBBLES_PASSWORD are required"
|
||||
)
|
||||
return False
|
||||
from aiohttp import web
|
||||
|
||||
# Tighter keepalive so idle CLOSE_WAIT drains promptly (#18451).
|
||||
from gateway.platforms._http_client_limits import platform_httpx_limits
|
||||
self.client = httpx.AsyncClient(timeout=30.0, limits=platform_httpx_limits())
|
||||
try:
|
||||
await self._api_get("/api/v1/ping")
|
||||
info = await self._api_get("/api/v1/server/info")
|
||||
server_data = (info or {}).get("data", {})
|
||||
self._private_api_enabled = bool(server_data.get("private_api"))
|
||||
self._helper_connected = bool(server_data.get("helper_connected"))
|
||||
logger.info(
|
||||
"[bluebubbles] connected to %s (private_api=%s, helper=%s)",
|
||||
self.server_url,
|
||||
self._private_api_enabled,
|
||||
self._helper_connected,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error(
|
||||
"[bluebubbles] cannot reach server at %s: %s", self.server_url, exc
|
||||
)
|
||||
if self.client:
|
||||
await self.client.aclose()
|
||||
self.client = None
|
||||
return False
|
||||
|
||||
app = web.Application()
|
||||
app.router.add_get("/health", lambda _: web.Response(text="ok"))
|
||||
app.router.add_post(self.webhook_path, self._handle_webhook)
|
||||
# The webhook auth value is carried in the query string because the
|
||||
# BlueBubbles webhook API cannot send custom headers. Do not let
|
||||
# aiohttp access logs write that request target to agent.log.
|
||||
self._runner = web.AppRunner(app, access_log=None)
|
||||
await self._runner.setup()
|
||||
site = web.TCPSite(self._runner, self.webhook_host, self.webhook_port)
|
||||
await site.start()
|
||||
self._mark_connected()
|
||||
logger.info(
|
||||
"[bluebubbles] webhook listening on http://%s:%s%s",
|
||||
self.webhook_host,
|
||||
self.webhook_port,
|
||||
self.webhook_path,
|
||||
)
|
||||
|
||||
# Register webhook with BlueBubbles server
|
||||
# This is required for the server to know where to send events
|
||||
await self._register_webhook()
|
||||
|
||||
return True
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
# Unregister webhook before cleaning up
|
||||
await self._unregister_webhook()
|
||||
|
||||
if self.client:
|
||||
await self.client.aclose()
|
||||
self.client = None
|
||||
if self._runner:
|
||||
await self._runner.cleanup()
|
||||
self._runner = None
|
||||
self._mark_disconnected()
|
||||
|
||||
@property
|
||||
def _webhook_url(self) -> str:
|
||||
"""Compute the external webhook URL for BlueBubbles registration."""
|
||||
host = self.webhook_host
|
||||
if host in {"0.0.0.0", "127.0.0.1", "localhost", "::"}:
|
||||
host = "localhost"
|
||||
return f"http://{host}:{self.webhook_port}{self.webhook_path}"
|
||||
|
||||
@property
|
||||
def _webhook_register_url(self) -> str:
|
||||
"""Webhook URL registered with BlueBubbles, including the password as
|
||||
a query param so inbound webhook POSTs carry credentials.
|
||||
|
||||
BlueBubbles posts events to the exact URL registered via
|
||||
``/api/v1/webhook``. Its webhook registration API does not support
|
||||
custom headers, so embedding the password in the URL is the only
|
||||
way to authenticate inbound webhooks without disabling auth.
|
||||
"""
|
||||
base = self._webhook_url
|
||||
if self.password:
|
||||
return f"{base}?password={quote(self.password, safe='')}"
|
||||
return base
|
||||
|
||||
@property
|
||||
def _webhook_register_url_for_log(self) -> str:
|
||||
"""Webhook registration URL safe for logs."""
|
||||
base = self._webhook_url
|
||||
if self.password:
|
||||
return f"{base}?password=***"
|
||||
return base
|
||||
|
||||
async def _find_registered_webhooks(self, url: str) -> list:
|
||||
"""Return list of BB webhook entries matching *url*."""
|
||||
try:
|
||||
res = await self._api_get("/api/v1/webhook")
|
||||
data = res.get("data")
|
||||
if isinstance(data, list):
|
||||
return [wh for wh in data if wh.get("url") == url]
|
||||
except Exception:
|
||||
pass
|
||||
return []
|
||||
|
||||
async def _register_webhook(self) -> bool:
|
||||
"""Register this webhook URL with the BlueBubbles server.
|
||||
|
||||
BlueBubbles requires webhooks to be registered via API before
|
||||
it will send events. Checks for an existing registration first
|
||||
to avoid duplicates (e.g. after a crash without clean shutdown).
|
||||
"""
|
||||
if not self.client:
|
||||
return False
|
||||
|
||||
webhook_url = self._webhook_register_url
|
||||
|
||||
# Crash resilience — reuse an existing registration if present
|
||||
existing = await self._find_registered_webhooks(webhook_url)
|
||||
if existing:
|
||||
logger.info(
|
||||
"[bluebubbles] webhook already registered: %s",
|
||||
self._webhook_register_url_for_log,
|
||||
)
|
||||
return True
|
||||
|
||||
payload = {
|
||||
"url": webhook_url,
|
||||
"events": ["new-message", "updated-message"],
|
||||
}
|
||||
|
||||
try:
|
||||
res = await self._api_post("/api/v1/webhook", payload)
|
||||
status = res.get("status", 0)
|
||||
if 200 <= status < 300:
|
||||
logger.info(
|
||||
"[bluebubbles] webhook registered with server: %s",
|
||||
self._webhook_register_url_for_log,
|
||||
)
|
||||
return True
|
||||
else:
|
||||
logger.warning(
|
||||
"[bluebubbles] webhook registration returned status %s: %s",
|
||||
status,
|
||||
res.get("message"),
|
||||
)
|
||||
return False
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"[bluebubbles] failed to register webhook with server: %s",
|
||||
exc,
|
||||
)
|
||||
return False
|
||||
|
||||
async def _unregister_webhook(self) -> bool:
|
||||
"""Unregister this webhook URL from the BlueBubbles server.
|
||||
|
||||
Removes *all* matching registrations to clean up any duplicates
|
||||
left by prior crashes.
|
||||
"""
|
||||
if not self.client:
|
||||
return False
|
||||
|
||||
webhook_url = self._webhook_register_url
|
||||
removed = False
|
||||
|
||||
try:
|
||||
for wh in await self._find_registered_webhooks(webhook_url):
|
||||
wh_id = wh.get("id")
|
||||
if wh_id:
|
||||
res = await self.client.delete(
|
||||
self._api_url(f"/api/v1/webhook/{wh_id}")
|
||||
)
|
||||
res.raise_for_status()
|
||||
removed = True
|
||||
if removed:
|
||||
logger.info(
|
||||
"[bluebubbles] webhook unregistered: %s",
|
||||
self._webhook_register_url_for_log,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.debug(
|
||||
"[bluebubbles] failed to unregister webhook (non-critical): %s",
|
||||
exc,
|
||||
)
|
||||
return removed
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Chat GUID resolution
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _resolve_chat_guid(self, target: str) -> Optional[str]:
|
||||
"""Resolve an email/phone to a BlueBubbles chat GUID.
|
||||
|
||||
If *target* already contains a semicolon (raw GUID format like
|
||||
``iMessage;-;user@example.com``), it is returned as-is. Otherwise
|
||||
the adapter queries the BlueBubbles chat list and matches on
|
||||
``chatIdentifier`` or participant address.
|
||||
"""
|
||||
target = (target or "").strip()
|
||||
if not target:
|
||||
return None
|
||||
# Already a raw GUID
|
||||
if ";" in target:
|
||||
return target
|
||||
if target in self._guid_cache:
|
||||
return self._guid_cache[target]
|
||||
try:
|
||||
payload = await self._api_post(
|
||||
"/api/v1/chat/query",
|
||||
{"limit": 100, "offset": 0, "with": ["participants"]},
|
||||
)
|
||||
for chat in payload.get("data", []) or []:
|
||||
guid = chat.get("guid") or chat.get("chatGuid")
|
||||
identifier = chat.get("chatIdentifier") or chat.get("identifier")
|
||||
if identifier == target:
|
||||
if guid:
|
||||
self._guid_cache[target] = guid
|
||||
return guid
|
||||
for part in chat.get("participants", []) or []:
|
||||
if (part.get("address") or "").strip() == target and guid:
|
||||
self._guid_cache[target] = guid
|
||||
return guid
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
async def _create_chat_for_handle(
|
||||
self, address: str, message: str
|
||||
) -> SendResult:
|
||||
"""Create a new chat by sending the first message to *address*."""
|
||||
payload = {
|
||||
"addresses": [address],
|
||||
"message": message,
|
||||
"tempGuid": f"temp-{datetime.utcnow().timestamp()}",
|
||||
}
|
||||
try:
|
||||
res = await self._api_post("/api/v1/chat/new", payload)
|
||||
data = res.get("data") or {}
|
||||
msg_id = data.get("guid") or data.get("messageGuid") or "ok"
|
||||
return SendResult(success=True, message_id=str(msg_id), raw_response=res)
|
||||
except Exception as exc:
|
||||
return SendResult(success=False, error=str(exc))
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Text sending
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@staticmethod
|
||||
def truncate_message(content: str, max_length: int = MAX_TEXT_LENGTH) -> List[str]:
|
||||
# Use the base splitter but skip pagination indicators — iMessage
|
||||
# bubbles flow naturally without "(1/3)" suffixes.
|
||||
chunks = BasePlatformAdapter.truncate_message(content, max_length)
|
||||
return [re.sub(r"\s*\(\d+/\d+\)$", "", c) for c in chunks]
|
||||
|
||||
async def send(
|
||||
self,
|
||||
chat_id: str,
|
||||
content: str,
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
text = self.format_message(content)
|
||||
if not text:
|
||||
return SendResult(success=False, error="BlueBubbles send requires text")
|
||||
# Split on paragraph breaks first (double newlines) so each thought
|
||||
# becomes its own iMessage bubble, then truncate any that are still
|
||||
# too long.
|
||||
paragraphs = [p.strip() for p in re.split(r'\n\s*\n', text) if p.strip()]
|
||||
chunks: List[str] = []
|
||||
for para in (paragraphs or [text]):
|
||||
if len(para) <= self.MAX_MESSAGE_LENGTH:
|
||||
chunks.append(para)
|
||||
else:
|
||||
chunks.extend(self.truncate_message(para, max_length=self.MAX_MESSAGE_LENGTH))
|
||||
last = SendResult(success=True)
|
||||
for chunk in chunks:
|
||||
guid = await self._resolve_chat_guid(chat_id)
|
||||
if not guid:
|
||||
# If the target looks like an address, try creating a new chat
|
||||
if self._private_api_enabled and (
|
||||
"@" in chat_id or re.match(r"^\+\d+", chat_id)
|
||||
):
|
||||
return await self._create_chat_for_handle(chat_id, chunk)
|
||||
return SendResult(
|
||||
success=False,
|
||||
error=f"BlueBubbles chat not found for target: {chat_id}",
|
||||
)
|
||||
payload: Dict[str, Any] = {
|
||||
"chatGuid": guid,
|
||||
"tempGuid": f"temp-{datetime.utcnow().timestamp()}",
|
||||
"message": chunk,
|
||||
}
|
||||
if reply_to and self._private_api_enabled and self._helper_connected:
|
||||
payload["method"] = "private-api"
|
||||
payload["selectedMessageGuid"] = reply_to
|
||||
payload["partIndex"] = 0
|
||||
try:
|
||||
res = await self._api_post("/api/v1/message/text", payload)
|
||||
data = res.get("data") or {}
|
||||
msg_id = data.get("guid") or data.get("messageGuid") or "ok"
|
||||
last = SendResult(
|
||||
success=True, message_id=str(msg_id), raw_response=res
|
||||
)
|
||||
except Exception as exc:
|
||||
return SendResult(success=False, error=str(exc))
|
||||
return last
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Media sending (outbound)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _send_attachment(
|
||||
self,
|
||||
chat_id: str,
|
||||
file_path: str,
|
||||
filename: Optional[str] = None,
|
||||
caption: Optional[str] = None,
|
||||
is_audio_message: bool = False,
|
||||
) -> SendResult:
|
||||
"""Send a file attachment via BlueBubbles multipart upload."""
|
||||
if not self.client:
|
||||
return SendResult(success=False, error="Not connected")
|
||||
if not os.path.isfile(file_path):
|
||||
return SendResult(success=False, error=f"File not found: {file_path}")
|
||||
|
||||
guid = await self._resolve_chat_guid(chat_id)
|
||||
if not guid:
|
||||
return SendResult(success=False, error=f"Chat not found: {chat_id}")
|
||||
|
||||
fname = filename or os.path.basename(file_path)
|
||||
try:
|
||||
with open(file_path, "rb") as f:
|
||||
files = {"attachment": (fname, f, "application/octet-stream")}
|
||||
data: Dict[str, str] = {
|
||||
"chatGuid": guid,
|
||||
"name": fname,
|
||||
"tempGuid": uuid.uuid4().hex,
|
||||
}
|
||||
if is_audio_message:
|
||||
data["isAudioMessage"] = "true"
|
||||
res = await self.client.post(
|
||||
self._api_url("/api/v1/message/attachment"),
|
||||
files=files,
|
||||
data=data,
|
||||
timeout=120,
|
||||
)
|
||||
res.raise_for_status()
|
||||
result = res.json()
|
||||
|
||||
if caption:
|
||||
await self.send(chat_id, caption)
|
||||
|
||||
if result.get("status") == 200:
|
||||
rdata = result.get("data") or {}
|
||||
msg_id = rdata.get("guid") if isinstance(rdata, dict) else None
|
||||
return SendResult(
|
||||
success=True, message_id=msg_id, raw_response=result
|
||||
)
|
||||
return SendResult(
|
||||
success=False,
|
||||
error=result.get("message", "Attachment upload failed"),
|
||||
)
|
||||
except Exception as e:
|
||||
return SendResult(success=False, error=str(e))
|
||||
|
||||
async def send_image(
|
||||
self,
|
||||
chat_id: str,
|
||||
image_url: str,
|
||||
caption: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
try:
|
||||
from gateway.platforms.base import cache_image_from_url
|
||||
|
||||
local_path = await cache_image_from_url(image_url)
|
||||
return await self._send_attachment(chat_id, local_path, caption=caption)
|
||||
except Exception:
|
||||
return await super().send_image(chat_id, image_url, caption, reply_to)
|
||||
|
||||
async def send_image_file(
|
||||
self,
|
||||
chat_id: str,
|
||||
image_path: str,
|
||||
caption: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
**kwargs,
|
||||
) -> SendResult:
|
||||
return await self._send_attachment(chat_id, image_path, caption=caption)
|
||||
|
||||
async def send_voice(
|
||||
self,
|
||||
chat_id: str,
|
||||
audio_path: str,
|
||||
caption: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
**kwargs,
|
||||
) -> SendResult:
|
||||
return await self._send_attachment(
|
||||
chat_id, audio_path, caption=caption, is_audio_message=True
|
||||
)
|
||||
|
||||
async def send_video(
|
||||
self,
|
||||
chat_id: str,
|
||||
video_path: str,
|
||||
caption: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
**kwargs,
|
||||
) -> SendResult:
|
||||
return await self._send_attachment(chat_id, video_path, caption=caption)
|
||||
|
||||
async def send_document(
|
||||
self,
|
||||
chat_id: str,
|
||||
file_path: str,
|
||||
caption: Optional[str] = None,
|
||||
file_name: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
**kwargs,
|
||||
) -> SendResult:
|
||||
return await self._send_attachment(
|
||||
chat_id, file_path, filename=file_name, caption=caption
|
||||
)
|
||||
|
||||
async def send_animation(
|
||||
self,
|
||||
chat_id: str,
|
||||
animation_url: str,
|
||||
caption: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
return await self.send_image(
|
||||
chat_id, animation_url, caption, reply_to, metadata
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Typing indicators
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def send_typing(self, chat_id: str, metadata=None) -> None:
|
||||
if not self._private_api_enabled or not self._helper_connected or not self.client:
|
||||
return
|
||||
try:
|
||||
guid = await self._resolve_chat_guid(chat_id)
|
||||
if guid:
|
||||
encoded = quote(guid, safe="")
|
||||
await self.client.post(
|
||||
self._api_url(f"/api/v1/chat/{encoded}/typing"), timeout=5
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
async def stop_typing(self, chat_id: str) -> None:
|
||||
if not self._private_api_enabled or not self._helper_connected or not self.client:
|
||||
return
|
||||
try:
|
||||
guid = await self._resolve_chat_guid(chat_id)
|
||||
if guid:
|
||||
encoded = quote(guid, safe="")
|
||||
await self.client.delete(
|
||||
self._api_url(f"/api/v1/chat/{encoded}/typing"), timeout=5
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Read receipts
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def mark_read(self, chat_id: str) -> bool:
|
||||
if not self._private_api_enabled or not self._helper_connected or not self.client:
|
||||
return False
|
||||
try:
|
||||
guid = await self._resolve_chat_guid(chat_id)
|
||||
if guid:
|
||||
encoded = quote(guid, safe="")
|
||||
await self.client.post(
|
||||
self._api_url(f"/api/v1/chat/{encoded}/read"), timeout=5
|
||||
)
|
||||
return True
|
||||
except Exception:
|
||||
pass
|
||||
return False
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Tapback reactions
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Chat info
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
|
||||
is_group = ";+;" in (chat_id or "")
|
||||
info: Dict[str, Any] = {
|
||||
"name": chat_id,
|
||||
"type": "group" if is_group else "dm",
|
||||
}
|
||||
try:
|
||||
guid = await self._resolve_chat_guid(chat_id)
|
||||
if guid:
|
||||
encoded = quote(guid, safe="")
|
||||
res = await self._api_get(
|
||||
f"/api/v1/chat/{encoded}?with=participants"
|
||||
)
|
||||
data = (res or {}).get("data", {})
|
||||
display_name = (
|
||||
data.get("displayName")
|
||||
or data.get("chatIdentifier")
|
||||
or chat_id
|
||||
)
|
||||
participants = []
|
||||
for p in data.get("participants", []) or []:
|
||||
addr = (p.get("address") or "").strip()
|
||||
if addr:
|
||||
participants.append(addr)
|
||||
info["name"] = display_name
|
||||
if participants:
|
||||
info["participants"] = participants
|
||||
except Exception:
|
||||
pass
|
||||
return info
|
||||
|
||||
def format_message(self, content: str) -> str:
|
||||
return strip_markdown(content)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Inbound attachment downloading (from #4588)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _download_attachment(
|
||||
self, att_guid: str, att_meta: Dict[str, Any]
|
||||
) -> Optional[str]:
|
||||
"""Download an attachment from BlueBubbles and cache it locally.
|
||||
|
||||
Returns the local file path on success, None on failure.
|
||||
"""
|
||||
if not self.client:
|
||||
return None
|
||||
try:
|
||||
encoded = quote(att_guid, safe="")
|
||||
resp = await self.client.get(
|
||||
self._api_url(f"/api/v1/attachment/{encoded}/download"),
|
||||
timeout=60,
|
||||
follow_redirects=True,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.content
|
||||
|
||||
mime = (att_meta.get("mimeType") or "").lower()
|
||||
transfer_name = att_meta.get("transferName", "")
|
||||
|
||||
if mime.startswith("image/"):
|
||||
ext_map = {
|
||||
"image/jpeg": ".jpg",
|
||||
"image/png": ".png",
|
||||
"image/gif": ".gif",
|
||||
"image/webp": ".webp",
|
||||
"image/heic": ".jpg",
|
||||
"image/heif": ".jpg",
|
||||
"image/tiff": ".jpg",
|
||||
}
|
||||
ext = ext_map.get(mime, ".jpg")
|
||||
return cache_image_from_bytes(data, ext)
|
||||
|
||||
if mime.startswith("audio/"):
|
||||
ext_map = {
|
||||
"audio/mp3": ".mp3",
|
||||
"audio/mpeg": ".mp3",
|
||||
"audio/ogg": ".ogg",
|
||||
"audio/wav": ".wav",
|
||||
"audio/x-caf": ".mp3",
|
||||
"audio/mp4": ".m4a",
|
||||
"audio/aac": ".m4a",
|
||||
}
|
||||
ext = ext_map.get(mime, ".mp3")
|
||||
return cache_audio_from_bytes(data, ext)
|
||||
|
||||
# Videos, documents, and everything else
|
||||
filename = transfer_name or f"file_{uuid.uuid4().hex[:8]}"
|
||||
return cache_document_from_bytes(data, filename)
|
||||
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"[bluebubbles] failed to download attachment %s: %s",
|
||||
_redact(att_guid),
|
||||
exc,
|
||||
)
|
||||
return None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Webhook handling
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _extract_payload_record(
|
||||
self, payload: Dict[str, Any]
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
data = payload.get("data")
|
||||
if isinstance(data, dict):
|
||||
return data
|
||||
if isinstance(data, list):
|
||||
for item in data:
|
||||
if isinstance(item, dict):
|
||||
return item
|
||||
if isinstance(payload.get("message"), dict):
|
||||
return payload.get("message")
|
||||
return payload if isinstance(payload, dict) else None
|
||||
|
||||
@staticmethod
|
||||
def _value(*candidates: Any) -> Optional[str]:
|
||||
for candidate in candidates:
|
||||
if isinstance(candidate, str) and candidate.strip():
|
||||
return candidate.strip()
|
||||
return None
|
||||
|
||||
async def _handle_webhook(self, request):
|
||||
from aiohttp import web
|
||||
|
||||
token = (
|
||||
request.query.get("password")
|
||||
or request.query.get("guid")
|
||||
or request.headers.get("x-password")
|
||||
or request.headers.get("x-guid")
|
||||
or request.headers.get("x-bluebubbles-guid")
|
||||
)
|
||||
if token != self.password:
|
||||
return web.json_response({"error": "unauthorized"}, status=401)
|
||||
try:
|
||||
raw = await request.read()
|
||||
body = raw.decode("utf-8", errors="replace")
|
||||
try:
|
||||
payload = json.loads(body)
|
||||
except Exception:
|
||||
from urllib.parse import parse_qs
|
||||
|
||||
form = parse_qs(body)
|
||||
payload_str = (
|
||||
form.get("payload")
|
||||
or form.get("data")
|
||||
or form.get("message")
|
||||
or [""]
|
||||
)[0]
|
||||
payload = json.loads(payload_str) if payload_str else {}
|
||||
except Exception as exc:
|
||||
logger.error("[bluebubbles] webhook parse error: %s", exc)
|
||||
return web.json_response({"error": "invalid payload"}, status=400)
|
||||
|
||||
event_type = self._value(payload.get("type"), payload.get("event")) or ""
|
||||
# Only process message events; silently acknowledge everything else
|
||||
if event_type and event_type not in _MESSAGE_EVENTS:
|
||||
return web.Response(text="ok")
|
||||
|
||||
record = self._extract_payload_record(payload) or {}
|
||||
is_from_me = bool(
|
||||
record.get("isFromMe")
|
||||
or record.get("fromMe")
|
||||
or record.get("is_from_me")
|
||||
)
|
||||
if is_from_me:
|
||||
return web.Response(text="ok")
|
||||
|
||||
# Skip tapback reactions delivered as messages
|
||||
assoc_type = record.get("associatedMessageType")
|
||||
if isinstance(assoc_type, int) and assoc_type in {
|
||||
**_TAPBACK_ADDED,
|
||||
**_TAPBACK_REMOVED,
|
||||
}:
|
||||
return web.Response(text="ok")
|
||||
|
||||
text = (
|
||||
self._value(
|
||||
record.get("text"), record.get("message"), record.get("body")
|
||||
)
|
||||
or ""
|
||||
)
|
||||
|
||||
# --- Inbound attachment handling ---
|
||||
attachments = record.get("attachments") or []
|
||||
media_urls: List[str] = []
|
||||
media_types: List[str] = []
|
||||
msg_type = MessageType.TEXT
|
||||
|
||||
for att in attachments:
|
||||
att_guid = att.get("guid", "")
|
||||
if not att_guid:
|
||||
continue
|
||||
cached = await self._download_attachment(att_guid, att)
|
||||
if cached:
|
||||
mime = (att.get("mimeType") or "").lower()
|
||||
media_urls.append(cached)
|
||||
media_types.append(mime)
|
||||
if mime.startswith("image/"):
|
||||
msg_type = MessageType.PHOTO
|
||||
elif mime.startswith("audio/") or (att.get("uti") or "").endswith(
|
||||
"caf"
|
||||
):
|
||||
msg_type = MessageType.VOICE
|
||||
elif mime.startswith("video/"):
|
||||
msg_type = MessageType.VIDEO
|
||||
else:
|
||||
msg_type = MessageType.DOCUMENT
|
||||
|
||||
# With multiple attachments, prefer PHOTO if any images present
|
||||
if len(media_urls) > 1:
|
||||
mime_prefixes = {(m or "").split("/")[0] for m in media_types}
|
||||
if "image" in mime_prefixes:
|
||||
msg_type = MessageType.PHOTO
|
||||
|
||||
if not text and media_urls:
|
||||
text = "(attachment)"
|
||||
# --- End attachment handling ---
|
||||
|
||||
chat_guid = self._value(
|
||||
record.get("chatGuid"),
|
||||
payload.get("chatGuid"),
|
||||
record.get("chat_guid"),
|
||||
payload.get("chat_guid"),
|
||||
payload.get("guid"),
|
||||
)
|
||||
# Fallback: BlueBubbles v1.9+ webhook payloads omit top-level chatGuid;
|
||||
# the chat GUID is nested under data.chats[0].guid instead.
|
||||
if not chat_guid:
|
||||
_chats = record.get("chats") or []
|
||||
if _chats and isinstance(_chats[0], dict):
|
||||
chat_guid = _chats[0].get("guid") or _chats[0].get("chatGuid")
|
||||
chat_identifier = self._value(
|
||||
record.get("chatIdentifier"),
|
||||
record.get("identifier"),
|
||||
payload.get("chatIdentifier"),
|
||||
payload.get("identifier"),
|
||||
)
|
||||
sender = (
|
||||
self._value(
|
||||
record.get("handle", {}).get("address")
|
||||
if isinstance(record.get("handle"), dict)
|
||||
else None,
|
||||
record.get("sender"),
|
||||
record.get("from"),
|
||||
record.get("address"),
|
||||
)
|
||||
or chat_identifier
|
||||
or chat_guid
|
||||
)
|
||||
if not (chat_guid or chat_identifier) and sender:
|
||||
chat_identifier = sender
|
||||
if not sender or not (chat_guid or chat_identifier) or not text:
|
||||
return web.json_response({"error": "missing message fields"}, status=400)
|
||||
|
||||
session_chat_id = chat_guid or chat_identifier
|
||||
is_group = bool(record.get("isGroup")) or (";+;" in (chat_guid or ""))
|
||||
source = self.build_source(
|
||||
chat_id=session_chat_id,
|
||||
chat_name=chat_identifier or sender,
|
||||
chat_type="group" if is_group else "dm",
|
||||
user_id=sender,
|
||||
user_name=sender,
|
||||
chat_id_alt=chat_identifier,
|
||||
)
|
||||
event = MessageEvent(
|
||||
text=text,
|
||||
message_type=msg_type,
|
||||
source=source,
|
||||
raw_message=payload,
|
||||
message_id=self._value(
|
||||
record.get("guid"),
|
||||
record.get("messageGuid"),
|
||||
record.get("id"),
|
||||
),
|
||||
reply_to_message_id=self._value(
|
||||
record.get("threadOriginatorGuid"),
|
||||
record.get("associatedMessageGuid"),
|
||||
),
|
||||
media_urls=media_urls,
|
||||
media_types=media_types,
|
||||
)
|
||||
task = asyncio.create_task(self.handle_message(event))
|
||||
self._background_tasks.add(task)
|
||||
task.add_done_callback(self._background_tasks.discard)
|
||||
|
||||
# Fire-and-forget read receipt
|
||||
if self.send_read_receipts and session_chat_id:
|
||||
asyncio.create_task(self.mark_read(session_chat_id))
|
||||
|
||||
return web.Response(text="ok")
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,773 @@
|
||||
"""
|
||||
Email platform adapter for the Hermes gateway.
|
||||
|
||||
Allows users to interact with Hermes by sending emails.
|
||||
Uses IMAP to receive and SMTP to send messages.
|
||||
|
||||
Environment variables:
|
||||
EMAIL_IMAP_HOST — IMAP server host (e.g., imap.gmail.com)
|
||||
EMAIL_IMAP_PORT — IMAP server port (default: 993)
|
||||
EMAIL_SMTP_HOST — SMTP server host (e.g., smtp.gmail.com)
|
||||
EMAIL_SMTP_PORT — SMTP server port (default: 587)
|
||||
EMAIL_ADDRESS — Email address for the agent
|
||||
EMAIL_PASSWORD — Email password or app-specific password
|
||||
EMAIL_POLL_INTERVAL — Seconds between mailbox checks (default: 15)
|
||||
EMAIL_ALLOWED_USERS — Comma-separated list of allowed sender addresses
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import email as email_lib
|
||||
import imaplib
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import smtplib
|
||||
import ssl
|
||||
import uuid
|
||||
from email.header import decode_header
|
||||
from email.mime.multipart import MIMEMultipart
|
||||
from email.mime.text import MIMEText
|
||||
from email.mime.base import MIMEBase
|
||||
from email.utils import formatdate
|
||||
from email import encoders
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
|
||||
from gateway.platforms.base import (
|
||||
BasePlatformAdapter,
|
||||
MessageEvent,
|
||||
MessageType,
|
||||
SendResult,
|
||||
cache_document_from_bytes,
|
||||
cache_image_from_bytes,
|
||||
)
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
# Automated sender patterns — emails from these are silently ignored
|
||||
_NOREPLY_PATTERNS = (
|
||||
"noreply", "no-reply", "no_reply", "donotreply", "do-not-reply",
|
||||
"mailer-daemon", "postmaster", "bounce", "notifications@",
|
||||
"automated@", "auto-confirm", "auto-reply", "automailer",
|
||||
)
|
||||
|
||||
# RFC headers that indicate bulk/automated mail
|
||||
_AUTOMATED_HEADERS = {
|
||||
"Auto-Submitted": lambda v: v.lower() != "no",
|
||||
"Precedence": lambda v: v.lower() in {"bulk", "list", "junk"},
|
||||
"X-Auto-Response-Suppress": lambda v: bool(v),
|
||||
"List-Unsubscribe": lambda v: bool(v),
|
||||
}
|
||||
|
||||
# Gmail-safe max length per email body
|
||||
MAX_MESSAGE_LENGTH = 50_000
|
||||
|
||||
# Supported image extensions for inline detection
|
||||
_IMAGE_EXTS = {".jpg", ".jpeg", ".png", ".gif", ".webp"}
|
||||
|
||||
def _send_imap_id(imap: "imaplib.IMAP4") -> None:
|
||||
"""Send RFC 2971 IMAP ID command identifying this client.
|
||||
|
||||
Required by 163/NetEase mailbox after LOGIN: without it, every UID
|
||||
SEARCH/FETCH returns ``BYE Unsafe Login`` and disconnects. Other
|
||||
IMAP servers either honor it silently or reject the unknown command;
|
||||
we swallow failures so non-supporting servers keep working.
|
||||
"""
|
||||
try:
|
||||
try:
|
||||
from hermes_cli import __version__ as _hermes_version
|
||||
except Exception: # noqa: BLE001 — keep ID best-effort if import fails
|
||||
_hermes_version = "0"
|
||||
imap.xatom(
|
||||
"ID",
|
||||
f'("name" "hermes-agent" "version" "{_hermes_version}" '
|
||||
'"vendor" "NousResearch" '
|
||||
'"support-email" "noreply@nousresearch.com")',
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 — best-effort, never fatal
|
||||
logger.debug("[Email] IMAP ID command not accepted: %s", e)
|
||||
|
||||
|
||||
def _is_automated_sender(address: str, headers: dict) -> bool:
|
||||
"""Return True if this email is from an automated/noreply source."""
|
||||
addr = address.lower()
|
||||
if any(pattern in addr for pattern in _NOREPLY_PATTERNS):
|
||||
return True
|
||||
for header, check in _AUTOMATED_HEADERS.items():
|
||||
value = headers.get(header, "")
|
||||
if value and check(value):
|
||||
return True
|
||||
return False
|
||||
|
||||
def check_email_requirements() -> bool:
|
||||
"""Check if email platform dependencies are available."""
|
||||
addr = os.getenv("EMAIL_ADDRESS")
|
||||
pwd = os.getenv("EMAIL_PASSWORD")
|
||||
imap = os.getenv("EMAIL_IMAP_HOST")
|
||||
smtp = os.getenv("EMAIL_SMTP_HOST")
|
||||
if not all([addr, pwd, imap, smtp]):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _decode_header_value(raw: str) -> str:
|
||||
"""Decode an RFC 2047 encoded email header into a plain string."""
|
||||
parts = decode_header(raw)
|
||||
decoded = []
|
||||
for part, charset in parts:
|
||||
if isinstance(part, bytes):
|
||||
decoded.append(part.decode(charset or "utf-8", errors="replace"))
|
||||
else:
|
||||
decoded.append(part)
|
||||
return " ".join(decoded)
|
||||
|
||||
|
||||
def _extract_text_body(msg: email_lib.message.Message) -> str:
|
||||
"""Extract the plain-text body from a potentially multipart email."""
|
||||
if msg.is_multipart():
|
||||
for part in msg.walk():
|
||||
content_type = part.get_content_type()
|
||||
disposition = str(part.get("Content-Disposition", ""))
|
||||
# Skip attachments
|
||||
if "attachment" in disposition:
|
||||
continue
|
||||
if content_type == "text/plain":
|
||||
payload = part.get_payload(decode=True)
|
||||
if payload:
|
||||
charset = part.get_content_charset() or "utf-8"
|
||||
return payload.decode(charset, errors="replace")
|
||||
# Fallback: try text/html and strip tags
|
||||
for part in msg.walk():
|
||||
content_type = part.get_content_type()
|
||||
disposition = str(part.get("Content-Disposition", ""))
|
||||
if "attachment" in disposition:
|
||||
continue
|
||||
if content_type == "text/html":
|
||||
payload = part.get_payload(decode=True)
|
||||
if payload:
|
||||
charset = part.get_content_charset() or "utf-8"
|
||||
html = payload.decode(charset, errors="replace")
|
||||
return _strip_html(html)
|
||||
return ""
|
||||
else:
|
||||
payload = msg.get_payload(decode=True)
|
||||
if payload:
|
||||
charset = msg.get_content_charset() or "utf-8"
|
||||
text = payload.decode(charset, errors="replace")
|
||||
if msg.get_content_type() == "text/html":
|
||||
return _strip_html(text)
|
||||
return text
|
||||
return ""
|
||||
|
||||
|
||||
def _strip_html(html: str) -> str:
|
||||
"""Naive HTML tag stripper for fallback text extraction."""
|
||||
text = re.sub(r"<br\s*/?>", "\n", html, flags=re.IGNORECASE)
|
||||
text = re.sub(r"<p[^>]*>", "\n", text, flags=re.IGNORECASE)
|
||||
text = re.sub(r"</p>", "\n", text, flags=re.IGNORECASE)
|
||||
text = re.sub(r"<[^>]+>", "", text)
|
||||
text = re.sub(r" ", " ", text)
|
||||
text = re.sub(r"&", "&", text)
|
||||
text = re.sub(r"<", "<", text)
|
||||
text = re.sub(r">", ">", text)
|
||||
text = re.sub(r"\n{3,}", "\n\n", text)
|
||||
return text.strip()
|
||||
|
||||
|
||||
def _extract_email_address(raw: str) -> str:
|
||||
"""Extract bare email address from 'Name <addr>' format."""
|
||||
match = re.search(r"<([^>]+)>", raw)
|
||||
if match:
|
||||
return match.group(1).strip().lower()
|
||||
return raw.strip().lower()
|
||||
|
||||
|
||||
def _extract_attachments(
|
||||
msg: email_lib.message.Message,
|
||||
skip_attachments: bool = False,
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Extract attachment metadata and cache files locally.
|
||||
|
||||
When *skip_attachments* is True, all attachment/inline parts are ignored
|
||||
(useful for malware protection or bandwidth savings).
|
||||
"""
|
||||
attachments = []
|
||||
if not msg.is_multipart():
|
||||
return attachments
|
||||
|
||||
for part in msg.walk():
|
||||
disposition = str(part.get("Content-Disposition", ""))
|
||||
if skip_attachments and ("attachment" in disposition or "inline" in disposition):
|
||||
continue
|
||||
if "attachment" not in disposition and "inline" not in disposition:
|
||||
continue
|
||||
# Skip text/plain and text/html body parts
|
||||
content_type = part.get_content_type()
|
||||
if content_type in {"text/plain", "text/html"} and "attachment" not in disposition:
|
||||
continue
|
||||
|
||||
filename = part.get_filename()
|
||||
if filename:
|
||||
filename = _decode_header_value(filename)
|
||||
else:
|
||||
ext = part.get_content_subtype() or "bin"
|
||||
filename = f"attachment.{ext}"
|
||||
|
||||
payload = part.get_payload(decode=True)
|
||||
if not payload:
|
||||
continue
|
||||
|
||||
ext = Path(filename).suffix.lower()
|
||||
if ext in _IMAGE_EXTS:
|
||||
try:
|
||||
cached_path = cache_image_from_bytes(payload, ext)
|
||||
except ValueError:
|
||||
logger.debug("Skipping non-image attachment %s (invalid magic bytes)", filename)
|
||||
continue
|
||||
attachments.append({
|
||||
"path": cached_path,
|
||||
"filename": filename,
|
||||
"type": "image",
|
||||
"media_type": content_type,
|
||||
})
|
||||
else:
|
||||
cached_path = cache_document_from_bytes(payload, filename)
|
||||
attachments.append({
|
||||
"path": cached_path,
|
||||
"filename": filename,
|
||||
"type": "document",
|
||||
"media_type": content_type,
|
||||
})
|
||||
|
||||
return attachments
|
||||
|
||||
|
||||
class EmailAdapter(BasePlatformAdapter):
|
||||
"""Email gateway adapter using IMAP (receive) and SMTP (send)."""
|
||||
|
||||
def __init__(self, config: PlatformConfig):
|
||||
super().__init__(config, Platform.EMAIL)
|
||||
|
||||
self._address = os.getenv("EMAIL_ADDRESS", "")
|
||||
self._password = os.getenv("EMAIL_PASSWORD", "")
|
||||
self._imap_host = os.getenv("EMAIL_IMAP_HOST", "")
|
||||
self._imap_port = int(os.getenv("EMAIL_IMAP_PORT", "993"))
|
||||
self._smtp_host = os.getenv("EMAIL_SMTP_HOST", "")
|
||||
self._smtp_port = int(os.getenv("EMAIL_SMTP_PORT", "587"))
|
||||
self._poll_interval = int(os.getenv("EMAIL_POLL_INTERVAL", "15"))
|
||||
|
||||
# Skip attachments — configured via config.yaml:
|
||||
# platforms:
|
||||
# email:
|
||||
# skip_attachments: true
|
||||
extra = config.extra or {}
|
||||
self._skip_attachments = extra.get("skip_attachments", False)
|
||||
|
||||
# Track message IDs we've already processed to avoid duplicates
|
||||
self._seen_uids: set = set()
|
||||
self._seen_uids_max: int = 2000 # cap to prevent unbounded memory growth
|
||||
self._poll_task: Optional[asyncio.Task] = None
|
||||
|
||||
# Map chat_id (sender email) -> last subject + message-id for threading
|
||||
self._thread_context: Dict[str, Dict[str, str]] = {}
|
||||
|
||||
logger.info("[Email] Adapter initialized for %s", self._address)
|
||||
|
||||
def _trim_seen_uids(self) -> None:
|
||||
"""Keep only the most recent UIDs to prevent unbounded memory growth.
|
||||
|
||||
IMAP UIDs are monotonically increasing integers. When the set grows
|
||||
beyond the cap, we keep only the highest half — old UIDs are safe to
|
||||
drop because new messages always have higher UIDs and IMAP's UNSEEN
|
||||
flag prevents re-delivery regardless.
|
||||
"""
|
||||
if len(self._seen_uids) <= self._seen_uids_max:
|
||||
return
|
||||
try:
|
||||
# UIDs are bytes like b'1234' — sort numerically and keep top half
|
||||
sorted_uids = sorted(self._seen_uids, key=lambda u: int(u))
|
||||
keep = self._seen_uids_max // 2
|
||||
self._seen_uids = set(sorted_uids[-keep:])
|
||||
logger.debug("[Email] Trimmed seen UIDs to %d entries", len(self._seen_uids))
|
||||
except (ValueError, TypeError):
|
||||
# Fallback: just clear old entries if sort fails
|
||||
self._seen_uids = set(list(self._seen_uids)[-self._seen_uids_max // 2:])
|
||||
|
||||
async def connect(self) -> bool:
|
||||
"""Connect to the IMAP server and start polling for new messages."""
|
||||
try:
|
||||
# Test IMAP connection
|
||||
imap = imaplib.IMAP4_SSL(self._imap_host, self._imap_port, timeout=30)
|
||||
imap.login(self._address, self._password)
|
||||
_send_imap_id(imap)
|
||||
# Mark all existing messages as seen so we only process new ones
|
||||
imap.select("INBOX")
|
||||
status, data = imap.uid("search", None, "ALL")
|
||||
if status == "OK" and data and data[0]:
|
||||
for uid in data[0].split():
|
||||
self._seen_uids.add(uid)
|
||||
# Keep only the most recent UIDs to prevent unbounded growth
|
||||
self._trim_seen_uids()
|
||||
imap.logout()
|
||||
logger.info("[Email] IMAP connection test passed. %d existing messages skipped.", len(self._seen_uids))
|
||||
except Exception as e:
|
||||
logger.error("[Email] IMAP connection failed: %s", e)
|
||||
return False
|
||||
|
||||
try:
|
||||
# Test SMTP connection
|
||||
smtp = smtplib.SMTP(self._smtp_host, self._smtp_port, timeout=30)
|
||||
smtp.starttls(context=ssl.create_default_context())
|
||||
smtp.login(self._address, self._password)
|
||||
smtp.quit()
|
||||
logger.info("[Email] SMTP connection test passed.")
|
||||
except Exception as e:
|
||||
logger.error("[Email] SMTP connection failed: %s", e)
|
||||
return False
|
||||
|
||||
self._running = True
|
||||
self._poll_task = asyncio.create_task(self._poll_loop())
|
||||
print(f"[Email] Connected as {self._address}")
|
||||
return True
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
"""Stop polling and disconnect."""
|
||||
self._running = False
|
||||
if self._poll_task:
|
||||
self._poll_task.cancel()
|
||||
try:
|
||||
await self._poll_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self._poll_task = None
|
||||
logger.info("[Email] Disconnected.")
|
||||
|
||||
async def _poll_loop(self) -> None:
|
||||
"""Poll IMAP for new messages at regular intervals."""
|
||||
while self._running:
|
||||
try:
|
||||
await self._check_inbox()
|
||||
except asyncio.CancelledError:
|
||||
break
|
||||
except Exception as e:
|
||||
logger.error("[Email] Poll error: %s", e)
|
||||
await asyncio.sleep(self._poll_interval)
|
||||
|
||||
async def _check_inbox(self) -> None:
|
||||
"""Check INBOX for unseen messages and dispatch them."""
|
||||
# Run IMAP operations in a thread to avoid blocking the event loop
|
||||
loop = asyncio.get_running_loop()
|
||||
messages = await loop.run_in_executor(None, self._fetch_new_messages)
|
||||
for msg_data in messages:
|
||||
await self._dispatch_message(msg_data)
|
||||
|
||||
def _fetch_new_messages(self) -> List[Dict[str, Any]]:
|
||||
"""Fetch new (unseen) messages from IMAP. Runs in executor thread."""
|
||||
results = []
|
||||
try:
|
||||
imap = imaplib.IMAP4_SSL(self._imap_host, self._imap_port, timeout=30)
|
||||
try:
|
||||
imap.login(self._address, self._password)
|
||||
_send_imap_id(imap)
|
||||
imap.select("INBOX")
|
||||
|
||||
status, data = imap.uid("search", None, "UNSEEN")
|
||||
if status != "OK" or not data or not data[0]:
|
||||
return results
|
||||
|
||||
for uid in data[0].split():
|
||||
if uid in self._seen_uids:
|
||||
continue
|
||||
self._seen_uids.add(uid)
|
||||
# Trim periodically to prevent unbounded memory growth
|
||||
if len(self._seen_uids) > self._seen_uids_max:
|
||||
self._trim_seen_uids()
|
||||
|
||||
status, msg_data = imap.uid("fetch", uid, "(RFC822)")
|
||||
if status != "OK":
|
||||
continue
|
||||
|
||||
raw_email = msg_data[0][1]
|
||||
msg = email_lib.message_from_bytes(raw_email)
|
||||
|
||||
sender_raw = msg.get("From", "")
|
||||
sender_addr = _extract_email_address(sender_raw)
|
||||
sender_name = _decode_header_value(sender_raw)
|
||||
# Remove email from name if present
|
||||
if "<" in sender_name:
|
||||
sender_name = sender_name.split("<")[0].strip().strip('"')
|
||||
|
||||
subject = _decode_header_value(msg.get("Subject", "(no subject)"))
|
||||
message_id = msg.get("Message-ID", "")
|
||||
in_reply_to = msg.get("In-Reply-To", "")
|
||||
# Skip automated/noreply senders before any processing
|
||||
msg_headers = dict(msg.items())
|
||||
if _is_automated_sender(sender_addr, msg_headers):
|
||||
logger.debug("[Email] Skipping automated sender: %s", sender_addr)
|
||||
continue
|
||||
body = _extract_text_body(msg)
|
||||
attachments = _extract_attachments(msg, skip_attachments=self._skip_attachments)
|
||||
|
||||
results.append({
|
||||
"uid": uid,
|
||||
"sender_addr": sender_addr,
|
||||
"sender_name": sender_name,
|
||||
"subject": subject,
|
||||
"message_id": message_id,
|
||||
"in_reply_to": in_reply_to,
|
||||
"body": body,
|
||||
"attachments": attachments,
|
||||
"date": msg.get("Date", ""),
|
||||
})
|
||||
finally:
|
||||
try:
|
||||
imap.logout()
|
||||
except Exception:
|
||||
pass
|
||||
except Exception as e:
|
||||
logger.error("[Email] IMAP fetch error: %s", e)
|
||||
return results
|
||||
|
||||
async def _dispatch_message(self, msg_data: Dict[str, Any]) -> None:
|
||||
"""Convert a fetched email into a MessageEvent and dispatch it."""
|
||||
sender_addr = msg_data["sender_addr"]
|
||||
|
||||
# Skip self-messages
|
||||
if sender_addr == self._address.lower():
|
||||
return
|
||||
|
||||
# Never reply to automated senders
|
||||
if _is_automated_sender(sender_addr, {}):
|
||||
logger.debug("[Email] Dropping automated sender at dispatch: %s", sender_addr)
|
||||
return
|
||||
|
||||
# Skip senders not in EMAIL_ALLOWED_USERS — prevents the adapter
|
||||
# from creating a MessageEvent (and thus thread context) for senders
|
||||
# that the gateway will never authorize. Without this early guard,
|
||||
# a race between dispatch and authorization can result in the adapter
|
||||
# sending a reply even though the handler returned None.
|
||||
allowed_raw = os.getenv("EMAIL_ALLOWED_USERS", "").strip()
|
||||
if allowed_raw:
|
||||
allowed = {addr.strip().lower() for addr in allowed_raw.split(",") if addr.strip()}
|
||||
if sender_addr.lower() not in allowed:
|
||||
logger.debug("[Email] Dropping non-allowlisted sender at dispatch: %s", sender_addr)
|
||||
return
|
||||
|
||||
subject = msg_data["subject"]
|
||||
body = msg_data["body"].strip()
|
||||
attachments = msg_data["attachments"]
|
||||
|
||||
# Build message text: include subject as context
|
||||
text = body
|
||||
if subject and not subject.startswith("Re:"):
|
||||
text = f"[Subject: {subject}]\n\n{body}"
|
||||
|
||||
# Determine message type and media
|
||||
media_urls = []
|
||||
media_types = []
|
||||
msg_type = MessageType.TEXT
|
||||
|
||||
for att in attachments:
|
||||
media_urls.append(att["path"])
|
||||
media_types.append(att["media_type"])
|
||||
if att["type"] == "image":
|
||||
msg_type = MessageType.PHOTO
|
||||
|
||||
# Store thread context for reply threading
|
||||
self._thread_context[sender_addr] = {
|
||||
"subject": subject,
|
||||
"message_id": msg_data["message_id"],
|
||||
}
|
||||
|
||||
source = self.build_source(
|
||||
chat_id=sender_addr,
|
||||
chat_name=msg_data["sender_name"] or sender_addr,
|
||||
chat_type="dm",
|
||||
user_id=sender_addr,
|
||||
user_name=msg_data["sender_name"] or sender_addr,
|
||||
)
|
||||
|
||||
event = MessageEvent(
|
||||
text=text or "(empty email)",
|
||||
message_type=msg_type,
|
||||
source=source,
|
||||
message_id=msg_data["message_id"],
|
||||
media_urls=media_urls,
|
||||
media_types=media_types,
|
||||
reply_to_message_id=msg_data["in_reply_to"] or None,
|
||||
)
|
||||
|
||||
logger.info("[Email] New message from %s: %s", sender_addr, subject)
|
||||
await self.handle_message(event)
|
||||
|
||||
async def send(
|
||||
self,
|
||||
chat_id: str,
|
||||
content: str,
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
"""Send an email reply to the given address."""
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
message_id = await loop.run_in_executor(
|
||||
None, self._send_email, chat_id, content, reply_to
|
||||
)
|
||||
return SendResult(success=True, message_id=message_id)
|
||||
except Exception as e:
|
||||
logger.error("[Email] Send failed to %s: %s", chat_id, e)
|
||||
return SendResult(success=False, error=str(e))
|
||||
|
||||
def _send_email(
|
||||
self,
|
||||
to_addr: str,
|
||||
body: str,
|
||||
reply_to_msg_id: Optional[str] = None,
|
||||
) -> str:
|
||||
"""Send an email via SMTP. Runs in executor thread."""
|
||||
msg = MIMEMultipart()
|
||||
msg["From"] = self._address
|
||||
msg["To"] = to_addr
|
||||
|
||||
# Thread context for reply
|
||||
ctx = self._thread_context.get(to_addr, {})
|
||||
subject = ctx.get("subject", "Hermes Agent")
|
||||
if not subject.startswith("Re:"):
|
||||
subject = f"Re: {subject}"
|
||||
msg["Subject"] = subject
|
||||
|
||||
# Threading headers
|
||||
original_msg_id = reply_to_msg_id or ctx.get("message_id")
|
||||
if original_msg_id:
|
||||
msg["In-Reply-To"] = original_msg_id
|
||||
msg["References"] = original_msg_id
|
||||
|
||||
msg["Date"] = formatdate(localtime=True)
|
||||
msg_id = f"<hermes-{uuid.uuid4().hex[:12]}@{self._address.split('@')[1]}>"
|
||||
msg["Message-ID"] = msg_id
|
||||
|
||||
msg.attach(MIMEText(body, "plain", "utf-8"))
|
||||
|
||||
smtp = smtplib.SMTP(self._smtp_host, self._smtp_port, timeout=30)
|
||||
try:
|
||||
smtp.starttls(context=ssl.create_default_context())
|
||||
smtp.login(self._address, self._password)
|
||||
smtp.send_message(msg)
|
||||
finally:
|
||||
try:
|
||||
smtp.quit()
|
||||
except Exception:
|
||||
smtp.close()
|
||||
|
||||
logger.info("[Email] Sent reply to %s (subject: %s)", to_addr, subject)
|
||||
return msg_id
|
||||
|
||||
async def send_typing(self, chat_id: str, metadata: Optional[Dict[str, Any]] = None) -> None:
|
||||
"""Email has no typing indicator — no-op."""
|
||||
|
||||
async def send_image(
|
||||
self,
|
||||
chat_id: str,
|
||||
image_url: str,
|
||||
caption: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
) -> SendResult:
|
||||
"""Send an image URL as part of an email body."""
|
||||
text = caption or ""
|
||||
text += f"\n\nImage: {image_url}"
|
||||
return await self.send(chat_id, text.strip(), reply_to)
|
||||
|
||||
async def send_multiple_images(
|
||||
self,
|
||||
chat_id: str,
|
||||
images: List[Tuple[str, str]],
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
human_delay: float = 0.0,
|
||||
) -> None:
|
||||
"""Send a batch of images as a single email with multiple MIME attachments.
|
||||
|
||||
Local files are attached directly. URL images have their URL
|
||||
appended to the body (email adapter does not download remote
|
||||
images). No hard cap — email clients handle dozens of
|
||||
attachments fine, subject to SMTP message size limits.
|
||||
"""
|
||||
if not images:
|
||||
return
|
||||
|
||||
from urllib.parse import unquote as _unquote
|
||||
|
||||
body_parts: List[str] = []
|
||||
local_paths: List[str] = []
|
||||
for image_url, alt_text in images:
|
||||
if alt_text:
|
||||
body_parts.append(alt_text)
|
||||
if image_url.startswith("file://"):
|
||||
local_path = _unquote(image_url[7:])
|
||||
if Path(local_path).exists():
|
||||
local_paths.append(local_path)
|
||||
else:
|
||||
logger.warning("[Email] Skipping missing image: %s", local_path)
|
||||
else:
|
||||
# Remote URLs just get linked in the body (parity with send_image)
|
||||
body_parts.append(f"Image: {image_url}")
|
||||
|
||||
if not local_paths and not body_parts:
|
||||
return
|
||||
|
||||
body = "\n\n".join(body_parts)
|
||||
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
await loop.run_in_executor(
|
||||
None,
|
||||
self._send_email_with_attachments,
|
||||
chat_id,
|
||||
body,
|
||||
local_paths,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error("[Email] Multi-image send failed, falling back: %s", e, exc_info=True)
|
||||
await super().send_multiple_images(chat_id, images, metadata, human_delay)
|
||||
|
||||
def _send_email_with_attachments(
|
||||
self,
|
||||
to_addr: str,
|
||||
body: str,
|
||||
file_paths: List[str],
|
||||
) -> str:
|
||||
"""Send an email with multiple file attachments via SMTP."""
|
||||
msg = MIMEMultipart()
|
||||
msg["From"] = self._address
|
||||
msg["To"] = to_addr
|
||||
|
||||
ctx = self._thread_context.get(to_addr, {})
|
||||
subject = ctx.get("subject", "Hermes Agent")
|
||||
if not subject.startswith("Re:"):
|
||||
subject = f"Re: {subject}"
|
||||
msg["Subject"] = subject
|
||||
|
||||
original_msg_id = ctx.get("message_id")
|
||||
if original_msg_id:
|
||||
msg["In-Reply-To"] = original_msg_id
|
||||
msg["References"] = original_msg_id
|
||||
|
||||
msg["Date"] = formatdate(localtime=True)
|
||||
msg_id = f"<hermes-{uuid.uuid4().hex[:12]}@{self._address.split('@')[1]}>"
|
||||
msg["Message-ID"] = msg_id
|
||||
|
||||
if body:
|
||||
msg.attach(MIMEText(body, "plain", "utf-8"))
|
||||
|
||||
for file_path in file_paths:
|
||||
p = Path(file_path)
|
||||
try:
|
||||
with open(p, "rb") as f:
|
||||
part = MIMEBase("application", "octet-stream")
|
||||
part.set_payload(f.read())
|
||||
encoders.encode_base64(part)
|
||||
part.add_header("Content-Disposition", f"attachment; filename={p.name}")
|
||||
msg.attach(part)
|
||||
except Exception as e:
|
||||
logger.warning("[Email] Failed to attach %s: %s", file_path, e)
|
||||
|
||||
smtp = smtplib.SMTP(self._smtp_host, self._smtp_port, timeout=30)
|
||||
try:
|
||||
smtp.starttls(context=ssl.create_default_context())
|
||||
smtp.login(self._address, self._password)
|
||||
smtp.send_message(msg)
|
||||
finally:
|
||||
try:
|
||||
smtp.quit()
|
||||
except Exception:
|
||||
smtp.close()
|
||||
|
||||
logger.info("[Email] Sent multi-attachment email to %s (%d files)", to_addr, len(file_paths))
|
||||
return msg_id
|
||||
|
||||
async def send_document(
|
||||
self,
|
||||
chat_id: str,
|
||||
file_path: str,
|
||||
caption: Optional[str] = None,
|
||||
file_name: Optional[str] = None,
|
||||
reply_to: Optional[str] = None,
|
||||
**kwargs,
|
||||
) -> SendResult:
|
||||
"""Send a file as an email attachment."""
|
||||
try:
|
||||
loop = asyncio.get_running_loop()
|
||||
message_id = await loop.run_in_executor(
|
||||
None,
|
||||
self._send_email_with_attachment,
|
||||
chat_id,
|
||||
caption or "",
|
||||
file_path,
|
||||
file_name,
|
||||
)
|
||||
return SendResult(success=True, message_id=message_id)
|
||||
except Exception as e:
|
||||
logger.error("[Email] Send document failed: %s", e)
|
||||
return SendResult(success=False, error=str(e))
|
||||
|
||||
def _send_email_with_attachment(
|
||||
self,
|
||||
to_addr: str,
|
||||
body: str,
|
||||
file_path: str,
|
||||
file_name: Optional[str] = None,
|
||||
) -> str:
|
||||
"""Send an email with a file attachment via SMTP."""
|
||||
msg = MIMEMultipart()
|
||||
msg["From"] = self._address
|
||||
msg["To"] = to_addr
|
||||
|
||||
ctx = self._thread_context.get(to_addr, {})
|
||||
subject = ctx.get("subject", "Hermes Agent")
|
||||
if not subject.startswith("Re:"):
|
||||
subject = f"Re: {subject}"
|
||||
msg["Subject"] = subject
|
||||
|
||||
original_msg_id = ctx.get("message_id")
|
||||
if original_msg_id:
|
||||
msg["In-Reply-To"] = original_msg_id
|
||||
msg["References"] = original_msg_id
|
||||
|
||||
msg["Date"] = formatdate(localtime=True)
|
||||
msg_id = f"<hermes-{uuid.uuid4().hex[:12]}@{self._address.split('@')[1]}>"
|
||||
msg["Message-ID"] = msg_id
|
||||
|
||||
if body:
|
||||
msg.attach(MIMEText(body, "plain", "utf-8"))
|
||||
|
||||
# Attach file
|
||||
p = Path(file_path)
|
||||
fname = file_name or p.name
|
||||
with open(p, "rb") as f:
|
||||
part = MIMEBase("application", "octet-stream")
|
||||
part.set_payload(f.read())
|
||||
encoders.encode_base64(part)
|
||||
part.add_header("Content-Disposition", f"attachment; filename={fname}")
|
||||
msg.attach(part)
|
||||
|
||||
smtp = smtplib.SMTP(self._smtp_host, self._smtp_port, timeout=30)
|
||||
try:
|
||||
smtp.starttls(context=ssl.create_default_context())
|
||||
smtp.login(self._address, self._password)
|
||||
smtp.send_message(msg)
|
||||
finally:
|
||||
try:
|
||||
smtp.quit()
|
||||
except Exception:
|
||||
smtp.close()
|
||||
|
||||
return msg_id
|
||||
|
||||
async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
|
||||
"""Return basic info about the email chat."""
|
||||
ctx = self._thread_context.get(chat_id, {})
|
||||
return {
|
||||
"name": chat_id,
|
||||
"type": "dm",
|
||||
"chat_id": chat_id,
|
||||
"subject": ctx.get("subject", ""),
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,429 @@
|
||||
"""
|
||||
Feishu document comment access-control rules.
|
||||
|
||||
3-tier rule resolution: exact doc > wildcard "*" > top-level > code defaults.
|
||||
Each field (enabled/policy/allow_from) falls back independently.
|
||||
Config: ~/.hermes/feishu_comment_rules.json (mtime-cached, hot-reload).
|
||||
Pairing store: ~/.hermes/feishu_comment_pairing.json.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from hermes_constants import get_hermes_home
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Paths
|
||||
# ---------------------------------------------------------------------------
|
||||
#
|
||||
# Uses the canonical ``get_hermes_home()`` helper (HERMES_HOME-aware and
|
||||
# profile-safe). Resolved at import time; this module is lazy-imported by
|
||||
# the Feishu comment event handler, which runs long after profile overrides
|
||||
# have been applied, so freezing paths here is safe.
|
||||
|
||||
RULES_FILE = get_hermes_home() / "feishu_comment_rules.json"
|
||||
PAIRING_FILE = get_hermes_home() / "feishu_comment_pairing.json"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Data models
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_VALID_POLICIES = ("allowlist", "pairing")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CommentDocumentRule:
|
||||
"""Per-document rule. ``None`` means 'inherit from lower tier'."""
|
||||
enabled: Optional[bool] = None
|
||||
policy: Optional[str] = None
|
||||
allow_from: Optional[frozenset] = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CommentsConfig:
|
||||
"""Top-level comment access config."""
|
||||
enabled: bool = True
|
||||
policy: str = "pairing"
|
||||
allow_from: frozenset = field(default_factory=frozenset)
|
||||
documents: Dict[str, CommentDocumentRule] = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ResolvedCommentRule:
|
||||
"""Fully resolved rule after field-by-field fallback."""
|
||||
enabled: bool
|
||||
policy: str
|
||||
allow_from: frozenset
|
||||
match_source: str # e.g. "exact:docx:xxx" | "wildcard" | "top" | "default"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Mtime-cached file loading
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class _MtimeCache:
|
||||
"""Generic mtime-based file cache. ``stat()`` per access, re-read only on change."""
|
||||
|
||||
def __init__(self, path: Path):
|
||||
self._path = path
|
||||
self._mtime: float = 0.0
|
||||
self._data: Optional[dict] = None
|
||||
|
||||
def load(self) -> dict:
|
||||
try:
|
||||
st = self._path.stat()
|
||||
mtime = st.st_mtime
|
||||
except FileNotFoundError:
|
||||
self._mtime = 0.0
|
||||
self._data = {}
|
||||
return {}
|
||||
|
||||
if mtime == self._mtime and self._data is not None:
|
||||
return self._data
|
||||
|
||||
try:
|
||||
with open(self._path, "r", encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
if not isinstance(data, dict):
|
||||
data = {}
|
||||
except (json.JSONDecodeError, OSError):
|
||||
logger.warning("[Feishu-Rules] Failed to read %s, using empty config", self._path)
|
||||
data = {}
|
||||
|
||||
self._mtime = mtime
|
||||
self._data = data
|
||||
return data
|
||||
|
||||
|
||||
_rules_cache = _MtimeCache(RULES_FILE)
|
||||
_pairing_cache = _MtimeCache(PAIRING_FILE)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Config parsing
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _parse_frozenset(raw: Any) -> Optional[frozenset]:
|
||||
"""Parse a list of strings into a frozenset; return None if key absent."""
|
||||
if raw is None:
|
||||
return None
|
||||
if isinstance(raw, (list, tuple)):
|
||||
return frozenset(str(u).strip() for u in raw if str(u).strip())
|
||||
return None
|
||||
|
||||
|
||||
def _parse_document_rule(raw: dict) -> CommentDocumentRule:
|
||||
enabled = raw.get("enabled")
|
||||
if enabled is not None:
|
||||
enabled = bool(enabled)
|
||||
policy = raw.get("policy")
|
||||
if policy is not None:
|
||||
policy = str(policy).strip().lower()
|
||||
if policy not in _VALID_POLICIES:
|
||||
policy = None
|
||||
allow_from = _parse_frozenset(raw.get("allow_from"))
|
||||
return CommentDocumentRule(enabled=enabled, policy=policy, allow_from=allow_from)
|
||||
|
||||
|
||||
def load_config() -> CommentsConfig:
|
||||
"""Load comment rules from disk (mtime-cached)."""
|
||||
raw = _rules_cache.load()
|
||||
if not raw:
|
||||
return CommentsConfig()
|
||||
|
||||
documents: Dict[str, CommentDocumentRule] = {}
|
||||
raw_docs = raw.get("documents", {})
|
||||
if isinstance(raw_docs, dict):
|
||||
for key, rule_raw in raw_docs.items():
|
||||
if isinstance(rule_raw, dict):
|
||||
documents[str(key)] = _parse_document_rule(rule_raw)
|
||||
|
||||
policy = str(raw.get("policy", "pairing")).strip().lower()
|
||||
if policy not in _VALID_POLICIES:
|
||||
policy = "pairing"
|
||||
|
||||
return CommentsConfig(
|
||||
enabled=raw.get("enabled", True),
|
||||
policy=policy,
|
||||
allow_from=_parse_frozenset(raw.get("allow_from")) or frozenset(),
|
||||
documents=documents,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Rule resolution (§8.4 field-by-field fallback)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def has_wiki_keys(cfg: CommentsConfig) -> bool:
|
||||
"""Check if any document rule key starts with 'wiki:'."""
|
||||
return any(k.startswith("wiki:") for k in cfg.documents)
|
||||
|
||||
|
||||
def resolve_rule(
|
||||
cfg: CommentsConfig,
|
||||
file_type: str,
|
||||
file_token: str,
|
||||
wiki_token: str = "",
|
||||
) -> ResolvedCommentRule:
|
||||
"""Resolve effective rule: exact doc → wiki key → wildcard → top-level → defaults."""
|
||||
exact_key = f"{file_type}:{file_token}"
|
||||
|
||||
exact = cfg.documents.get(exact_key)
|
||||
exact_src = f"exact:{exact_key}"
|
||||
if exact is None and wiki_token:
|
||||
wiki_key = f"wiki:{wiki_token}"
|
||||
exact = cfg.documents.get(wiki_key)
|
||||
exact_src = f"exact:{wiki_key}"
|
||||
|
||||
wildcard = cfg.documents.get("*")
|
||||
|
||||
layers = []
|
||||
if exact is not None:
|
||||
layers.append((exact, exact_src))
|
||||
if wildcard is not None:
|
||||
layers.append((wildcard, "wildcard"))
|
||||
|
||||
def _pick(field_name: str):
|
||||
for layer, source in layers:
|
||||
val = getattr(layer, field_name)
|
||||
if val is not None:
|
||||
return val, source
|
||||
return getattr(cfg, field_name), "top"
|
||||
|
||||
enabled, en_src = _pick("enabled")
|
||||
policy, pol_src = _pick("policy")
|
||||
allow_from, _ = _pick("allow_from")
|
||||
|
||||
# match_source = highest-priority tier that contributed any field
|
||||
priority_order = {"exact": 0, "wildcard": 1, "top": 2}
|
||||
best_src = min(
|
||||
[en_src, pol_src],
|
||||
key=lambda s: priority_order.get(s.split(":")[0], 3),
|
||||
)
|
||||
|
||||
return ResolvedCommentRule(
|
||||
enabled=enabled,
|
||||
policy=policy,
|
||||
allow_from=allow_from,
|
||||
match_source=best_src,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pairing store
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _load_pairing_approved() -> set:
|
||||
"""Return set of approved user open_ids (mtime-cached)."""
|
||||
data = _pairing_cache.load()
|
||||
approved = data.get("approved", {})
|
||||
if isinstance(approved, dict):
|
||||
return set(approved.keys())
|
||||
if isinstance(approved, list):
|
||||
return {str(u) for u in approved if u}
|
||||
return set()
|
||||
|
||||
|
||||
def _save_pairing(data: dict) -> None:
|
||||
PAIRING_FILE.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = PAIRING_FILE.with_suffix(".tmp")
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, indent=2, ensure_ascii=False)
|
||||
tmp.replace(PAIRING_FILE)
|
||||
# Invalidate cache so next load picks up change
|
||||
_pairing_cache._mtime = 0.0
|
||||
_pairing_cache._data = None
|
||||
|
||||
|
||||
def pairing_add(user_open_id: str) -> bool:
|
||||
"""Add a user to the pairing-approved list. Returns True if newly added."""
|
||||
data = _pairing_cache.load()
|
||||
approved = data.get("approved", {})
|
||||
if not isinstance(approved, dict):
|
||||
approved = {}
|
||||
if user_open_id in approved:
|
||||
return False
|
||||
approved[user_open_id] = {"approved_at": time.time()}
|
||||
data["approved"] = approved
|
||||
_save_pairing(data)
|
||||
return True
|
||||
|
||||
|
||||
def pairing_remove(user_open_id: str) -> bool:
|
||||
"""Remove a user from the pairing-approved list. Returns True if removed."""
|
||||
data = _pairing_cache.load()
|
||||
approved = data.get("approved", {})
|
||||
if not isinstance(approved, dict):
|
||||
return False
|
||||
if user_open_id not in approved:
|
||||
return False
|
||||
del approved[user_open_id]
|
||||
data["approved"] = approved
|
||||
_save_pairing(data)
|
||||
return True
|
||||
|
||||
|
||||
def pairing_list() -> Dict[str, Any]:
|
||||
"""Return the approved dict {user_open_id: {approved_at: ...}}."""
|
||||
data = _pairing_cache.load()
|
||||
approved = data.get("approved", {})
|
||||
return dict(approved) if isinstance(approved, dict) else {}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Access check (public API for feishu_comment.py)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def is_user_allowed(rule: ResolvedCommentRule, user_open_id: str) -> bool:
|
||||
"""Check if user passes the resolved rule's policy gate."""
|
||||
if user_open_id in rule.allow_from:
|
||||
return True
|
||||
if rule.policy == "pairing":
|
||||
return user_open_id in _load_pairing_approved()
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _print_status() -> None:
|
||||
cfg = load_config()
|
||||
print(f"Rules file: {RULES_FILE}")
|
||||
print(f" exists: {RULES_FILE.exists()}")
|
||||
print(f"Pairing file: {PAIRING_FILE}")
|
||||
print(f" exists: {PAIRING_FILE.exists()}")
|
||||
print()
|
||||
print(f"Top-level:")
|
||||
print(f" enabled: {cfg.enabled}")
|
||||
print(f" policy: {cfg.policy}")
|
||||
print(f" allow_from: {sorted(cfg.allow_from) if cfg.allow_from else '[]'}")
|
||||
print()
|
||||
if cfg.documents:
|
||||
print(f"Document rules ({len(cfg.documents)}):")
|
||||
for key, rule in sorted(cfg.documents.items()):
|
||||
parts = []
|
||||
if rule.enabled is not None:
|
||||
parts.append(f"enabled={rule.enabled}")
|
||||
if rule.policy is not None:
|
||||
parts.append(f"policy={rule.policy}")
|
||||
if rule.allow_from is not None:
|
||||
parts.append(f"allow_from={sorted(rule.allow_from)}")
|
||||
print(f" [{key}] {', '.join(parts) if parts else '(empty — inherits all)'}")
|
||||
else:
|
||||
print("Document rules: (none)")
|
||||
print()
|
||||
approved = pairing_list()
|
||||
print(f"Pairing approved ({len(approved)}):")
|
||||
for uid, meta in sorted(approved.items()):
|
||||
ts = meta.get("approved_at", 0)
|
||||
print(f" {uid} (approved_at={ts})")
|
||||
|
||||
|
||||
def _do_check(doc_key: str, user_open_id: str) -> None:
|
||||
cfg = load_config()
|
||||
parts = doc_key.split(":", 1)
|
||||
if len(parts) != 2:
|
||||
print(f"Error: doc_key must be 'fileType:fileToken', got '{doc_key}'")
|
||||
return
|
||||
file_type, file_token = parts
|
||||
rule = resolve_rule(cfg, file_type, file_token)
|
||||
allowed = is_user_allowed(rule, user_open_id)
|
||||
print(f"Document: {doc_key}")
|
||||
print(f"User: {user_open_id}")
|
||||
print(f"Resolved rule:")
|
||||
print(f" enabled: {rule.enabled}")
|
||||
print(f" policy: {rule.policy}")
|
||||
print(f" allow_from: {sorted(rule.allow_from) if rule.allow_from else '[]'}")
|
||||
print(f" match_source: {rule.match_source}")
|
||||
print(f"Result: {'ALLOWED' if allowed else 'DENIED'}")
|
||||
|
||||
|
||||
def _main() -> int:
|
||||
import sys
|
||||
|
||||
try:
|
||||
from hermes_cli.env_loader import load_hermes_dotenv
|
||||
load_hermes_dotenv()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
usage = (
|
||||
"Usage: python -m gateway.platforms.feishu_comment_rules <command> [args]\n"
|
||||
"\n"
|
||||
"Commands:\n"
|
||||
" status Show rules config and pairing state\n"
|
||||
" check <fileType:token> <user> Simulate access check\n"
|
||||
" pairing add <user_open_id> Add user to pairing-approved list\n"
|
||||
" pairing remove <user_open_id> Remove user from pairing-approved list\n"
|
||||
" pairing list List pairing-approved users\n"
|
||||
"\n"
|
||||
f"Rules config file: {RULES_FILE}\n"
|
||||
" Edit this JSON file directly to configure policies and document rules.\n"
|
||||
" Changes take effect on the next comment event (no restart needed).\n"
|
||||
)
|
||||
|
||||
args = sys.argv[1:]
|
||||
if not args:
|
||||
print(usage)
|
||||
return 1
|
||||
|
||||
cmd = args[0]
|
||||
|
||||
if cmd == "status":
|
||||
_print_status()
|
||||
|
||||
elif cmd == "check":
|
||||
if len(args) < 3:
|
||||
print("Usage: check <fileType:fileToken> <user_open_id>")
|
||||
return 1
|
||||
_do_check(args[1], args[2])
|
||||
|
||||
elif cmd == "pairing":
|
||||
if len(args) < 2:
|
||||
print("Usage: pairing <add|remove|list> [args]")
|
||||
return 1
|
||||
sub = args[1]
|
||||
if sub == "add":
|
||||
if len(args) < 3:
|
||||
print("Usage: pairing add <user_open_id>")
|
||||
return 1
|
||||
if pairing_add(args[2]):
|
||||
print(f"Added: {args[2]}")
|
||||
else:
|
||||
print(f"Already approved: {args[2]}")
|
||||
elif sub == "remove":
|
||||
if len(args) < 3:
|
||||
print("Usage: pairing remove <user_open_id>")
|
||||
return 1
|
||||
if pairing_remove(args[2]):
|
||||
print(f"Removed: {args[2]}")
|
||||
else:
|
||||
print(f"Not in approved list: {args[2]}")
|
||||
elif sub == "list":
|
||||
approved = pairing_list()
|
||||
if not approved:
|
||||
print("(no approved users)")
|
||||
for uid, meta in sorted(approved.items()):
|
||||
print(f" {uid} approved_at={meta.get('approved_at', '?')}")
|
||||
else:
|
||||
print(f"Unknown pairing subcommand: {sub}")
|
||||
return 1
|
||||
else:
|
||||
print(f"Unknown command: {cmd}\n")
|
||||
print(usage)
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
sys.exit(_main())
|
||||
@@ -0,0 +1,278 @@
|
||||
"""Shared helper classes for gateway platform adapters.
|
||||
|
||||
Extracts common patterns that were duplicated across 5-7 adapters:
|
||||
message deduplication, text batch aggregation, markdown stripping,
|
||||
and thread participation tracking.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Dict
|
||||
|
||||
from utils import atomic_json_write
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from gateway.platforms.base import MessageEvent
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ─── Message Deduplication ────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class MessageDeduplicator:
|
||||
"""TTL-based message deduplication cache.
|
||||
|
||||
Replaces the identical ``_seen_messages`` / ``_is_duplicate()`` pattern
|
||||
previously duplicated in discord, slack, dingtalk, wecom, weixin,
|
||||
mattermost, and feishu adapters.
|
||||
|
||||
Usage::
|
||||
|
||||
self._dedup = MessageDeduplicator()
|
||||
|
||||
# In message handler:
|
||||
if self._dedup.is_duplicate(msg_id):
|
||||
return
|
||||
"""
|
||||
|
||||
def __init__(self, max_size: int = 2000, ttl_seconds: float = 300):
|
||||
self._seen: Dict[str, float] = {}
|
||||
self._max_size = max_size
|
||||
self._ttl = ttl_seconds
|
||||
|
||||
def is_duplicate(self, msg_id: str) -> bool:
|
||||
"""Return True if *msg_id* was already seen within the TTL window."""
|
||||
if not msg_id:
|
||||
return False
|
||||
now = time.time()
|
||||
if msg_id in self._seen:
|
||||
if now - self._seen[msg_id] < self._ttl:
|
||||
return True
|
||||
# Entry has expired — remove it and treat as new
|
||||
del self._seen[msg_id]
|
||||
self._seen[msg_id] = now
|
||||
if len(self._seen) > self._max_size:
|
||||
cutoff = now - self._ttl
|
||||
self._seen = {k: v for k, v in self._seen.items() if v > cutoff}
|
||||
if len(self._seen) > self._max_size:
|
||||
# TTL pruning alone does not cap the cache when every entry is
|
||||
# still fresh. Keep the newest entries so the helper's
|
||||
# max_size bound is enforced under sustained traffic.
|
||||
newest = sorted(
|
||||
self._seen.items(),
|
||||
key=lambda item: item[1],
|
||||
)[-self._max_size:]
|
||||
self._seen = dict(newest)
|
||||
return False
|
||||
|
||||
def clear(self):
|
||||
"""Clear all tracked messages."""
|
||||
self._seen.clear()
|
||||
|
||||
|
||||
# ─── Text Batch Aggregation ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
class TextBatchAggregator:
|
||||
"""Aggregates rapid-fire text events into single messages.
|
||||
|
||||
Replaces the ``_enqueue_text_event`` / ``_flush_text_batch`` pattern
|
||||
previously duplicated in telegram, discord, matrix, wecom, and feishu.
|
||||
|
||||
Usage::
|
||||
|
||||
self._text_batcher = TextBatchAggregator(
|
||||
handler=self._message_handler,
|
||||
batch_delay=0.6,
|
||||
split_threshold=1900,
|
||||
)
|
||||
|
||||
# In message dispatch:
|
||||
if msg_type == MessageType.TEXT and self._text_batcher.is_enabled():
|
||||
self._text_batcher.enqueue(event, session_key)
|
||||
return
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
handler,
|
||||
*,
|
||||
batch_delay: float = 0.6,
|
||||
split_delay: float = 2.0,
|
||||
split_threshold: int = 4000,
|
||||
):
|
||||
self._handler = handler
|
||||
self._batch_delay = batch_delay
|
||||
self._split_delay = split_delay
|
||||
self._split_threshold = split_threshold
|
||||
self._pending: Dict[str, "MessageEvent"] = {}
|
||||
self._pending_tasks: Dict[str, asyncio.Task] = {}
|
||||
|
||||
def is_enabled(self) -> bool:
|
||||
"""Return True if batching is active (delay > 0)."""
|
||||
return self._batch_delay > 0
|
||||
|
||||
def enqueue(self, event: "MessageEvent", key: str) -> None:
|
||||
"""Add *event* to the pending batch for *key*."""
|
||||
chunk_len = len(event.text or "")
|
||||
existing = self._pending.get(key)
|
||||
if not existing:
|
||||
event._last_chunk_len = chunk_len # type: ignore[attr-defined]
|
||||
self._pending[key] = event
|
||||
else:
|
||||
existing.text = f"{existing.text}\n{event.text}"
|
||||
existing._last_chunk_len = chunk_len # type: ignore[attr-defined]
|
||||
|
||||
# Cancel prior flush timer, start a new one
|
||||
prior = self._pending_tasks.get(key)
|
||||
if prior and not prior.done():
|
||||
prior.cancel()
|
||||
self._pending_tasks[key] = asyncio.create_task(self._flush(key))
|
||||
|
||||
async def _flush(self, key: str) -> None:
|
||||
"""Wait then dispatch the batched event for *key*."""
|
||||
current_task = self._pending_tasks.get(key)
|
||||
pending = self._pending.get(key)
|
||||
last_len = getattr(pending, "_last_chunk_len", 0) if pending else 0
|
||||
|
||||
# Use longer delay when the last chunk looks like a split message
|
||||
delay = self._split_delay if last_len >= self._split_threshold else self._batch_delay
|
||||
await asyncio.sleep(delay)
|
||||
|
||||
event = self._pending.pop(key, None)
|
||||
if event:
|
||||
try:
|
||||
await self._handler(event)
|
||||
except Exception:
|
||||
logger.exception("[TextBatchAggregator] Error dispatching batched event for %s", key)
|
||||
|
||||
if self._pending_tasks.get(key) is current_task:
|
||||
self._pending_tasks.pop(key, None)
|
||||
|
||||
def cancel_all(self) -> None:
|
||||
"""Cancel all pending flush tasks."""
|
||||
for task in self._pending_tasks.values():
|
||||
if not task.done():
|
||||
task.cancel()
|
||||
self._pending_tasks.clear()
|
||||
self._pending.clear()
|
||||
|
||||
|
||||
# ─── Markdown Stripping ──────────────────────────────────────────────────────
|
||||
|
||||
# Pre-compiled regexes for performance
|
||||
_RE_BOLD = re.compile(r"\*\*(.+?)\*\*", re.DOTALL)
|
||||
_RE_ITALIC_STAR = re.compile(r"\*(.+?)\*", re.DOTALL)
|
||||
_RE_BOLD_UNDER = re.compile(r"\b__(?![\s_])(.+?)(?<![\s_])__\b", re.DOTALL)
|
||||
_RE_ITALIC_UNDER = re.compile(r"\b_(?![\s_])(.+?)(?<![\s_])_\b", re.DOTALL)
|
||||
_RE_CODE_BLOCK = re.compile(r"```[a-zA-Z0-9_+-]*\n?")
|
||||
_RE_INLINE_CODE = re.compile(r"`(.+?)`")
|
||||
_RE_HEADING = re.compile(r"^#{1,6}\s+", re.MULTILINE)
|
||||
_RE_LINK = re.compile(r"\[([^\]]+)\]\([^\)]+\)")
|
||||
_RE_MULTI_NEWLINE = re.compile(r"\n{3,}")
|
||||
|
||||
|
||||
def strip_markdown(text: str) -> str:
|
||||
"""Strip markdown formatting for plain-text platforms (SMS, iMessage, etc.).
|
||||
|
||||
Replaces the identical ``_strip_markdown()`` functions previously
|
||||
duplicated in sms.py, bluebubbles.py, and feishu.py.
|
||||
"""
|
||||
text = _RE_BOLD.sub(r"\1", text)
|
||||
text = _RE_ITALIC_STAR.sub(r"\1", text)
|
||||
text = _RE_BOLD_UNDER.sub(r"\1", text)
|
||||
text = _RE_ITALIC_UNDER.sub(r"\1", text)
|
||||
text = _RE_CODE_BLOCK.sub("", text)
|
||||
text = _RE_INLINE_CODE.sub(r"\1", text)
|
||||
text = _RE_HEADING.sub("", text)
|
||||
text = _RE_LINK.sub(r"\1", text)
|
||||
text = _RE_MULTI_NEWLINE.sub("\n\n", text)
|
||||
return text.strip()
|
||||
|
||||
|
||||
# ─── Thread Participation Tracking ───────────────────────────────────────────
|
||||
|
||||
|
||||
class ThreadParticipationTracker:
|
||||
"""Persistent tracking of threads the bot has participated in.
|
||||
|
||||
Replaces the identical ``_load/_save_participated_threads`` +
|
||||
``_mark_thread_participated`` pattern previously duplicated in
|
||||
discord.py and matrix.py.
|
||||
|
||||
Usage::
|
||||
|
||||
self._threads = ThreadParticipationTracker("discord")
|
||||
|
||||
# Check membership:
|
||||
if thread_id in self._threads:
|
||||
...
|
||||
|
||||
# Mark participation:
|
||||
self._threads.mark(thread_id)
|
||||
"""
|
||||
|
||||
_MAX_TRACKED = 500
|
||||
|
||||
def __init__(self, platform_name: str, max_tracked: int = 500):
|
||||
self._platform = platform_name
|
||||
self._max_tracked = max_tracked
|
||||
self._threads: dict[str, None] = {
|
||||
str(thread_id): None for thread_id in self._load()
|
||||
}
|
||||
|
||||
def _state_path(self) -> Path:
|
||||
from hermes_constants import get_hermes_home
|
||||
return get_hermes_home() / f"{self._platform}_threads.json"
|
||||
|
||||
def _load(self) -> list[str]:
|
||||
path = self._state_path()
|
||||
if path.exists():
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
if isinstance(data, list):
|
||||
return [str(thread_id) for thread_id in data]
|
||||
except Exception:
|
||||
pass
|
||||
return []
|
||||
|
||||
def _save(self) -> None:
|
||||
path = self._state_path()
|
||||
thread_list = list(self._threads)
|
||||
if len(thread_list) > self._max_tracked:
|
||||
thread_list = thread_list[-self._max_tracked:]
|
||||
self._threads = dict.fromkeys(thread_list)
|
||||
atomic_json_write(path, thread_list, indent=None)
|
||||
|
||||
def mark(self, thread_id: str) -> None:
|
||||
"""Mark *thread_id* as participated and persist."""
|
||||
if thread_id not in self._threads:
|
||||
self._threads[thread_id] = None
|
||||
self._save()
|
||||
|
||||
def __contains__(self, thread_id: str) -> bool:
|
||||
return thread_id in self._threads
|
||||
|
||||
def clear(self) -> None:
|
||||
self._threads.clear()
|
||||
|
||||
|
||||
# ─── Phone Number Redaction ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
def redact_phone(phone: str) -> str:
|
||||
"""Redact a phone number for logging, preserving country code and last 4.
|
||||
|
||||
Replaces the identical ``_redact_phone()`` functions in signal.py,
|
||||
sms.py, and bluebubbles.py.
|
||||
"""
|
||||
if not phone:
|
||||
return "<none>"
|
||||
if len(phone) <= 8:
|
||||
return phone[:2] + "****" + phone[-2:] if len(phone) > 4 else "****"
|
||||
return phone[:4] + "****" + phone[-4:]
|
||||
@@ -0,0 +1,449 @@
|
||||
"""
|
||||
Home Assistant platform adapter.
|
||||
|
||||
Connects to the HA WebSocket API for real-time event monitoring.
|
||||
State-change events are converted to MessageEvent objects and forwarded
|
||||
to the agent for processing. Outbound messages are delivered as HA
|
||||
persistent notifications.
|
||||
|
||||
Requires:
|
||||
- aiohttp (already in messaging extras)
|
||||
- HASS_TOKEN env var (Long-Lived Access Token)
|
||||
- HASS_URL env var (default: http://homeassistant.local:8123)
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, Optional, Set
|
||||
|
||||
try:
|
||||
import aiohttp
|
||||
AIOHTTP_AVAILABLE = True
|
||||
except ImportError:
|
||||
AIOHTTP_AVAILABLE = False
|
||||
aiohttp = None # type: ignore[assignment]
|
||||
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
from gateway.platforms.base import (
|
||||
BasePlatformAdapter,
|
||||
MessageEvent,
|
||||
MessageType,
|
||||
SendResult,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def check_ha_requirements() -> bool:
|
||||
"""Check if Home Assistant dependencies are available and configured."""
|
||||
if not AIOHTTP_AVAILABLE:
|
||||
return False
|
||||
if not os.getenv("HASS_TOKEN"):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
class HomeAssistantAdapter(BasePlatformAdapter):
|
||||
"""
|
||||
Home Assistant WebSocket adapter.
|
||||
|
||||
Subscribes to ``state_changed`` events and forwards them as
|
||||
MessageEvent objects. Supports domain/entity filtering and
|
||||
per-entity cooldowns to avoid event floods.
|
||||
"""
|
||||
|
||||
MAX_MESSAGE_LENGTH = 4096
|
||||
|
||||
# Reconnection backoff schedule (seconds)
|
||||
_BACKOFF_STEPS = [5, 10, 30, 60]
|
||||
|
||||
def __init__(self, config: PlatformConfig):
|
||||
super().__init__(config, Platform.HOMEASSISTANT)
|
||||
|
||||
# Connection state
|
||||
self._session: Optional["aiohttp.ClientSession"] = None
|
||||
self._ws: Optional["aiohttp.ClientWebSocketResponse"] = None
|
||||
self._rest_session: Optional["aiohttp.ClientSession"] = None
|
||||
self._listen_task: Optional[asyncio.Task] = None
|
||||
self._msg_id: int = 0
|
||||
|
||||
# Configuration from extra
|
||||
extra = config.extra or {}
|
||||
token = config.token or os.getenv("HASS_TOKEN", "")
|
||||
url = extra.get("url") or os.getenv("HASS_URL", "http://homeassistant.local:8123")
|
||||
self._hass_url: str = url.rstrip("/")
|
||||
self._hass_token: str = token
|
||||
|
||||
# Event filtering
|
||||
self._watch_domains: Set[str] = set(extra.get("watch_domains", []))
|
||||
self._watch_entities: Set[str] = set(extra.get("watch_entities", []))
|
||||
self._ignore_entities: Set[str] = set(extra.get("ignore_entities", []))
|
||||
self._watch_all: bool = bool(extra.get("watch_all", False))
|
||||
self._cooldown_seconds: int = int(extra.get("cooldown_seconds", 30))
|
||||
|
||||
# Cooldown tracking: entity_id -> last_event_timestamp
|
||||
self._last_event_time: Dict[str, float] = {}
|
||||
|
||||
def _next_id(self) -> int:
|
||||
"""Return the next WebSocket message ID."""
|
||||
self._msg_id += 1
|
||||
return self._msg_id
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Connection lifecycle
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def connect(self) -> bool:
|
||||
"""Connect to HA WebSocket API and subscribe to events."""
|
||||
if not AIOHTTP_AVAILABLE:
|
||||
logger.warning("[%s] aiohttp not installed. Run: pip install aiohttp", self.name)
|
||||
return False
|
||||
|
||||
if not self._hass_token:
|
||||
logger.warning("[%s] No HASS_TOKEN configured", self.name)
|
||||
return False
|
||||
|
||||
try:
|
||||
success = await self._ws_connect()
|
||||
if not success:
|
||||
return False
|
||||
|
||||
# Dedicated REST session for send() calls
|
||||
self._rest_session = aiohttp.ClientSession(
|
||||
timeout=aiohttp.ClientTimeout(total=30)
|
||||
)
|
||||
|
||||
# Warn if no event filters are configured
|
||||
if not self._watch_domains and not self._watch_entities and not self._watch_all:
|
||||
logger.warning(
|
||||
"[%s] No watch_domains, watch_entities, or watch_all configured. "
|
||||
"All state_changed events will be dropped. Configure filters in "
|
||||
"your HA platform config to receive events.",
|
||||
self.name,
|
||||
)
|
||||
|
||||
# Start background listener
|
||||
self._listen_task = asyncio.create_task(self._listen_loop())
|
||||
self._running = True
|
||||
logger.info("[%s] Connected to %s", self.name, self._hass_url)
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
logger.error("[%s] Failed to connect: %s", self.name, e)
|
||||
return False
|
||||
|
||||
async def _ws_connect(self) -> bool:
|
||||
"""Establish WebSocket connection and authenticate."""
|
||||
ws_url = self._hass_url.replace("https://", "wss://").replace("http://", "ws://")
|
||||
ws_url = f"{ws_url}/api/websocket"
|
||||
|
||||
self._session = aiohttp.ClientSession(
|
||||
timeout=aiohttp.ClientTimeout(total=30)
|
||||
)
|
||||
self._ws = await self._session.ws_connect(ws_url, heartbeat=30, timeout=30)
|
||||
|
||||
# Step 1: Receive auth_required
|
||||
msg = await self._ws.receive_json()
|
||||
if msg.get("type") != "auth_required":
|
||||
logger.error("Expected auth_required, got: %s", msg.get("type"))
|
||||
await self._cleanup_ws()
|
||||
return False
|
||||
|
||||
# Step 2: Send auth
|
||||
await self._ws.send_json({
|
||||
"type": "auth",
|
||||
"access_token": self._hass_token,
|
||||
})
|
||||
|
||||
# Step 3: Wait for auth_ok
|
||||
msg = await self._ws.receive_json()
|
||||
if msg.get("type") != "auth_ok":
|
||||
logger.error("Auth failed: %s", msg)
|
||||
await self._cleanup_ws()
|
||||
return False
|
||||
|
||||
# Step 4: Subscribe to state_changed events
|
||||
sub_id = self._next_id()
|
||||
await self._ws.send_json({
|
||||
"id": sub_id,
|
||||
"type": "subscribe_events",
|
||||
"event_type": "state_changed",
|
||||
})
|
||||
|
||||
# Verify subscription acknowledgement
|
||||
msg = await self._ws.receive_json()
|
||||
if not msg.get("success"):
|
||||
logger.error("Failed to subscribe to events: %s", msg)
|
||||
await self._cleanup_ws()
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
async def _cleanup_ws(self) -> None:
|
||||
"""Close WebSocket and session."""
|
||||
if self._ws and not self._ws.closed:
|
||||
await self._ws.close()
|
||||
self._ws = None
|
||||
if self._session and not self._session.closed:
|
||||
await self._session.close()
|
||||
self._session = None
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
"""Disconnect from Home Assistant."""
|
||||
self._running = False
|
||||
if self._listen_task:
|
||||
self._listen_task.cancel()
|
||||
try:
|
||||
await self._listen_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self._listen_task = None
|
||||
|
||||
await self._cleanup_ws()
|
||||
if self._rest_session and not self._rest_session.closed:
|
||||
await self._rest_session.close()
|
||||
self._rest_session = None
|
||||
logger.info("[%s] Disconnected", self.name)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Event listener
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _listen_loop(self) -> None:
|
||||
"""Main event loop with automatic reconnection."""
|
||||
backoff_idx = 0
|
||||
|
||||
while self._running:
|
||||
try:
|
||||
await self._read_events()
|
||||
except asyncio.CancelledError:
|
||||
return
|
||||
except Exception as e:
|
||||
logger.warning("[%s] WebSocket error: %s", self.name, e)
|
||||
|
||||
if not self._running:
|
||||
return
|
||||
|
||||
# Reconnect with backoff
|
||||
delay = self._BACKOFF_STEPS[min(backoff_idx, len(self._BACKOFF_STEPS) - 1)]
|
||||
logger.info("[%s] Reconnecting in %ds...", self.name, delay)
|
||||
await asyncio.sleep(delay)
|
||||
backoff_idx += 1
|
||||
|
||||
try:
|
||||
await self._cleanup_ws()
|
||||
success = await self._ws_connect()
|
||||
if success:
|
||||
backoff_idx = 0 # Reset on successful reconnect
|
||||
logger.info("[%s] Reconnected", self.name)
|
||||
except Exception as e:
|
||||
logger.warning("[%s] Reconnection failed: %s", self.name, e)
|
||||
|
||||
async def _read_events(self) -> None:
|
||||
"""Read events from WebSocket until disconnected."""
|
||||
if self._ws is None or self._ws.closed:
|
||||
return
|
||||
async for ws_msg in self._ws:
|
||||
if ws_msg.type == aiohttp.WSMsgType.TEXT:
|
||||
try:
|
||||
data = json.loads(ws_msg.data)
|
||||
if data.get("type") == "event":
|
||||
await self._handle_ha_event(data.get("event", {}))
|
||||
except json.JSONDecodeError:
|
||||
logger.debug("Invalid JSON from HA WS: %s", ws_msg.data[:200])
|
||||
elif ws_msg.type in {aiohttp.WSMsgType.CLOSED, aiohttp.WSMsgType.ERROR}:
|
||||
break
|
||||
|
||||
async def _handle_ha_event(self, event: Dict[str, Any]) -> None:
|
||||
"""Process a state_changed event from Home Assistant."""
|
||||
event_data = event.get("data", {})
|
||||
entity_id: str = event_data.get("entity_id", "")
|
||||
|
||||
if not entity_id:
|
||||
return
|
||||
|
||||
# Apply ignore filter
|
||||
if entity_id in self._ignore_entities:
|
||||
return
|
||||
|
||||
# Apply domain/entity watch filters (closed by default — require
|
||||
# explicit watch_domains, watch_entities, or watch_all to forward)
|
||||
domain = entity_id.split(".")[0] if "." in entity_id else ""
|
||||
if self._watch_domains or self._watch_entities:
|
||||
domain_match = domain in self._watch_domains if self._watch_domains else False
|
||||
entity_match = entity_id in self._watch_entities if self._watch_entities else False
|
||||
if not domain_match and not entity_match:
|
||||
return
|
||||
elif not self._watch_all:
|
||||
# No filters configured and watch_all is off — drop the event
|
||||
return
|
||||
|
||||
# Apply cooldown
|
||||
now = time.time()
|
||||
last = self._last_event_time.get(entity_id, 0)
|
||||
if (now - last) < self._cooldown_seconds:
|
||||
return
|
||||
self._last_event_time[entity_id] = now
|
||||
|
||||
# Build human-readable message
|
||||
old_state = event_data.get("old_state", {})
|
||||
new_state = event_data.get("new_state", {})
|
||||
message = self._format_state_change(entity_id, old_state, new_state)
|
||||
|
||||
if not message:
|
||||
return
|
||||
|
||||
# Build MessageEvent and forward to handler
|
||||
source = self.build_source(
|
||||
chat_id="ha_events",
|
||||
chat_name="Home Assistant Events",
|
||||
chat_type="channel",
|
||||
user_id="homeassistant",
|
||||
user_name="Home Assistant",
|
||||
)
|
||||
|
||||
msg_event = MessageEvent(
|
||||
text=message,
|
||||
message_type=MessageType.TEXT,
|
||||
source=source,
|
||||
message_id=f"ha_{entity_id}_{int(now)}",
|
||||
timestamp=datetime.now(),
|
||||
)
|
||||
|
||||
await self.handle_message(msg_event)
|
||||
|
||||
@staticmethod
|
||||
def _format_state_change(
|
||||
entity_id: str,
|
||||
old_state: Dict[str, Any],
|
||||
new_state: Dict[str, Any],
|
||||
) -> Optional[str]:
|
||||
"""Convert a state_changed event into a human-readable description."""
|
||||
if not new_state:
|
||||
return None
|
||||
|
||||
old_val = old_state.get("state", "unknown") if old_state else "unknown"
|
||||
new_val = new_state.get("state", "unknown")
|
||||
|
||||
# Skip if state didn't actually change
|
||||
if old_val == new_val:
|
||||
return None
|
||||
|
||||
friendly_name = new_state.get("attributes", {}).get("friendly_name", entity_id)
|
||||
domain = entity_id.split(".")[0] if "." in entity_id else ""
|
||||
|
||||
# Domain-specific formatting
|
||||
if domain == "climate":
|
||||
attrs = new_state.get("attributes", {})
|
||||
temp = attrs.get("current_temperature", "?")
|
||||
target = attrs.get("temperature", "?")
|
||||
return (
|
||||
f"[Home Assistant] {friendly_name}: HVAC mode changed from "
|
||||
f"'{old_val}' to '{new_val}' (current: {temp}, target: {target})"
|
||||
)
|
||||
|
||||
if domain == "sensor":
|
||||
unit = new_state.get("attributes", {}).get("unit_of_measurement", "")
|
||||
return (
|
||||
f"[Home Assistant] {friendly_name}: changed from "
|
||||
f"{old_val}{unit} to {new_val}{unit}"
|
||||
)
|
||||
|
||||
if domain == "binary_sensor":
|
||||
return (
|
||||
f"[Home Assistant] {friendly_name}: "
|
||||
f"{'triggered' if new_val == 'on' else 'cleared'} "
|
||||
f"(was {'triggered' if old_val == 'on' else 'cleared'})"
|
||||
)
|
||||
|
||||
if domain in {"light", "switch", "fan"}:
|
||||
return (
|
||||
f"[Home Assistant] {friendly_name}: turned "
|
||||
f"{'on' if new_val == 'on' else 'off'}"
|
||||
)
|
||||
|
||||
if domain == "alarm_control_panel":
|
||||
return (
|
||||
f"[Home Assistant] {friendly_name}: alarm state changed from "
|
||||
f"'{old_val}' to '{new_val}'"
|
||||
)
|
||||
|
||||
# Generic fallback
|
||||
return (
|
||||
f"[Home Assistant] {friendly_name} ({entity_id}): "
|
||||
f"changed from '{old_val}' to '{new_val}'"
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Outbound messaging
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def send(
|
||||
self,
|
||||
chat_id: str,
|
||||
content: str,
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
"""Send a notification via HA REST API (persistent_notification.create).
|
||||
|
||||
Uses the REST API instead of WebSocket to avoid a race condition
|
||||
with the event listener loop that reads from the same WS connection.
|
||||
"""
|
||||
url = f"{self._hass_url}/api/services/persistent_notification/create"
|
||||
headers = {
|
||||
"Authorization": f"Bearer {self._hass_token}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
payload = {
|
||||
"title": "Hermes Agent",
|
||||
"message": content[:self.MAX_MESSAGE_LENGTH],
|
||||
}
|
||||
|
||||
try:
|
||||
if self._rest_session:
|
||||
async with self._rest_session.post(
|
||||
url,
|
||||
headers=headers,
|
||||
json=payload,
|
||||
timeout=aiohttp.ClientTimeout(total=10),
|
||||
) as resp:
|
||||
if resp.status < 300:
|
||||
return SendResult(success=True, message_id=uuid.uuid4().hex[:12])
|
||||
else:
|
||||
body = await resp.text()
|
||||
return SendResult(success=False, error=f"HTTP {resp.status}: {body}")
|
||||
else:
|
||||
async with aiohttp.ClientSession() as session:
|
||||
async with session.post(
|
||||
url,
|
||||
headers=headers,
|
||||
json=payload,
|
||||
timeout=aiohttp.ClientTimeout(total=10),
|
||||
) as resp:
|
||||
if resp.status < 300:
|
||||
return SendResult(success=True, message_id=uuid.uuid4().hex[:12])
|
||||
else:
|
||||
body = await resp.text()
|
||||
return SendResult(success=False, error=f"HTTP {resp.status}: {body}")
|
||||
|
||||
except asyncio.TimeoutError:
|
||||
return SendResult(success=False, error="Timeout sending notification to HA")
|
||||
except Exception as e:
|
||||
return SendResult(success=False, error=str(e))
|
||||
|
||||
async def send_typing(self, chat_id: str, metadata=None) -> None:
|
||||
"""No typing indicator for Home Assistant."""
|
||||
|
||||
async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
|
||||
"""Return basic info about the HA event channel."""
|
||||
return {
|
||||
"name": "Home Assistant Events",
|
||||
"type": "channel",
|
||||
"url": self._hass_url,
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,421 @@
|
||||
"""Microsoft Graph webhook adapter for change-notification ingress."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import hmac
|
||||
import ipaddress
|
||||
import json
|
||||
import logging
|
||||
from collections import deque
|
||||
from hashlib import sha1
|
||||
from typing import Any, Awaitable, Callable, Dict, Optional
|
||||
|
||||
try:
|
||||
from aiohttp import web
|
||||
|
||||
AIOHTTP_AVAILABLE = True
|
||||
except ImportError:
|
||||
AIOHTTP_AVAILABLE = False
|
||||
web = None # type: ignore[assignment]
|
||||
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
from gateway.platforms.base import (
|
||||
BasePlatformAdapter,
|
||||
MessageEvent,
|
||||
MessageType,
|
||||
SendResult,
|
||||
is_network_accessible,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
DEFAULT_HOST = "0.0.0.0"
|
||||
DEFAULT_PORT = 8646
|
||||
DEFAULT_WEBHOOK_PATH = "/msgraph/webhook"
|
||||
DEFAULT_MAX_SEEN_RECEIPTS = 5000
|
||||
NotificationScheduler = Callable[[Dict[str, Any], MessageEvent], Awaitable[None] | None]
|
||||
|
||||
|
||||
def check_msgraph_webhook_requirements() -> bool:
|
||||
"""Return whether required webhook dependencies are available."""
|
||||
return AIOHTTP_AVAILABLE
|
||||
|
||||
|
||||
class MSGraphWebhookAdapter(BasePlatformAdapter):
|
||||
"""Receive Microsoft Graph change notifications and surface them internally."""
|
||||
|
||||
def __init__(self, config: PlatformConfig):
|
||||
super().__init__(config, Platform.MSGRAPH_WEBHOOK)
|
||||
extra = config.extra or {}
|
||||
self._host: str = str(extra.get("host", DEFAULT_HOST))
|
||||
self._port: int = int(extra.get("port", DEFAULT_PORT))
|
||||
self._webhook_path: str = self._normalize_path(
|
||||
extra.get("webhook_path", DEFAULT_WEBHOOK_PATH)
|
||||
)
|
||||
self._health_path: str = self._normalize_path(extra.get("health_path", "/health"))
|
||||
self._accepted_resources: list[str] = [
|
||||
str(value).strip()
|
||||
for value in (extra.get("accepted_resources") or [])
|
||||
if str(value).strip()
|
||||
]
|
||||
self._client_state: Optional[str] = self._string_or_none(extra.get("client_state"))
|
||||
self._max_seen_receipts = max(
|
||||
1, int(extra.get("max_seen_receipts", DEFAULT_MAX_SEEN_RECEIPTS))
|
||||
)
|
||||
self._allowed_source_networks: list[ipaddress._BaseNetwork] = (
|
||||
self._parse_allowed_source_cidrs(extra.get("allowed_source_cidrs"))
|
||||
)
|
||||
self._runner = None
|
||||
self._notification_scheduler: Optional[NotificationScheduler] = None
|
||||
self._seen_receipts: set[str] = set()
|
||||
self._seen_receipt_order: deque[str] = deque()
|
||||
self._accepted_count = 0
|
||||
self._duplicate_count = 0
|
||||
|
||||
@staticmethod
|
||||
def _string_or_none(value: Any) -> Optional[str]:
|
||||
if value is None:
|
||||
return None
|
||||
text = str(value).strip()
|
||||
return text or None
|
||||
|
||||
@staticmethod
|
||||
def _normalize_path(path: Any) -> str:
|
||||
raw = str(path or "").strip() or "/"
|
||||
return raw if raw.startswith("/") else f"/{raw}"
|
||||
|
||||
@staticmethod
|
||||
def _build_receipt_key(notification: Dict[str, Any]) -> Optional[str]:
|
||||
explicit_id = str(notification.get("id") or "").strip()
|
||||
if explicit_id:
|
||||
return f"id:{explicit_id}"
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _normalize_resource_value(resource: str) -> str:
|
||||
return str(resource or "").strip().strip("/")
|
||||
|
||||
@staticmethod
|
||||
def _parse_allowed_source_cidrs(
|
||||
raw: Any,
|
||||
) -> list[ipaddress._BaseNetwork]:
|
||||
"""Parse an optional list of CIDR ranges allowed to POST to the webhook.
|
||||
|
||||
An empty or missing value means "allow everything" (same behavior as
|
||||
before this field existed). When populated, requests from source IPs
|
||||
outside every listed CIDR are rejected with 403 before the body is
|
||||
parsed. Use this to restrict the endpoint to Microsoft Graph's
|
||||
published webhook source ranges in production deployments.
|
||||
"""
|
||||
if raw is None:
|
||||
return []
|
||||
if isinstance(raw, str):
|
||||
candidates = [chunk.strip() for chunk in raw.split(",")]
|
||||
elif isinstance(raw, (list, tuple, set)):
|
||||
candidates = [str(chunk).strip() for chunk in raw]
|
||||
else:
|
||||
return []
|
||||
|
||||
networks: list[ipaddress._BaseNetwork] = []
|
||||
for chunk in candidates:
|
||||
if not chunk:
|
||||
continue
|
||||
try:
|
||||
networks.append(ipaddress.ip_network(chunk, strict=False))
|
||||
except ValueError:
|
||||
logger.warning(
|
||||
"[msgraph_webhook] Ignoring invalid allowed_source_cidrs entry: %r",
|
||||
chunk,
|
||||
)
|
||||
return networks
|
||||
|
||||
def set_notification_scheduler(self, scheduler: Optional[NotificationScheduler]) -> None:
|
||||
self._notification_scheduler = scheduler
|
||||
|
||||
def _source_allowlist_required_but_missing(self) -> bool:
|
||||
return is_network_accessible(self._host) and not self._allowed_source_networks
|
||||
|
||||
async def connect(self) -> bool:
|
||||
if self._client_state is None:
|
||||
logger.error(
|
||||
"[msgraph_webhook] Refusing to start without extra.client_state configured"
|
||||
)
|
||||
return False
|
||||
if self._source_allowlist_required_but_missing():
|
||||
logger.error(
|
||||
"[msgraph_webhook] Refusing to start: binding to %s requires "
|
||||
"extra.allowed_source_cidrs. Configure the Microsoft Graph "
|
||||
"source CIDRs or bind to loopback (127.0.0.1/::1) behind a "
|
||||
"tunnel or reverse proxy.",
|
||||
self._host,
|
||||
)
|
||||
return False
|
||||
|
||||
app = web.Application()
|
||||
app.router.add_get(self._health_path, self._handle_health)
|
||||
app.router.add_get(self._webhook_path, self._handle_validation)
|
||||
app.router.add_post(self._webhook_path, self._handle_notification)
|
||||
|
||||
self._runner = web.AppRunner(app)
|
||||
await self._runner.setup()
|
||||
site = web.TCPSite(self._runner, self._host, self._port)
|
||||
await site.start()
|
||||
self._mark_connected()
|
||||
logger.info(
|
||||
"[msgraph_webhook] Listening on %s:%d%s",
|
||||
self._host,
|
||||
self._port,
|
||||
self._webhook_path,
|
||||
)
|
||||
return True
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
if self._runner is not None:
|
||||
await self._runner.cleanup()
|
||||
self._runner = None
|
||||
self._mark_disconnected()
|
||||
|
||||
async def send(
|
||||
self,
|
||||
chat_id: str,
|
||||
content: str,
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
logger.info("[msgraph_webhook] Response for %s: %s", chat_id, content[:200])
|
||||
return SendResult(success=True)
|
||||
|
||||
async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
|
||||
return {"name": chat_id, "type": "webhook"}
|
||||
|
||||
async def _handle_health(self, request: "web.Request") -> "web.Response":
|
||||
if not self._source_ip_allowed(request):
|
||||
return web.Response(status=403)
|
||||
return web.json_response(
|
||||
{
|
||||
"status": "ok",
|
||||
"platform": self.platform.value,
|
||||
"webhook_path": self._webhook_path,
|
||||
"accepted": self._accepted_count,
|
||||
"duplicates": self._duplicate_count,
|
||||
}
|
||||
)
|
||||
|
||||
async def _handle_validation(self, request: "web.Request") -> "web.Response":
|
||||
"""Handle Microsoft Graph subscription validation handshake.
|
||||
|
||||
Graph validates a subscription endpoint by sending a GET with
|
||||
``validationToken`` in the query string; the service must echo the
|
||||
token verbatim as ``text/plain`` within 10 seconds. Anything else
|
||||
(bare GET, GET without the token) is rejected so the endpoint can't
|
||||
be enumerated or mistakenly used for data exfiltration.
|
||||
"""
|
||||
if not self._source_ip_allowed(request):
|
||||
return web.Response(status=403)
|
||||
validation_token = request.query.get("validationToken", "")
|
||||
if not validation_token:
|
||||
return web.Response(status=400)
|
||||
return web.Response(text=validation_token, content_type="text/plain")
|
||||
|
||||
async def _handle_notification(self, request: "web.Request") -> "web.Response":
|
||||
if not self._source_ip_allowed(request):
|
||||
return web.Response(status=403)
|
||||
|
||||
# Graph never sends validationToken on POST, but tolerate it for
|
||||
# defensive clients that replay the handshake in-band.
|
||||
validation_token = request.query.get("validationToken", "")
|
||||
if validation_token:
|
||||
return web.Response(text=validation_token, content_type="text/plain")
|
||||
|
||||
try:
|
||||
body = await request.json()
|
||||
except Exception:
|
||||
return web.Response(status=400)
|
||||
|
||||
notifications = body.get("value")
|
||||
if not isinstance(notifications, list):
|
||||
return web.Response(status=400)
|
||||
|
||||
accepted = 0
|
||||
duplicates = 0
|
||||
auth_rejected = 0
|
||||
other_rejected = 0
|
||||
|
||||
for raw_notification in notifications:
|
||||
if not isinstance(raw_notification, dict):
|
||||
other_rejected += 1
|
||||
continue
|
||||
notification = dict(raw_notification)
|
||||
if not self._resource_accepted(str(notification.get("resource") or "")):
|
||||
other_rejected += 1
|
||||
continue
|
||||
if not self._verify_client_state(notification):
|
||||
# Treat bad clientState as an auth failure: if the whole
|
||||
# batch is forged, we want to signal 403 so the sender
|
||||
# stops retrying. Legitimate Graph retries have valid
|
||||
# clientState and hit the accepted/duplicate paths.
|
||||
auth_rejected += 1
|
||||
continue
|
||||
|
||||
receipt_key = self._build_receipt_key(notification)
|
||||
if receipt_key is not None:
|
||||
if self._has_seen_receipt(receipt_key):
|
||||
duplicates += 1
|
||||
continue
|
||||
self._remember_receipt(receipt_key)
|
||||
|
||||
accepted += 1
|
||||
self._accepted_count += 1
|
||||
event = self._build_message_event(notification, receipt_key)
|
||||
self._schedule_notification(notification, event)
|
||||
|
||||
self._duplicate_count += duplicates
|
||||
# If anything ingested OR deduped, return 202 with empty body so
|
||||
# Graph acks successfully and we don't leak internal counters. If
|
||||
# every item failed auth, return 403 so an attacker POSTing fake
|
||||
# notifications gets a clear reject. Other failures (malformed,
|
||||
# resource-not-accepted) are the sender's configuration problem,
|
||||
# so 400.
|
||||
if accepted or duplicates:
|
||||
return web.Response(status=202)
|
||||
if auth_rejected and not other_rejected:
|
||||
return web.Response(status=403)
|
||||
return web.Response(status=400)
|
||||
|
||||
def _source_ip_allowed(self, request: "web.Request") -> bool:
|
||||
"""Return True if the request's source IP is in the configured allowlist.
|
||||
|
||||
Loopback-only binds may omit ``allowed_source_cidrs`` for local reverse
|
||||
proxies and dev tunnels. Network-accessible binds fail closed until an
|
||||
explicit CIDR allowlist is configured.
|
||||
"""
|
||||
if self._source_allowlist_required_but_missing():
|
||||
return False
|
||||
if not self._allowed_source_networks:
|
||||
return True
|
||||
peer = request.remote or ""
|
||||
if not peer:
|
||||
return False
|
||||
try:
|
||||
peer_addr = ipaddress.ip_address(peer)
|
||||
except ValueError:
|
||||
return False
|
||||
return any(peer_addr in network for network in self._allowed_source_networks)
|
||||
|
||||
def _resource_accepted(self, resource: str) -> bool:
|
||||
if not self._accepted_resources:
|
||||
return True
|
||||
normalized_resource = self._normalize_resource_value(resource)
|
||||
for pattern in self._accepted_resources:
|
||||
normalized_pattern = self._normalize_resource_value(pattern)
|
||||
if not normalized_pattern:
|
||||
continue
|
||||
if normalized_pattern.endswith("*"):
|
||||
prefix = normalized_pattern[:-1].rstrip("/")
|
||||
if normalized_resource == prefix or normalized_resource.startswith(f"{prefix}/"):
|
||||
return True
|
||||
continue
|
||||
if (
|
||||
normalized_resource == normalized_pattern
|
||||
or normalized_resource.startswith(f"{normalized_pattern}/")
|
||||
):
|
||||
return True
|
||||
return False
|
||||
|
||||
def _verify_client_state(self, notification: Dict[str, Any]) -> bool:
|
||||
"""Verify the Graph-supplied clientState matches the configured secret.
|
||||
|
||||
Uses ``hmac.compare_digest`` instead of ``==`` so that a mismatch
|
||||
doesn't leak how many leading characters matched via string-compare
|
||||
timing. The configured client_state is a shared secret (documented in
|
||||
the setup guide as "generate with ``openssl rand -hex 32``"), so a
|
||||
timing-safe compare is the right primitive.
|
||||
"""
|
||||
expected = self._client_state
|
||||
if expected is None:
|
||||
return False
|
||||
provided = self._string_or_none(notification.get("clientState"))
|
||||
if provided is None:
|
||||
return False
|
||||
return hmac.compare_digest(provided, expected)
|
||||
|
||||
def _has_seen_receipt(self, receipt_key: str) -> bool:
|
||||
return receipt_key in self._seen_receipts
|
||||
|
||||
def _remember_receipt(self, receipt_key: str) -> None:
|
||||
self._seen_receipts.add(receipt_key)
|
||||
self._seen_receipt_order.append(receipt_key)
|
||||
while len(self._seen_receipt_order) > self._max_seen_receipts:
|
||||
oldest = self._seen_receipt_order.popleft()
|
||||
self._seen_receipts.discard(oldest)
|
||||
|
||||
def _build_message_event(
|
||||
self,
|
||||
notification: Dict[str, Any],
|
||||
receipt_key: Optional[str],
|
||||
) -> MessageEvent:
|
||||
message_id = receipt_key or f"sha1:{sha1(json.dumps(notification, sort_keys=True).encode('utf-8')).hexdigest()}"
|
||||
source = self.build_source(
|
||||
chat_id=f"msgraph:{notification.get('subscriptionId', 'unknown')}",
|
||||
chat_name="msgraph/webhook",
|
||||
chat_type="webhook",
|
||||
user_id="msgraph",
|
||||
user_name="Microsoft Graph",
|
||||
)
|
||||
return MessageEvent(
|
||||
text=self._render_prompt(notification),
|
||||
message_type=MessageType.TEXT,
|
||||
source=source,
|
||||
raw_message=notification,
|
||||
message_id=message_id,
|
||||
internal=True,
|
||||
)
|
||||
|
||||
def _render_prompt(self, notification: Dict[str, Any]) -> str:
|
||||
template = self.config.extra.get("prompt", "")
|
||||
if template:
|
||||
payload = {
|
||||
"notification": notification,
|
||||
"resource": notification.get("resource", ""),
|
||||
"change_type": notification.get("changeType", ""),
|
||||
"subscription_id": notification.get("subscriptionId", ""),
|
||||
}
|
||||
return self._render_template(template, payload)
|
||||
rendered = json.dumps(notification, indent=2, sort_keys=True)[:4000]
|
||||
return f"Microsoft Graph change notification:\n\n```json\n{rendered}\n```"
|
||||
|
||||
def _render_template(self, template: str, payload: Dict[str, Any]) -> str:
|
||||
import re
|
||||
|
||||
def _resolve(match: "re.Match[str]") -> str:
|
||||
key = match.group(1)
|
||||
value: Any = payload
|
||||
for part in key.split("."):
|
||||
if isinstance(value, dict):
|
||||
value = value.get(part, f"{{{key}}}")
|
||||
else:
|
||||
return f"{{{key}}}"
|
||||
if isinstance(value, (dict, list)):
|
||||
return json.dumps(value, sort_keys=True)[:2000]
|
||||
return str(value)
|
||||
|
||||
return re.sub(r"\{([a-zA-Z0-9_.]+)\}", _resolve, template)
|
||||
|
||||
def _schedule_notification(
|
||||
self,
|
||||
notification: Dict[str, Any],
|
||||
event: MessageEvent,
|
||||
) -> None:
|
||||
scheduler = self._notification_scheduler
|
||||
if scheduler is not None:
|
||||
result = scheduler(notification, event)
|
||||
if asyncio.iscoroutine(result):
|
||||
task = asyncio.create_task(result)
|
||||
self._background_tasks.add(task)
|
||||
task.add_done_callback(self._background_tasks.discard)
|
||||
return
|
||||
|
||||
task = asyncio.create_task(self.handle_message(event))
|
||||
self._background_tasks.add(task)
|
||||
task.add_done_callback(self._background_tasks.discard)
|
||||
@@ -0,0 +1,91 @@
|
||||
"""
|
||||
QQBot platform package.
|
||||
|
||||
Re-exports the main adapter symbols from ``adapter.py`` (the original
|
||||
``qqbot.py``) so that **all existing import paths remain unchanged**::
|
||||
|
||||
from gateway.platforms.qqbot import QQAdapter # works
|
||||
from gateway.platforms.qqbot import check_qq_requirements # works
|
||||
|
||||
New modules:
|
||||
- ``constants`` — shared constants (API URLs, timeouts, message types)
|
||||
- ``utils`` — User-Agent builder, config helpers
|
||||
- ``crypto`` — AES-256-GCM key generation and decryption
|
||||
- ``onboard`` — QR-code scan-to-configure flow
|
||||
"""
|
||||
|
||||
# -- Adapter (original qqbot.py) ------------------------------------------
|
||||
from .adapter import ( # noqa: F401
|
||||
QQAdapter,
|
||||
QQCloseError,
|
||||
check_qq_requirements,
|
||||
_coerce_list,
|
||||
_ssrf_redirect_guard,
|
||||
)
|
||||
|
||||
# -- Onboard (QR-code scan-to-configure) -----------------------------------
|
||||
from .onboard import ( # noqa: F401
|
||||
BindStatus,
|
||||
build_connect_url,
|
||||
qr_register,
|
||||
)
|
||||
from .crypto import decrypt_secret, generate_bind_key # noqa: F401
|
||||
|
||||
# -- Utils -----------------------------------------------------------------
|
||||
from .utils import build_user_agent, get_api_headers, coerce_list # noqa: F401
|
||||
|
||||
# -- Chunked upload --------------------------------------------------------
|
||||
from .chunked_upload import ( # noqa: F401
|
||||
ChunkedUploader,
|
||||
UploadDailyLimitExceededError,
|
||||
UploadFileTooLargeError,
|
||||
)
|
||||
|
||||
# -- Inline keyboards ------------------------------------------------------
|
||||
from .keyboards import ( # noqa: F401
|
||||
ApprovalRequest,
|
||||
ApprovalSender,
|
||||
InlineKeyboard,
|
||||
InteractionEvent,
|
||||
build_approval_keyboard,
|
||||
build_approval_text,
|
||||
build_update_prompt_keyboard,
|
||||
parse_approval_button_data,
|
||||
parse_interaction_event,
|
||||
parse_update_prompt_button_data,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
# adapter
|
||||
"QQAdapter",
|
||||
"QQCloseError",
|
||||
"check_qq_requirements",
|
||||
"_coerce_list",
|
||||
"_ssrf_redirect_guard",
|
||||
# onboard
|
||||
"BindStatus",
|
||||
"build_connect_url",
|
||||
"qr_register",
|
||||
# crypto
|
||||
"decrypt_secret",
|
||||
"generate_bind_key",
|
||||
# utils
|
||||
"build_user_agent",
|
||||
"get_api_headers",
|
||||
"coerce_list",
|
||||
# chunked upload
|
||||
"ChunkedUploader",
|
||||
"UploadDailyLimitExceededError",
|
||||
"UploadFileTooLargeError",
|
||||
# keyboards
|
||||
"ApprovalRequest",
|
||||
"ApprovalSender",
|
||||
"InlineKeyboard",
|
||||
"InteractionEvent",
|
||||
"build_approval_keyboard",
|
||||
"build_approval_text",
|
||||
"build_update_prompt_keyboard",
|
||||
"parse_approval_button_data",
|
||||
"parse_interaction_event",
|
||||
"parse_update_prompt_button_data",
|
||||
]
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,602 @@
|
||||
"""QQ Bot chunked upload flow.
|
||||
|
||||
The QQ v2 API caps inline base64 uploads (``file_data`` / ``url``) at ~10 MB.
|
||||
For files between 10 MB and ~100 MB we have to use the three-step chunked
|
||||
upload flow::
|
||||
|
||||
1. POST /v2/{users|groups}/{id}/upload_prepare
|
||||
→ returns upload_id, block_size, and an array of pre-signed COS part URLs.
|
||||
2. For each part:
|
||||
PUT the part bytes to its pre-signed COS URL,
|
||||
then POST /v2/{users|groups}/{id}/upload_part_finish to acknowledge.
|
||||
3. POST /v2/{users|groups}/{id}/files with {"upload_id": ...}
|
||||
→ returns the ``file_info`` token the caller uses in a RichMedia
|
||||
message.
|
||||
|
||||
Error-code semantics (from the QQ Bot v2 API spec):
|
||||
|
||||
- ``40093001`` — ``upload_part_finish`` retryable. Retry until the server-provided
|
||||
``retry_timeout`` elapses (or a local cap).
|
||||
- ``40093002`` — daily cumulative upload quota exceeded. Not retryable; surface
|
||||
as :class:`UploadDailyLimitExceededError` so the caller can build a
|
||||
user-friendly reply.
|
||||
|
||||
Exceptions:
|
||||
|
||||
- :class:`UploadDailyLimitExceededError` — daily quota hit (non-retryable).
|
||||
- :class:`UploadFileTooLargeError` — file exceeds the platform per-file limit.
|
||||
- :class:`RuntimeError` — generic upload failure (network, part PUT, complete).
|
||||
|
||||
Ported from WideLee's qqbot-agent-sdk v1.2.2 (``media_loader.py::ChunkedUploader``)
|
||||
so the heavy-upload path stays in-tree. Authorship preserved via Co-authored-by.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import functools
|
||||
import hashlib
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Awaitable, Callable, Dict, List, Optional
|
||||
|
||||
from gateway.platforms.qqbot.constants import FILE_UPLOAD_TIMEOUT
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ── Error codes ──────────────────────────────────────────────────────
|
||||
_BIZ_CODE_DAILY_LIMIT = 40093002 # upload_prepare: daily cumulative limit
|
||||
_BIZ_CODE_PART_RETRYABLE = 40093001 # upload_part_finish: transient
|
||||
|
||||
# ── Part upload tuning ───────────────────────────────────────────────
|
||||
_DEFAULT_CONCURRENT_PARTS = 1
|
||||
_MAX_CONCURRENT_PARTS = 10
|
||||
|
||||
_PART_UPLOAD_TIMEOUT = 300.0 # 5 minutes per COS PUT
|
||||
_PART_UPLOAD_MAX_RETRIES = 2
|
||||
_PART_FINISH_RETRY_INTERVAL = 1.0
|
||||
_PART_FINISH_DEFAULT_TIMEOUT = 120.0
|
||||
_PART_FINISH_MAX_TIMEOUT = 600.0
|
||||
|
||||
_COMPLETE_UPLOAD_MAX_RETRIES = 2
|
||||
_COMPLETE_UPLOAD_BASE_DELAY = 2.0
|
||||
|
||||
# First 10,002,432 bytes used for the ``md5_10m`` hash (per QQ API spec).
|
||||
_MD5_10M_SIZE = 10_002_432
|
||||
|
||||
|
||||
# ── Exceptions ───────────────────────────────────────────────────────
|
||||
|
||||
class UploadDailyLimitExceededError(Exception):
|
||||
"""Raised when ``upload_prepare`` returns biz_code 40093002.
|
||||
|
||||
The daily cumulative upload quota for this bot has been reached. Callers
|
||||
should surface :attr:`file_name` + :attr:`file_size_human` so the model
|
||||
can compose a helpful reply.
|
||||
"""
|
||||
|
||||
def __init__(self, file_name: str, file_size: int, message: str = "") -> None:
|
||||
self.file_name = file_name
|
||||
self.file_size = file_size
|
||||
super().__init__(
|
||||
message or f"Daily upload limit exceeded for {file_name!r}"
|
||||
)
|
||||
|
||||
@property
|
||||
def file_size_human(self) -> str:
|
||||
return format_size(self.file_size)
|
||||
|
||||
|
||||
class UploadFileTooLargeError(Exception):
|
||||
"""Raised when a file exceeds the platform per-file size limit."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
file_name: str,
|
||||
file_size: int,
|
||||
limit_bytes: int = 0,
|
||||
message: str = "",
|
||||
) -> None:
|
||||
self.file_name = file_name
|
||||
self.file_size = file_size
|
||||
self.limit_bytes = limit_bytes
|
||||
limit_str = f" ({format_size(limit_bytes)})" if limit_bytes else ""
|
||||
super().__init__(
|
||||
message
|
||||
or (
|
||||
f"File {file_name!r} ({format_size(file_size)}) "
|
||||
f"exceeds platform limit{limit_str}"
|
||||
)
|
||||
)
|
||||
|
||||
@property
|
||||
def file_size_human(self) -> str:
|
||||
return format_size(self.file_size)
|
||||
|
||||
@property
|
||||
def limit_human(self) -> str:
|
||||
return format_size(self.limit_bytes) if self.limit_bytes else "unknown"
|
||||
|
||||
|
||||
# ── Progress tracking ────────────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class _UploadProgress:
|
||||
total_parts: int = 0
|
||||
total_bytes: int = 0
|
||||
completed_parts: int = 0
|
||||
uploaded_bytes: int = 0
|
||||
|
||||
|
||||
# ── Prepare-response shape ───────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class _PreparePart:
|
||||
index: int
|
||||
presigned_url: str
|
||||
block_size: int = 0
|
||||
|
||||
|
||||
@dataclass
|
||||
class _PrepareResult:
|
||||
upload_id: str
|
||||
block_size: int
|
||||
parts: List[_PreparePart]
|
||||
concurrency: int = _DEFAULT_CONCURRENT_PARTS
|
||||
retry_timeout: float = 0.0
|
||||
|
||||
|
||||
def _parse_prepare_response(raw: Dict[str, Any]) -> _PrepareResult:
|
||||
"""Parse the upload_prepare API response into a normalized shape.
|
||||
|
||||
The API may return the response directly or wrapped in ``data``.
|
||||
"""
|
||||
src = raw.get("data") if isinstance(raw.get("data"), dict) else raw
|
||||
upload_id = str(src.get("upload_id", ""))
|
||||
if not upload_id:
|
||||
raise ValueError(
|
||||
f"upload_prepare response missing upload_id: {str(raw)[:200]}"
|
||||
)
|
||||
block_size = int(src.get("block_size", 0))
|
||||
raw_parts = src.get("parts") or src.get("part_list") or []
|
||||
if not isinstance(raw_parts, list) or not raw_parts:
|
||||
raise ValueError(
|
||||
f"upload_prepare response missing parts: {str(raw)[:200]}"
|
||||
)
|
||||
parts: List[_PreparePart] = []
|
||||
for p in raw_parts:
|
||||
if not isinstance(p, dict):
|
||||
continue
|
||||
parts.append(
|
||||
_PreparePart(
|
||||
index=int(p.get("part_index") or p.get("index") or 0),
|
||||
presigned_url=str(
|
||||
p.get("presigned_url") or p.get("url") or ""
|
||||
),
|
||||
block_size=int(p.get("block_size", 0)),
|
||||
)
|
||||
)
|
||||
return _PrepareResult(
|
||||
upload_id=upload_id,
|
||||
block_size=block_size,
|
||||
parts=parts,
|
||||
concurrency=int(src.get("concurrency", _DEFAULT_CONCURRENT_PARTS)) or _DEFAULT_CONCURRENT_PARTS,
|
||||
retry_timeout=float(src.get("retry_timeout", 0.0) or 0.0),
|
||||
)
|
||||
|
||||
|
||||
# ── Chunked upload driver ────────────────────────────────────────────
|
||||
|
||||
ApiRequestFn = Callable[..., Awaitable[Dict[str, Any]]]
|
||||
"""Signature of the adapter's ``_api_request`` callable.
|
||||
|
||||
We pass the bound method in rather than importing the adapter, to avoid
|
||||
circular imports and keep this module testable in isolation.
|
||||
"""
|
||||
|
||||
|
||||
class ChunkedUploader:
|
||||
"""Run the prepare → PUT parts → complete sequence.
|
||||
|
||||
:param api_request: Bound ``_api_request(method, path, body=..., timeout=...)``
|
||||
coroutine from the adapter. Must raise ``RuntimeError`` with the biz_code
|
||||
embedded in the message on API errors.
|
||||
:param http_put: Coroutine ``(url, data, headers, timeout) -> response`` for
|
||||
COS part uploads. Typically wraps ``httpx.AsyncClient.put``.
|
||||
:param log_tag: Log prefix.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
api_request: ApiRequestFn,
|
||||
http_put: Callable[..., Awaitable[Any]],
|
||||
log_tag: str = "QQBot",
|
||||
) -> None:
|
||||
self._api_request = api_request
|
||||
self._http_put = http_put
|
||||
self._log_tag = log_tag
|
||||
|
||||
async def upload(
|
||||
self,
|
||||
chat_type: str,
|
||||
target_id: str,
|
||||
file_path: str,
|
||||
file_type: int,
|
||||
file_name: str,
|
||||
) -> Dict[str, Any]:
|
||||
"""Run the full chunked upload and return the ``complete_upload`` response.
|
||||
|
||||
:param chat_type: ``'c2c'`` or ``'group'``.
|
||||
:param target_id: User or group openid.
|
||||
:param file_path: Absolute path to a local file.
|
||||
:param file_type: ``MEDIA_TYPE_*`` constant.
|
||||
:param file_name: Original filename (for upload_prepare).
|
||||
:returns: The raw response dict from ``complete_upload`` — contains
|
||||
``file_info`` that the caller uses in a RichMedia message body.
|
||||
:raises UploadDailyLimitExceededError: On biz_code 40093002.
|
||||
:raises UploadFileTooLargeError: When the file exceeds the platform limit.
|
||||
:raises RuntimeError: On other API or I/O failures.
|
||||
"""
|
||||
if chat_type not in {"c2c", "group"}:
|
||||
raise ValueError(
|
||||
f"ChunkedUploader: unsupported chat_type {chat_type!r}"
|
||||
)
|
||||
|
||||
path = Path(file_path)
|
||||
file_size = path.stat().st_size
|
||||
|
||||
logger.info(
|
||||
"[%s] Chunked upload start: file=%s size=%s type=%d",
|
||||
self._log_tag, file_name, format_size(file_size), file_type,
|
||||
)
|
||||
|
||||
# Step 1: compute hashes (blocking I/O → executor).
|
||||
hashes = await asyncio.get_running_loop().run_in_executor(
|
||||
None, _compute_file_hashes, file_path, file_size
|
||||
)
|
||||
|
||||
# Step 2: upload_prepare.
|
||||
prepare = await self._prepare(
|
||||
chat_type, target_id, file_type, file_name, file_size, hashes
|
||||
)
|
||||
max_concurrent = min(prepare.concurrency, _MAX_CONCURRENT_PARTS)
|
||||
retry_timeout = min(
|
||||
prepare.retry_timeout if prepare.retry_timeout > 0 else _PART_FINISH_DEFAULT_TIMEOUT,
|
||||
_PART_FINISH_MAX_TIMEOUT,
|
||||
)
|
||||
logger.info(
|
||||
"[%s] Prepared: upload_id=%s block_size=%s parts=%d concurrency=%d",
|
||||
self._log_tag, prepare.upload_id, format_size(prepare.block_size),
|
||||
len(prepare.parts), max_concurrent,
|
||||
)
|
||||
|
||||
progress = _UploadProgress(
|
||||
total_parts=len(prepare.parts),
|
||||
total_bytes=file_size,
|
||||
)
|
||||
|
||||
# Step 3: PUT each part + notify.
|
||||
tasks: List[Callable[[], Awaitable[None]]] = [
|
||||
functools.partial(
|
||||
self._upload_one_part,
|
||||
chat_type=chat_type,
|
||||
target_id=target_id,
|
||||
file_path=file_path,
|
||||
file_size=file_size,
|
||||
upload_id=prepare.upload_id,
|
||||
rsp_block_size=prepare.block_size,
|
||||
part=part,
|
||||
retry_timeout=retry_timeout,
|
||||
progress=progress,
|
||||
)
|
||||
for part in prepare.parts
|
||||
]
|
||||
await _run_with_concurrency(tasks, max_concurrent)
|
||||
|
||||
logger.info(
|
||||
"[%s] All %d parts uploaded, completing…",
|
||||
self._log_tag, len(prepare.parts),
|
||||
)
|
||||
|
||||
# Step 4: complete_upload (retry on transient errors).
|
||||
return await self._complete(chat_type, target_id, prepare.upload_id)
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────
|
||||
# Step 1 — upload_prepare
|
||||
# ──────────────────────────────────────────────────────────────────
|
||||
|
||||
async def _prepare(
|
||||
self,
|
||||
chat_type: str,
|
||||
target_id: str,
|
||||
file_type: int,
|
||||
file_name: str,
|
||||
file_size: int,
|
||||
hashes: Dict[str, str],
|
||||
) -> _PrepareResult:
|
||||
base = "/v2/users" if chat_type == "c2c" else "/v2/groups"
|
||||
path = f"{base}/{target_id}/upload_prepare"
|
||||
body = {
|
||||
"file_type": file_type,
|
||||
"file_name": file_name,
|
||||
"file_size": file_size,
|
||||
"md5": hashes["md5"],
|
||||
"sha1": hashes["sha1"],
|
||||
"md5_10m": hashes["md5_10m"],
|
||||
}
|
||||
try:
|
||||
raw = await self._api_request(
|
||||
"POST", path, body=body, timeout=FILE_UPLOAD_TIMEOUT
|
||||
)
|
||||
except RuntimeError as exc:
|
||||
err_msg = str(exc)
|
||||
if f"{_BIZ_CODE_DAILY_LIMIT}" in err_msg:
|
||||
raise UploadDailyLimitExceededError(
|
||||
file_name, file_size, err_msg
|
||||
) from exc
|
||||
raise
|
||||
return _parse_prepare_response(raw)
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────
|
||||
# Step 2 — PUT one part + part_finish
|
||||
# ──────────────────────────────────────────────────────────────────
|
||||
|
||||
async def _upload_one_part(
|
||||
self,
|
||||
chat_type: str,
|
||||
target_id: str,
|
||||
file_path: str,
|
||||
file_size: int,
|
||||
upload_id: str,
|
||||
rsp_block_size: int,
|
||||
part: _PreparePart,
|
||||
retry_timeout: float,
|
||||
progress: _UploadProgress,
|
||||
) -> None:
|
||||
"""PUT one part to COS, then call ``upload_part_finish``."""
|
||||
part_index = part.index
|
||||
# Per-part block_size wins; fall back to the response-level value.
|
||||
actual_block_size = part.block_size if part.block_size > 0 else rsp_block_size
|
||||
offset = (part_index - 1) * rsp_block_size
|
||||
length = min(actual_block_size, file_size - offset)
|
||||
|
||||
# Read this slice of the file (blocking → executor).
|
||||
data = await asyncio.get_running_loop().run_in_executor(
|
||||
None, _read_file_chunk, file_path, offset, length
|
||||
)
|
||||
md5_hex = hashlib.md5(data).hexdigest()
|
||||
|
||||
logger.debug(
|
||||
"[%s] Part %d/%d: uploading %s (offset=%d md5=%s)",
|
||||
self._log_tag, part_index, progress.total_parts,
|
||||
format_size(length), offset, md5_hex,
|
||||
)
|
||||
|
||||
await self._put_to_presigned_url(
|
||||
part.presigned_url, data, part_index, progress.total_parts
|
||||
)
|
||||
await self._part_finish_with_retry(
|
||||
chat_type, target_id, upload_id,
|
||||
part_index, length, md5_hex, retry_timeout,
|
||||
)
|
||||
|
||||
progress.completed_parts += 1
|
||||
progress.uploaded_bytes += length
|
||||
logger.debug(
|
||||
"[%s] Part %d/%d done (%d/%d total)",
|
||||
self._log_tag, part_index, progress.total_parts,
|
||||
progress.completed_parts, progress.total_parts,
|
||||
)
|
||||
|
||||
async def _put_to_presigned_url(
|
||||
self,
|
||||
url: str,
|
||||
data: bytes,
|
||||
part_index: int,
|
||||
total_parts: int,
|
||||
) -> None:
|
||||
"""PUT part data to a pre-signed COS URL with retry."""
|
||||
last_exc: Optional[Exception] = None
|
||||
for attempt in range(_PART_UPLOAD_MAX_RETRIES + 1):
|
||||
try:
|
||||
resp = await asyncio.wait_for(
|
||||
self._http_put(
|
||||
url,
|
||||
data=data,
|
||||
headers={"Content-Length": str(len(data))},
|
||||
),
|
||||
timeout=_PART_UPLOAD_TIMEOUT,
|
||||
)
|
||||
# Caller's http_put is expected to return an httpx-like response.
|
||||
status = getattr(resp, "status_code", 0)
|
||||
if 200 <= status < 300:
|
||||
logger.debug(
|
||||
"[%s] PUT part %d/%d: %d OK",
|
||||
self._log_tag, part_index, total_parts, status,
|
||||
)
|
||||
return
|
||||
body_preview = ""
|
||||
try:
|
||||
body_preview = getattr(resp, "text", "")[:200]
|
||||
except Exception: # pragma: no cover — defensive
|
||||
pass
|
||||
raise RuntimeError(
|
||||
f"COS PUT returned {status}: {body_preview}"
|
||||
)
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt < _PART_UPLOAD_MAX_RETRIES:
|
||||
delay = 1.0 * (2 ** attempt)
|
||||
logger.warning(
|
||||
"[%s] PUT part %d/%d attempt %d failed, retry in %.1fs: %s",
|
||||
self._log_tag, part_index, total_parts,
|
||||
attempt + 1, delay, exc,
|
||||
)
|
||||
await asyncio.sleep(delay)
|
||||
raise RuntimeError(
|
||||
f"Part {part_index}/{total_parts} upload failed after "
|
||||
f"{_PART_UPLOAD_MAX_RETRIES + 1} attempts: {last_exc}"
|
||||
)
|
||||
|
||||
async def _part_finish_with_retry(
|
||||
self,
|
||||
chat_type: str,
|
||||
target_id: str,
|
||||
upload_id: str,
|
||||
part_index: int,
|
||||
block_size: int,
|
||||
md5: str,
|
||||
retry_timeout: float,
|
||||
) -> None:
|
||||
"""Call ``upload_part_finish``, retrying on biz_code 40093001."""
|
||||
base = "/v2/users" if chat_type == "c2c" else "/v2/groups"
|
||||
path = f"{base}/{target_id}/upload_part_finish"
|
||||
body = {
|
||||
"upload_id": upload_id,
|
||||
"part_index": part_index,
|
||||
"block_size": block_size,
|
||||
"md5": md5,
|
||||
}
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
start = loop.time()
|
||||
attempt = 0
|
||||
while True:
|
||||
try:
|
||||
await self._api_request(
|
||||
"POST", path, body=body, timeout=FILE_UPLOAD_TIMEOUT
|
||||
)
|
||||
return
|
||||
except RuntimeError as exc:
|
||||
err_msg = str(exc)
|
||||
if f"{_BIZ_CODE_PART_RETRYABLE}" not in err_msg:
|
||||
raise
|
||||
elapsed = loop.time() - start
|
||||
if elapsed >= retry_timeout:
|
||||
raise RuntimeError(
|
||||
f"upload_part_finish persistent retry timed out "
|
||||
f"after {retry_timeout:.0f}s ({attempt} retries): {exc}"
|
||||
) from exc
|
||||
attempt += 1
|
||||
logger.debug(
|
||||
"[%s] part_finish retryable error, attempt %d, "
|
||||
"elapsed=%.1fs: %s",
|
||||
self._log_tag, attempt, elapsed, exc,
|
||||
)
|
||||
await asyncio.sleep(_PART_FINISH_RETRY_INTERVAL)
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────
|
||||
# Step 3 — complete_upload
|
||||
# ──────────────────────────────────────────────────────────────────
|
||||
|
||||
async def _complete(
|
||||
self,
|
||||
chat_type: str,
|
||||
target_id: str,
|
||||
upload_id: str,
|
||||
) -> Dict[str, Any]:
|
||||
"""Call ``complete_upload`` with retry.
|
||||
|
||||
This reuses the ``/files`` endpoint (same as the simple URL-based upload)
|
||||
but signals the chunked-completion path by sending only ``upload_id``.
|
||||
"""
|
||||
base = "/v2/users" if chat_type == "c2c" else "/v2/groups"
|
||||
path = f"{base}/{target_id}/files"
|
||||
body = {"upload_id": upload_id}
|
||||
|
||||
last_exc: Optional[Exception] = None
|
||||
for attempt in range(_COMPLETE_UPLOAD_MAX_RETRIES + 1):
|
||||
try:
|
||||
return await self._api_request(
|
||||
"POST", path, body=body, timeout=FILE_UPLOAD_TIMEOUT
|
||||
)
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt < _COMPLETE_UPLOAD_MAX_RETRIES:
|
||||
delay = _COMPLETE_UPLOAD_BASE_DELAY * (2 ** attempt)
|
||||
logger.warning(
|
||||
"[%s] complete_upload attempt %d failed, "
|
||||
"retry in %.1fs: %s",
|
||||
self._log_tag, attempt + 1, delay, exc,
|
||||
)
|
||||
await asyncio.sleep(delay)
|
||||
raise RuntimeError(
|
||||
f"complete_upload failed after "
|
||||
f"{_COMPLETE_UPLOAD_MAX_RETRIES + 1} attempts: {last_exc}"
|
||||
)
|
||||
|
||||
|
||||
# ── Helpers (module-level for testability) ───────────────────────────
|
||||
|
||||
def format_size(size_bytes: int) -> str:
|
||||
"""Return a human-readable file size string (e.g. ``'12.3 MB'``)."""
|
||||
size = float(size_bytes)
|
||||
for unit in ("B", "KB", "MB", "GB"):
|
||||
if size < 1024.0:
|
||||
return f"{size:.1f} {unit}"
|
||||
size /= 1024.0
|
||||
return f"{size:.1f} TB"
|
||||
|
||||
|
||||
def _read_file_chunk(file_path: str, offset: int, length: int) -> bytes:
|
||||
"""Read *length* bytes from *file_path* starting at *offset*.
|
||||
|
||||
:raises IOError: If fewer bytes were read than expected (truncated file).
|
||||
"""
|
||||
with open(file_path, "rb") as fh:
|
||||
fh.seek(offset)
|
||||
data = fh.read(length)
|
||||
if len(data) != length:
|
||||
raise IOError(
|
||||
f"Short read from {file_path}: expected {length} bytes at "
|
||||
f"offset {offset}, got {len(data)} (file may be truncated)"
|
||||
)
|
||||
return data
|
||||
|
||||
|
||||
def _compute_file_hashes(file_path: str, file_size: int) -> Dict[str, str]:
|
||||
"""Compute md5, sha1, and md5_10m in a single pass."""
|
||||
md5 = hashlib.md5()
|
||||
sha1 = hashlib.sha1()
|
||||
md5_10m = hashlib.md5()
|
||||
|
||||
need_10m = file_size > _MD5_10M_SIZE
|
||||
bytes_read = 0
|
||||
|
||||
with open(file_path, "rb") as fh:
|
||||
while True:
|
||||
chunk = fh.read(65536)
|
||||
if not chunk:
|
||||
break
|
||||
md5.update(chunk)
|
||||
sha1.update(chunk)
|
||||
if need_10m:
|
||||
remaining = _MD5_10M_SIZE - bytes_read
|
||||
if remaining > 0:
|
||||
md5_10m.update(chunk[:remaining])
|
||||
bytes_read += len(chunk)
|
||||
|
||||
full_md5 = md5.hexdigest()
|
||||
return {
|
||||
"md5": full_md5,
|
||||
"sha1": sha1.hexdigest(),
|
||||
# For small files the "10m" hash is just the full md5.
|
||||
"md5_10m": md5_10m.hexdigest() if need_10m else full_md5,
|
||||
}
|
||||
|
||||
|
||||
async def _run_with_concurrency(
|
||||
tasks: List[Callable[[], Awaitable[None]]],
|
||||
concurrency: int,
|
||||
) -> None:
|
||||
"""Run a list of thunks with a bounded number in flight at once."""
|
||||
concurrency = max(concurrency, 1)
|
||||
sem = asyncio.Semaphore(concurrency)
|
||||
|
||||
async def _wrap(thunk: Callable[[], Awaitable[None]]) -> None:
|
||||
async with sem:
|
||||
await thunk()
|
||||
|
||||
await asyncio.gather(*(_wrap(t) for t in tasks))
|
||||
@@ -0,0 +1,74 @@
|
||||
"""QQBot package-level constants shared across adapter, onboard, and other modules."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# QQBot adapter version — bump on functional changes to the adapter package.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
QQBOT_VERSION = "1.1.0"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# API endpoints
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# The portal domain is configurable via QQ_API_HOST for corporate proxies
|
||||
# or test environments. Default: q.qq.com (production).
|
||||
PORTAL_HOST = os.getenv("QQ_PORTAL_HOST", "q.qq.com")
|
||||
|
||||
API_BASE = "https://api.sgroup.qq.com"
|
||||
TOKEN_URL = "https://bots.qq.com/app/getAppAccessToken"
|
||||
GATEWAY_URL_PATH = "/gateway"
|
||||
|
||||
# QR-code onboard endpoints (on the portal host)
|
||||
ONBOARD_CREATE_PATH = "/lite/create_bind_task"
|
||||
ONBOARD_POLL_PATH = "/lite/poll_bind_result"
|
||||
QR_URL_TEMPLATE = (
|
||||
"https://q.qq.com/qqbot/openclaw/connect.html"
|
||||
"?task_id={task_id}&_wv=2&source=hermes"
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Timeouts & retry
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
DEFAULT_API_TIMEOUT = 30.0
|
||||
FILE_UPLOAD_TIMEOUT = 120.0
|
||||
CONNECT_TIMEOUT_SECONDS = 20.0
|
||||
|
||||
RECONNECT_BACKOFF = [2, 5, 10, 30, 60]
|
||||
MAX_RECONNECT_ATTEMPTS = 100
|
||||
RATE_LIMIT_DELAY = 60 # seconds
|
||||
QUICK_DISCONNECT_THRESHOLD = 5.0 # seconds
|
||||
MAX_QUICK_DISCONNECT_COUNT = 3
|
||||
|
||||
ONBOARD_POLL_INTERVAL = 2.0 # seconds between poll_bind_result calls
|
||||
ONBOARD_API_TIMEOUT = 10.0
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Message limits
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
MAX_MESSAGE_LENGTH = 4000
|
||||
DEDUP_WINDOW_SECONDS = 300
|
||||
DEDUP_MAX_SIZE = 1000
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# QQ Bot message types
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
MSG_TYPE_TEXT = 0
|
||||
MSG_TYPE_MARKDOWN = 2
|
||||
MSG_TYPE_MEDIA = 7
|
||||
MSG_TYPE_INPUT_NOTIFY = 6
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# QQ Bot file media types
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
MEDIA_TYPE_IMAGE = 1
|
||||
MEDIA_TYPE_VIDEO = 2
|
||||
MEDIA_TYPE_VOICE = 3
|
||||
MEDIA_TYPE_FILE = 4
|
||||
@@ -0,0 +1,45 @@
|
||||
"""AES-256-GCM utilities for QQBot scan-to-configure credential decryption."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import os
|
||||
|
||||
|
||||
def generate_bind_key() -> str:
|
||||
"""Generate a 256-bit random AES key and return it as base64.
|
||||
|
||||
The key is passed to ``create_bind_task`` so the server can encrypt
|
||||
the bot's *client_secret* before returning it. Only this CLI holds
|
||||
the key, ensuring the secret never travels in plaintext.
|
||||
"""
|
||||
return base64.b64encode(os.urandom(32)).decode()
|
||||
|
||||
|
||||
def decrypt_secret(encrypted_base64: str, key_base64: str) -> str:
|
||||
"""Decrypt a base64-encoded AES-256-GCM ciphertext.
|
||||
|
||||
Ciphertext layout (after base64-decoding)::
|
||||
|
||||
IV (12 bytes) ‖ ciphertext (N bytes) ‖ AuthTag (16 bytes)
|
||||
|
||||
Args:
|
||||
encrypted_base64: The ``bot_encrypt_secret`` value from
|
||||
``poll_bind_result``.
|
||||
key_base64: The base64 AES key generated by
|
||||
:func:`generate_bind_key`.
|
||||
|
||||
Returns:
|
||||
The decrypted *client_secret* as a UTF-8 string.
|
||||
"""
|
||||
from cryptography.hazmat.primitives.ciphers.aead import AESGCM
|
||||
|
||||
key = base64.b64decode(key_base64)
|
||||
raw = base64.b64decode(encrypted_base64)
|
||||
|
||||
iv = raw[:12]
|
||||
ciphertext_with_tag = raw[12:] # AESGCM expects ciphertext + tag concatenated
|
||||
|
||||
aesgcm = AESGCM(key)
|
||||
plaintext = aesgcm.decrypt(iv, ciphertext_with_tag, None)
|
||||
return plaintext.decode("utf-8")
|
||||
@@ -0,0 +1,473 @@
|
||||
"""QQ Bot inline keyboards + approval / update-prompt senders.
|
||||
|
||||
QQ Bot v2 supports attaching inline keyboards to outbound messages. When a
|
||||
user clicks a button, the platform dispatches an ``INTERACTION_CREATE``
|
||||
gateway event containing the button's ``data`` payload. The bot must ACK the
|
||||
interaction promptly via ``PUT /interactions/{id}`` or the user sees an
|
||||
error indicator on the button.
|
||||
|
||||
This module provides:
|
||||
|
||||
- :class:`InlineKeyboard` + button dataclasses — serialized into the
|
||||
``keyboard`` field of the outbound message body.
|
||||
- :func:`build_approval_keyboard` — 3-button ✅ once / ⭐ always / ❌ deny
|
||||
keyboard for tool-approval flows.
|
||||
- :func:`build_update_prompt_keyboard` — Yes/No keyboard for update confirms.
|
||||
- :func:`parse_approval_button_data` / :func:`parse_update_prompt_button_data`
|
||||
— decode the ``button_data`` payload from ``INTERACTION_CREATE``.
|
||||
- :class:`ApprovalRequest` + :class:`ApprovalSender` — high-level helper that
|
||||
builds an approval message with keyboard and posts it to a c2c / group chat.
|
||||
|
||||
``button_data`` formats::
|
||||
|
||||
approve:<session_key>:<decision> # decision = allow-once|allow-always|deny
|
||||
update_prompt:<answer> # answer = y|n
|
||||
|
||||
Ported from WideLee's qqbot-agent-sdk v1.2.2 (``approval.py`` + ``dto.py``
|
||||
keyboard types). Authorship preserved via Co-authored-by.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Awaitable, Callable, Dict, List, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ── button_data prefixes + patterns ──────────────────────────────────
|
||||
|
||||
APPROVAL_BUTTON_PREFIX = "approve:"
|
||||
UPDATE_PROMPT_PREFIX = "update_prompt:"
|
||||
|
||||
# Pattern: approve:<session_key>:<decision>
|
||||
# session_key may itself contain colons (e.g. agent:main:qqbot:c2c:OPENID),
|
||||
# so the session_key group is greedy but trails the decision.
|
||||
_APPROVAL_DATA_RE = re.compile(
|
||||
r"^approve:(.+):(allow-once|allow-always|deny)$"
|
||||
)
|
||||
|
||||
# Pattern: update_prompt:y | update_prompt:n
|
||||
_UPDATE_PROMPT_RE = re.compile(r"^update_prompt:(y|n)$")
|
||||
|
||||
|
||||
# ── Keyboard dataclasses ─────────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class KeyboardButtonPermission:
|
||||
"""Button permission metadata. ``type=2`` means all users can click."""
|
||||
type: int = 2
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {"type": self.type}
|
||||
|
||||
|
||||
@dataclass
|
||||
class KeyboardButtonAction:
|
||||
"""What happens when the button is clicked.
|
||||
|
||||
:param type: ``1`` (Callback — triggers ``INTERACTION_CREATE``) or
|
||||
``2`` (Link — opens a URL).
|
||||
:param data: Payload delivered in ``data.resolved.button_data`` when
|
||||
``type=1``.
|
||||
:param permission: :class:`KeyboardButtonPermission`.
|
||||
:param click_limit: Max clicks per user (``1`` = single-use).
|
||||
"""
|
||||
type: int
|
||||
data: str
|
||||
permission: KeyboardButtonPermission = field(
|
||||
default_factory=KeyboardButtonPermission
|
||||
)
|
||||
click_limit: int = 1
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"type": self.type,
|
||||
"data": self.data,
|
||||
"permission": self.permission.to_dict(),
|
||||
"click_limit": self.click_limit,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class KeyboardButtonRenderData:
|
||||
"""Visual rendering of a button.
|
||||
|
||||
:param label: Pre-click label.
|
||||
:param visited_label: Post-click label (button stays greyed in place).
|
||||
:param style: ``0`` = grey, ``1`` = blue.
|
||||
"""
|
||||
label: str
|
||||
visited_label: str
|
||||
style: int = 1
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"label": self.label,
|
||||
"visited_label": self.visited_label,
|
||||
"style": self.style,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class KeyboardButton:
|
||||
"""One button in a keyboard.
|
||||
|
||||
:param group_id: Buttons sharing a ``group_id`` are mutually exclusive —
|
||||
clicking one greys the rest.
|
||||
"""
|
||||
id: str
|
||||
render_data: KeyboardButtonRenderData
|
||||
action: KeyboardButtonAction
|
||||
group_id: str = "default"
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {
|
||||
"id": self.id,
|
||||
"render_data": self.render_data.to_dict(),
|
||||
"action": self.action.to_dict(),
|
||||
"group_id": self.group_id,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class KeyboardRow:
|
||||
buttons: List[KeyboardButton] = field(default_factory=list)
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {"buttons": [b.to_dict() for b in self.buttons]}
|
||||
|
||||
|
||||
@dataclass
|
||||
class KeyboardContent:
|
||||
rows: List[KeyboardRow] = field(default_factory=list)
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {"rows": [r.to_dict() for r in self.rows]}
|
||||
|
||||
|
||||
@dataclass
|
||||
class InlineKeyboard:
|
||||
"""Top-level keyboard payload — goes into ``MessageToCreate.keyboard``."""
|
||||
content: KeyboardContent = field(default_factory=KeyboardContent)
|
||||
|
||||
def to_dict(self) -> Dict[str, Any]:
|
||||
return {"content": self.content.to_dict()}
|
||||
|
||||
|
||||
# ── INTERACTION_CREATE parsing ───────────────────────────────────────
|
||||
|
||||
def parse_approval_button_data(button_data: str) -> Optional[tuple[str, str]]:
|
||||
"""Parse approval ``button_data`` into ``(session_key, decision)``.
|
||||
|
||||
:param button_data: Raw ``data.resolved.button_data`` from
|
||||
``INTERACTION_CREATE``.
|
||||
:returns: ``(session_key, decision)`` or ``None`` if not an approval button.
|
||||
"""
|
||||
m = _APPROVAL_DATA_RE.match(button_data or "")
|
||||
if not m:
|
||||
return None
|
||||
return m.group(1), m.group(2)
|
||||
|
||||
|
||||
def parse_update_prompt_button_data(button_data: str) -> Optional[str]:
|
||||
"""Parse update-prompt ``button_data`` into ``'y'`` or ``'n'``."""
|
||||
m = _UPDATE_PROMPT_RE.match(button_data or "")
|
||||
if not m:
|
||||
return None
|
||||
return m.group(1)
|
||||
|
||||
|
||||
# ── Keyboard builders ────────────────────────────────────────────────
|
||||
|
||||
def _make_callback_button(
|
||||
btn_id: str,
|
||||
label: str,
|
||||
visited_label: str,
|
||||
data: str,
|
||||
style: int,
|
||||
group_id: str,
|
||||
) -> KeyboardButton:
|
||||
return KeyboardButton(
|
||||
id=btn_id,
|
||||
render_data=KeyboardButtonRenderData(
|
||||
label=label,
|
||||
visited_label=visited_label,
|
||||
style=style,
|
||||
),
|
||||
action=KeyboardButtonAction(type=1, data=data),
|
||||
group_id=group_id,
|
||||
)
|
||||
|
||||
|
||||
def build_approval_keyboard(session_key: str) -> InlineKeyboard:
|
||||
"""Build the 3-button approval keyboard.
|
||||
|
||||
Layout: ``[✅ 允许一次] [⭐ 始终允许] [❌ 拒绝]`` — all three share
|
||||
``group_id='approval'`` so clicking one greys out the rest.
|
||||
|
||||
:param session_key: Embedded into ``button_data`` so the decision
|
||||
routes back to the right pending approval.
|
||||
"""
|
||||
return InlineKeyboard(
|
||||
content=KeyboardContent(
|
||||
rows=[
|
||||
KeyboardRow(buttons=[
|
||||
_make_callback_button(
|
||||
btn_id="allow",
|
||||
label="✅ 允许一次",
|
||||
visited_label="已允许",
|
||||
data=f"{APPROVAL_BUTTON_PREFIX}{session_key}:allow-once",
|
||||
style=1,
|
||||
group_id="approval",
|
||||
),
|
||||
_make_callback_button(
|
||||
btn_id="always",
|
||||
label="⭐ 始终允许",
|
||||
visited_label="已始终允许",
|
||||
data=f"{APPROVAL_BUTTON_PREFIX}{session_key}:allow-always",
|
||||
style=1,
|
||||
group_id="approval",
|
||||
),
|
||||
_make_callback_button(
|
||||
btn_id="deny",
|
||||
label="❌ 拒绝",
|
||||
visited_label="已拒绝",
|
||||
data=f"{APPROVAL_BUTTON_PREFIX}{session_key}:deny",
|
||||
style=0,
|
||||
group_id="approval",
|
||||
),
|
||||
]),
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def build_update_prompt_keyboard() -> InlineKeyboard:
|
||||
"""Build a Yes/No keyboard for update confirmation prompts."""
|
||||
return InlineKeyboard(
|
||||
content=KeyboardContent(
|
||||
rows=[
|
||||
KeyboardRow(buttons=[
|
||||
_make_callback_button(
|
||||
btn_id="yes",
|
||||
label="✓ 确认",
|
||||
visited_label="已确认",
|
||||
data=f"{UPDATE_PROMPT_PREFIX}y",
|
||||
style=1,
|
||||
group_id="update_prompt",
|
||||
),
|
||||
_make_callback_button(
|
||||
btn_id="no",
|
||||
label="✗ 取消",
|
||||
visited_label="已取消",
|
||||
data=f"{UPDATE_PROMPT_PREFIX}n",
|
||||
style=0,
|
||||
group_id="update_prompt",
|
||||
),
|
||||
]),
|
||||
]
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
# ── ApprovalRequest + text builder ───────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class ApprovalRequest:
|
||||
"""Structured approval-request display data.
|
||||
|
||||
:param session_key: Routes the decision back to the waiting caller.
|
||||
:param title: Short title at the top.
|
||||
:param description: Optional longer description.
|
||||
:param command_preview: Command text (exec approvals).
|
||||
:param cwd: Working directory (exec approvals).
|
||||
:param tool_name: Tool name (plugin approvals).
|
||||
:param severity: ``'critical' | 'info' | ''``.
|
||||
:param timeout_sec: Seconds until the approval expires.
|
||||
"""
|
||||
session_key: str
|
||||
title: str
|
||||
description: str = ""
|
||||
command_preview: str = ""
|
||||
cwd: str = ""
|
||||
tool_name: str = ""
|
||||
severity: str = ""
|
||||
timeout_sec: int = 120
|
||||
|
||||
|
||||
def build_approval_text(req: ApprovalRequest) -> str:
|
||||
"""Render an :class:`ApprovalRequest` into the message body (markdown)."""
|
||||
if req.command_preview or req.cwd:
|
||||
return _build_exec_text(req)
|
||||
return _build_plugin_text(req)
|
||||
|
||||
|
||||
def _build_exec_text(req: ApprovalRequest) -> str:
|
||||
lines: List[str] = ["🔐 **命令执行审批**", ""]
|
||||
if req.command_preview:
|
||||
preview = req.command_preview[:300]
|
||||
lines.append(f"```\n{preview}\n```")
|
||||
if req.cwd:
|
||||
lines.append(f"📁 目录: {req.cwd}")
|
||||
if req.title and req.title != req.command_preview:
|
||||
lines.append(f"📋 {req.title}")
|
||||
if req.description:
|
||||
lines.append(f"📝 {req.description}")
|
||||
lines.append("")
|
||||
lines.append(f"⏱️ 超时: {req.timeout_sec} 秒")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _build_plugin_text(req: ApprovalRequest) -> str:
|
||||
icon = (
|
||||
"🔴" if req.severity == "critical"
|
||||
else "🔵" if req.severity == "info"
|
||||
else "🟡"
|
||||
)
|
||||
lines: List[str] = [f"{icon} **审批请求**", ""]
|
||||
lines.append(f"📋 {req.title}")
|
||||
if req.description:
|
||||
lines.append(f"📝 {req.description}")
|
||||
if req.tool_name:
|
||||
lines.append(f"🔧 工具: {req.tool_name}")
|
||||
lines.append("")
|
||||
lines.append(f"⏱️ 超时: {req.timeout_sec} 秒")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
# ── ApprovalSender ───────────────────────────────────────────────────
|
||||
|
||||
PostMessageFn = Callable[..., Awaitable[Dict[str, Any]]]
|
||||
"""Signature of an async POST to ``/v2/{users|groups}/{id}/messages``.
|
||||
|
||||
Implementations accept a body dict and return the raw API response.
|
||||
"""
|
||||
|
||||
|
||||
class ApprovalSender:
|
||||
"""Send an approval-request message with an inline keyboard.
|
||||
|
||||
Decoupled from the adapter via callables so it can be unit-tested in
|
||||
isolation. Pass the adapter's ``_send_message_with_keyboard`` helper
|
||||
(or any equivalent) as ``post_message``.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
post_c2c: PostMessageFn,
|
||||
post_group: PostMessageFn,
|
||||
log_tag: str = "QQBot",
|
||||
) -> None:
|
||||
self._post_c2c = post_c2c
|
||||
self._post_group = post_group
|
||||
self._log_tag = log_tag
|
||||
|
||||
async def send(
|
||||
self,
|
||||
chat_type: str,
|
||||
chat_id: str,
|
||||
req: ApprovalRequest,
|
||||
msg_id: Optional[str] = None,
|
||||
) -> bool:
|
||||
"""Send an approval message to *chat_id*.
|
||||
|
||||
:param chat_type: ``'c2c'`` or ``'group'``.
|
||||
:param chat_id: User openid or group openid.
|
||||
:param req: :class:`ApprovalRequest`.
|
||||
:param msg_id: Reply-to message id (required for passive messages).
|
||||
:returns: ``True`` on success, ``False`` on failure.
|
||||
"""
|
||||
text = build_approval_text(req)
|
||||
keyboard = build_approval_keyboard(req.session_key)
|
||||
|
||||
logger.info(
|
||||
"[%s] Sending approval request to %s:%s (session=%.20s…)",
|
||||
self._log_tag, chat_type, chat_id, req.session_key,
|
||||
)
|
||||
|
||||
try:
|
||||
if chat_type == "c2c":
|
||||
await self._post_c2c(chat_id, text, msg_id, keyboard)
|
||||
elif chat_type == "group":
|
||||
await self._post_group(chat_id, text, msg_id, keyboard)
|
||||
else:
|
||||
logger.warning(
|
||||
"[%s] Approval: unsupported chat_type %r",
|
||||
self._log_tag, chat_type,
|
||||
)
|
||||
return False
|
||||
logger.info(
|
||||
"[%s] Approval message sent to %s:%s",
|
||||
self._log_tag, chat_type, chat_id,
|
||||
)
|
||||
return True
|
||||
except Exception as exc:
|
||||
logger.error(
|
||||
"[%s] Failed to send approval message to %s:%s: %s",
|
||||
self._log_tag, chat_type, chat_id, exc,
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
# ── INTERACTION_CREATE event shape ───────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class InteractionEvent:
|
||||
"""Parsed ``INTERACTION_CREATE`` event payload.
|
||||
|
||||
See https://bot.q.qq.com/wiki/develop/api-v2/dev-prepare/interface-framework/event-emit.html
|
||||
"""
|
||||
id: str = ""
|
||||
"""Interaction event id — required for the ``PUT /interactions/{id}`` ACK."""
|
||||
|
||||
type: int = 0
|
||||
"""Event type code (``11`` = message button)."""
|
||||
|
||||
chat_type: int = 0
|
||||
"""``0`` = guild, ``1`` = group, ``2`` = c2c."""
|
||||
|
||||
scene: str = ""
|
||||
"""``'guild'`` | ``'group'`` | ``'c2c'`` — human-readable scene."""
|
||||
|
||||
group_openid: str = ""
|
||||
group_member_openid: str = ""
|
||||
user_openid: str = ""
|
||||
channel_id: str = ""
|
||||
guild_id: str = ""
|
||||
|
||||
button_data: str = ""
|
||||
button_id: str = ""
|
||||
resolver_user_id: str = ""
|
||||
|
||||
@property
|
||||
def operator_openid(self) -> str:
|
||||
"""Best available operator openid (group → member; c2c → user)."""
|
||||
return (
|
||||
self.group_member_openid
|
||||
or self.user_openid
|
||||
or self.resolver_user_id
|
||||
)
|
||||
|
||||
|
||||
def parse_interaction_event(raw: Dict[str, Any]) -> InteractionEvent:
|
||||
"""Parse a raw ``INTERACTION_CREATE`` dispatch payload (``d``)."""
|
||||
data_raw = raw.get("data") or {}
|
||||
resolved = data_raw.get("resolved") or {}
|
||||
scene_code = int(raw.get("chat_type", 0) or 0)
|
||||
scene = {0: "guild", 1: "group", 2: "c2c"}.get(scene_code, "")
|
||||
return InteractionEvent(
|
||||
id=str(raw.get("id", "")),
|
||||
type=int(data_raw.get("type", 0) or 0),
|
||||
chat_type=scene_code,
|
||||
scene=scene,
|
||||
group_openid=str(raw.get("group_openid", "")),
|
||||
group_member_openid=str(raw.get("group_member_openid", "")),
|
||||
user_openid=str(raw.get("user_openid", "")),
|
||||
channel_id=str(raw.get("channel_id", "")),
|
||||
guild_id=str(raw.get("guild_id", "")),
|
||||
button_data=str(resolved.get("button_data", "")),
|
||||
button_id=str(resolved.get("button_id", "")),
|
||||
resolver_user_id=str(resolved.get("user_id", "")),
|
||||
)
|
||||
@@ -0,0 +1,220 @@
|
||||
"""
|
||||
QQBot scan-to-configure (QR code onboard) module.
|
||||
|
||||
Mirrors the Feishu onboarding pattern: synchronous HTTP + a single public
|
||||
entry-point ``qr_register()`` that handles the full flow (create task →
|
||||
display QR code → poll → decrypt credentials).
|
||||
|
||||
Calls the ``q.qq.com`` ``create_bind_task`` / ``poll_bind_result`` APIs to
|
||||
generate a QR-code URL and poll for scan completion. On success the caller
|
||||
receives the bot's *app_id*, *client_secret* (decrypted locally), and the
|
||||
scanner's *user_openid* — enough to fully configure the QQBot gateway.
|
||||
|
||||
Reference: https://bot.q.qq.com/wiki/develop/api-v2/
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import time
|
||||
from enum import IntEnum
|
||||
from typing import Optional, Tuple
|
||||
from urllib.parse import quote
|
||||
|
||||
from .constants import (
|
||||
ONBOARD_API_TIMEOUT,
|
||||
ONBOARD_CREATE_PATH,
|
||||
ONBOARD_POLL_INTERVAL,
|
||||
ONBOARD_POLL_PATH,
|
||||
PORTAL_HOST,
|
||||
QR_URL_TEMPLATE,
|
||||
)
|
||||
from .crypto import decrypt_secret, generate_bind_key
|
||||
from .utils import get_api_headers
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Bind status
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class BindStatus(IntEnum):
|
||||
"""Status codes returned by ``_poll_bind_result``."""
|
||||
|
||||
NONE = 0
|
||||
PENDING = 1
|
||||
COMPLETED = 2
|
||||
EXPIRED = 3
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# QR rendering
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
try:
|
||||
import qrcode as _qrcode_mod
|
||||
except (ImportError, TypeError):
|
||||
_qrcode_mod = None # type: ignore[assignment]
|
||||
|
||||
|
||||
def _render_qr(url: str) -> bool:
|
||||
"""Try to render a QR code in the terminal. Returns True if successful."""
|
||||
if _qrcode_mod is None:
|
||||
return False
|
||||
try:
|
||||
qr = _qrcode_mod.QRCode(
|
||||
error_correction=_qrcode_mod.constants.ERROR_CORRECT_M,
|
||||
border=2,
|
||||
)
|
||||
qr.add_data(url)
|
||||
qr.make(fit=True)
|
||||
qr.print_ascii(invert=True)
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Synchronous HTTP helpers (mirrors Feishu _post_registration pattern)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _create_bind_task(timeout: float = ONBOARD_API_TIMEOUT) -> Tuple[str, str]:
|
||||
"""Create a bind task and return *(task_id, aes_key_base64)*.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If the API returns a non-zero ``retcode``.
|
||||
"""
|
||||
import httpx
|
||||
|
||||
url = f"https://{PORTAL_HOST}{ONBOARD_CREATE_PATH}"
|
||||
key = generate_bind_key()
|
||||
|
||||
with httpx.Client(timeout=timeout, follow_redirects=True) as client:
|
||||
resp = client.post(url, json={"key": key}, headers=get_api_headers())
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
|
||||
if data.get("retcode") != 0:
|
||||
raise RuntimeError(data.get("msg", "create_bind_task failed"))
|
||||
|
||||
task_id = data.get("data", {}).get("task_id")
|
||||
if not task_id:
|
||||
raise RuntimeError("create_bind_task: missing task_id in response")
|
||||
|
||||
logger.debug("create_bind_task ok: task_id=%s", task_id)
|
||||
return task_id, key
|
||||
|
||||
|
||||
def _poll_bind_result(
|
||||
task_id: str,
|
||||
timeout: float = ONBOARD_API_TIMEOUT,
|
||||
) -> Tuple[BindStatus, str, str, str]:
|
||||
"""Poll the bind result for *task_id*.
|
||||
|
||||
Returns:
|
||||
A 4-tuple of ``(status, bot_appid, bot_encrypt_secret, user_openid)``.
|
||||
|
||||
Raises:
|
||||
RuntimeError: If the API returns a non-zero ``retcode``.
|
||||
"""
|
||||
import httpx
|
||||
|
||||
url = f"https://{PORTAL_HOST}{ONBOARD_POLL_PATH}"
|
||||
|
||||
with httpx.Client(timeout=timeout, follow_redirects=True) as client:
|
||||
resp = client.post(url, json={"task_id": task_id}, headers=get_api_headers())
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
|
||||
if data.get("retcode") != 0:
|
||||
raise RuntimeError(data.get("msg", "poll_bind_result failed"))
|
||||
|
||||
d = data.get("data", {})
|
||||
return (
|
||||
BindStatus(d.get("status", 0)),
|
||||
str(d.get("bot_appid", "")),
|
||||
d.get("bot_encrypt_secret", ""),
|
||||
d.get("user_openid", ""),
|
||||
)
|
||||
|
||||
|
||||
def build_connect_url(task_id: str) -> str:
|
||||
"""Build the QR-code target URL for a given *task_id*."""
|
||||
return QR_URL_TEMPLATE.format(task_id=quote(task_id))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public entry-point
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_MAX_REFRESHES = 3
|
||||
|
||||
|
||||
def qr_register(timeout_seconds: int = 600) -> Optional[dict]:
|
||||
"""Run the QQBot scan-to-configure QR registration flow.
|
||||
|
||||
Mirrors ``feishu.qr_register()``: handles create → display → poll →
|
||||
decrypt in one call. Unexpected errors propagate to the caller.
|
||||
|
||||
:returns:
|
||||
``{"app_id": ..., "client_secret": ..., "user_openid": ...}`` on
|
||||
success, or ``None`` on failure / expiry / cancellation.
|
||||
"""
|
||||
deadline = time.monotonic() + timeout_seconds
|
||||
|
||||
for refresh_count in range(_MAX_REFRESHES + 1):
|
||||
# ── Create bind task ──
|
||||
try:
|
||||
task_id, aes_key = _create_bind_task()
|
||||
except Exception as exc:
|
||||
logger.warning("[QQBot onboard] Failed to create bind task: %s", exc)
|
||||
return None
|
||||
|
||||
url = build_connect_url(task_id)
|
||||
|
||||
# ── Display QR code + URL ──
|
||||
print()
|
||||
if _render_qr(url):
|
||||
print(f" Scan the QR code above, or open this URL directly:\n {url}")
|
||||
else:
|
||||
print(f" Open this URL in QQ on your phone:\n {url}")
|
||||
print(" Tip: pip install qrcode to display a scannable QR code here")
|
||||
print()
|
||||
|
||||
# ── Poll loop ──
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
status, app_id, encrypted_secret, user_openid = _poll_bind_result(task_id)
|
||||
except Exception:
|
||||
time.sleep(ONBOARD_POLL_INTERVAL)
|
||||
continue
|
||||
|
||||
if status == BindStatus.COMPLETED:
|
||||
client_secret = decrypt_secret(encrypted_secret, aes_key)
|
||||
print()
|
||||
print(f" QR scan complete! (App ID: {app_id})")
|
||||
if user_openid:
|
||||
print(f" Scanner's OpenID: {user_openid}")
|
||||
return {
|
||||
"app_id": app_id,
|
||||
"client_secret": client_secret,
|
||||
"user_openid": user_openid,
|
||||
}
|
||||
|
||||
if status == BindStatus.EXPIRED:
|
||||
if refresh_count >= _MAX_REFRESHES:
|
||||
logger.warning("[QQBot onboard] QR code expired %d times — giving up", _MAX_REFRESHES)
|
||||
return None
|
||||
print(f"\n QR code expired, refreshing... ({refresh_count + 1}/{_MAX_REFRESHES})")
|
||||
break # next for-loop iteration creates a new task
|
||||
|
||||
time.sleep(ONBOARD_POLL_INTERVAL)
|
||||
else:
|
||||
# deadline reached without completing
|
||||
logger.warning("[QQBot onboard] Poll timed out after %ds", timeout_seconds)
|
||||
return None
|
||||
|
||||
return None
|
||||
@@ -0,0 +1,71 @@
|
||||
"""QQBot shared utilities — User-Agent, HTTP helpers, config coercion."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import platform
|
||||
import sys
|
||||
from typing import Any, Dict, List
|
||||
|
||||
from .constants import QQBOT_VERSION
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# User-Agent
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _get_hermes_version() -> str:
|
||||
"""Return the hermes-agent package version, or 'dev' if unavailable."""
|
||||
try:
|
||||
from importlib.metadata import version
|
||||
return version("hermes-agent")
|
||||
except Exception:
|
||||
return "dev"
|
||||
|
||||
|
||||
def build_user_agent() -> str:
|
||||
"""Build a descriptive User-Agent string.
|
||||
|
||||
Format::
|
||||
|
||||
QQBotAdapter/<qqbot_version> (Python/<py_version>; <os>; Hermes/<hermes_version>)
|
||||
|
||||
Example::
|
||||
|
||||
QQBotAdapter/1.0.0 (Python/3.11.15; darwin; Hermes/0.9.0)
|
||||
"""
|
||||
py_version = f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}"
|
||||
os_name = platform.system().lower()
|
||||
hermes_version = _get_hermes_version()
|
||||
return f"QQBotAdapter/{QQBOT_VERSION} (Python/{py_version}; {os_name}; Hermes/{hermes_version})"
|
||||
|
||||
|
||||
def get_api_headers() -> Dict[str, str]:
|
||||
"""Return standard HTTP headers for QQBot API requests.
|
||||
|
||||
Includes ``Content-Type``, ``Accept``, and a dynamic ``User-Agent``.
|
||||
``q.qq.com`` requires ``Accept: application/json`` — without it,
|
||||
the server returns a JavaScript anti-bot challenge page.
|
||||
"""
|
||||
return {
|
||||
"Content-Type": "application/json",
|
||||
"Accept": "application/json",
|
||||
"User-Agent": build_user_agent(),
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Config helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def coerce_list(value: Any) -> List[str]:
|
||||
"""Coerce config values into a trimmed string list.
|
||||
|
||||
Accepts comma-separated strings, lists, tuples, sets, or single values.
|
||||
"""
|
||||
if value is None:
|
||||
return []
|
||||
if isinstance(value, str):
|
||||
return [item.strip() for item in value.split(",") if item.strip()]
|
||||
if isinstance(value, (list, tuple, set)):
|
||||
return [str(item).strip() for item in value if str(item).strip()]
|
||||
return [str(value).strip()] if str(value).strip() else []
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,369 @@
|
||||
"""
|
||||
Signal attachment rate-limit scheduler.
|
||||
|
||||
Process-wide token-bucket simulator that mirrors the per-account
|
||||
attachment rate limit signal-cli/Signal-Server enforce. Producers
|
||||
(``SignalAdapter.send_multiple_images`` and the ``send_message`` tool's
|
||||
Signal path) call ``acquire(n)`` before an attachment send; on a 429
|
||||
they call ``feedback(retry_after, n)`` so the model recalibrates from
|
||||
the server's authoritative hint.
|
||||
|
||||
The scheduler serializes concurrent calls through an ``asyncio.Lock``,
|
||||
giving FIFO fairness across agent sessions sharing one signal-cli
|
||||
daemon.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import re
|
||||
import time
|
||||
from typing import Any, Optional
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
SIGNAL_MAX_ATTACHMENTS_PER_MSG = 32 # per-message attachment cap (source: Signal-{Android,Desktop} source code)
|
||||
SIGNAL_RATE_LIMIT_BUCKET_CAPACITY = 50 # server-side token-bucket capacity for attachments rate limiting
|
||||
SIGNAL_RATE_LIMIT_DEFAULT_RETRY_AFTER = 4 # fallback token refill interval for signal-cli < v0.14.3
|
||||
SIGNAL_RATE_LIMIT_MAX_ATTEMPTS = 2 # initial attempt + 1 retry
|
||||
SIGNAL_BATCH_PACING_NOTICE_THRESHOLD = 10.0 # if estimated waiting time > 10s, notify the user about the delay
|
||||
SIGNAL_RPC_ERROR_RATELIMIT = -5 # signal-cli (v0.14.3+) JSON-RPC error code for RateLimitException
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Errors
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class SignalRateLimitError(Exception):
|
||||
"""
|
||||
Raised by ``SignalAdapter._rpc`` for rate-limit responses when the
|
||||
caller has opted in via ``raise_on_rate_limit=True``.
|
||||
|
||||
Carries the server-supplied per-token Retry-After (in seconds) on
|
||||
signal-cli ≥ v0.14.3
|
||||
``retry_after`` is None when the version doesn't expose it.
|
||||
"""
|
||||
|
||||
def __init__(self, message: str, retry_after: Optional[float] = None) -> None:
|
||||
super().__init__(message)
|
||||
self.retry_after = retry_after
|
||||
|
||||
|
||||
class SignalSchedulerError(Exception):
|
||||
pass
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Detection helpers — used to fish a 429 out of signal-cli's various error
|
||||
# shapes (typed code, [429] substring, libsignal-net RetryLaterException
|
||||
# leaked through AttachmentInvalidException).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# "Retry after 4 seconds" / "retry after 4 second" — libsignal-net's
|
||||
# RetryLaterException string form, surfaced when 429s hit during
|
||||
# attachment upload (signal-cli wraps these as AttachmentInvalidException
|
||||
# rather than RateLimitException, so the typed path doesn't fire).
|
||||
_RETRY_AFTER_RE = re.compile(r"Retry after (\d+(?:\.\d+)?)\s*second", re.IGNORECASE)
|
||||
|
||||
|
||||
def _extract_retry_after_seconds(err: Any) -> Optional[float]:
|
||||
"""Pull the per-token Retry-After window from a signal-cli rate-limit error.
|
||||
|
||||
Tries two sources, in order:
|
||||
1. ``error.data.response.results[*].retryAfterSeconds`` — the
|
||||
structured field signal-cli ≥ v0.14.3 surfaces for plain
|
||||
RateLimitException.
|
||||
2. ``"Retry after N seconds"`` parsed out of the message — covers
|
||||
libsignal-net's RetryLaterException that gets wrapped as
|
||||
AttachmentInvalidException during attachment upload, where the
|
||||
structured field stays null.
|
||||
|
||||
Returns None when neither yields a value.
|
||||
"""
|
||||
msg = ""
|
||||
if isinstance(err, dict):
|
||||
data = err.get("data") or {}
|
||||
response = data.get("response") or {}
|
||||
results = response.get("results") or []
|
||||
candidates = [
|
||||
r.get("retryAfterSeconds") for r in results
|
||||
if isinstance(r, dict) and r.get("retryAfterSeconds")
|
||||
]
|
||||
if candidates:
|
||||
return float(max(candidates))
|
||||
msg = str(err.get("message", ""))
|
||||
else:
|
||||
msg = str(err)
|
||||
match = _RETRY_AFTER_RE.search(msg)
|
||||
return float(match.group(1)) if match else None
|
||||
|
||||
|
||||
def _is_signal_rate_limit_error(err: Any) -> bool:
|
||||
"""True if a signal-cli RPC error reflects a rate-limit failure.
|
||||
|
||||
Matches three layers:
|
||||
- typed ``RATELIMIT_ERROR`` code (signal-cli ≥ v0.14.3, plain
|
||||
RateLimitException)
|
||||
- legacy ``[429] / RateLimitException`` substrings
|
||||
- libsignal-net's ``RetryLaterException`` / ``Retry after N seconds``
|
||||
surfaced inside ``AttachmentInvalidException`` when the rate
|
||||
limit is hit during attachment upload — signal-cli never re-tags
|
||||
these as RateLimitException, so substring is the only signal.
|
||||
"""
|
||||
if isinstance(err, dict) and err.get("code") == SIGNAL_RPC_ERROR_RATELIMIT:
|
||||
return True
|
||||
|
||||
message = (
|
||||
str(err.get("message", ""))
|
||||
if isinstance(err, dict)
|
||||
else str(err)
|
||||
)
|
||||
msg_lower = message.lower()
|
||||
return (
|
||||
"[429]" in message
|
||||
or "ratelimit" in msg_lower
|
||||
or "retrylaterexception" in msg_lower
|
||||
or "retry after" in msg_lower
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Misc helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _format_wait(seconds: float) -> str:
|
||||
"""Human-friendly wait label for user-facing pacing notices."""
|
||||
s = max(0.0, seconds)
|
||||
if s < 90:
|
||||
return f"{int(round(s))}s"
|
||||
return f"{max(1, int(round(s / 60)))} min"
|
||||
|
||||
|
||||
def _signal_send_timeout(num_attachments: int) -> float:
|
||||
"""HTTP timeout for a Signal ``send`` RPC.
|
||||
|
||||
signal-cli uploads attachments serially during the call, so the
|
||||
server-side time scales with batch size. Default 30s is fine for
|
||||
text-only sends but truncates large attachment batches mid-upload —
|
||||
we then log a phantom failure even though signal-cli completes the
|
||||
send a few seconds later. Scale at 5s/attachment with a 60s floor.
|
||||
"""
|
||||
if num_attachments <= 0:
|
||||
return 30.0
|
||||
return max(60.0, 5.0 * num_attachments)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Scheduler
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class SignalAttachmentScheduler:
|
||||
"""Process-wide token-bucket simulator for Signal attachment sends.
|
||||
|
||||
The bucket holds up to ``capacity`` tokens (default 50, matching
|
||||
Signal's server-side rate-limit bucket size). Each attachment consumes one
|
||||
token. Tokens refill at ``refill_rate`` tokens/second, calibrated
|
||||
from the per-token Retry-After hint we get from the server when a
|
||||
429 fires. Until we've observed one, we use the documented default
|
||||
(1 token / 4 seconds).
|
||||
|
||||
Concurrent ``acquire(n)`` calls serialize through an
|
||||
``asyncio.Lock`` — natural FIFO across agent sessions hitting the
|
||||
same daemon.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
capacity: float = float(SIGNAL_RATE_LIMIT_BUCKET_CAPACITY),
|
||||
default_retry_after: float = float(SIGNAL_RATE_LIMIT_DEFAULT_RETRY_AFTER),
|
||||
) -> None:
|
||||
self.capacity = float(capacity)
|
||||
self.tokens = float(capacity)
|
||||
self.refill_rate = 1.0 / float(default_retry_after)
|
||||
self.last_refill = time.monotonic()
|
||||
self._lock = asyncio.Lock()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Internals
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _refill(self) -> None:
|
||||
now = time.monotonic()
|
||||
elapsed = now - self.last_refill
|
||||
if elapsed > 0 and self.tokens < self.capacity:
|
||||
self.tokens = min(self.capacity, self.tokens + elapsed * self.refill_rate)
|
||||
self.last_refill = now
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Public API
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def estimate_wait(self, n: int) -> float:
|
||||
"""Best-effort estimate of the seconds until ``n`` tokens would
|
||||
be available. Used to decide whether to emit a user-facing
|
||||
pacing notice *before* committing to an ``acquire`` that may
|
||||
block silently. Lock-free; small races vs. concurrent acquires
|
||||
are benign for an informational notice.
|
||||
"""
|
||||
now = time.monotonic()
|
||||
elapsed = now - self.last_refill
|
||||
projected = self.tokens
|
||||
if elapsed > 0 and projected < self.capacity:
|
||||
projected = min(self.capacity, projected + elapsed * self.refill_rate)
|
||||
deficit = n - projected
|
||||
if deficit <= 0:
|
||||
return 0.0
|
||||
return deficit / self.refill_rate
|
||||
|
||||
async def acquire(self, n: int) -> float:
|
||||
"""Block until at least ``n`` tokens are available, return the
|
||||
seconds slept.
|
||||
|
||||
Does **not** deduct tokens — the bucket is a read-only model of
|
||||
server-side capacity. Call ``report_rpc_duration()`` after the
|
||||
RPC to synchronise the model with the server timeline.
|
||||
|
||||
Not perfect in case lots of coroutines try to acquire for big
|
||||
uploads (``report_rpc_duration`` will take a long time to get hit)
|
||||
but this is just a simulation. Signal server is ground truth and
|
||||
will raise rate-limit exceptions triggering requeues.
|
||||
|
||||
The lock is released during ``asyncio.sleep`` so other callers
|
||||
can interleave. A retry loop re-checks after each sleep in
|
||||
case the deadline was pessimistic.
|
||||
"""
|
||||
if n <= 0:
|
||||
return 0.0
|
||||
if n > self.capacity:
|
||||
raise SignalSchedulerError(
|
||||
f"Signal scheduler was called requesting {n} tokens "
|
||||
f"(max is {self.capacity})",
|
||||
)
|
||||
|
||||
total_slept = 0.0
|
||||
first_pass = True
|
||||
while True:
|
||||
async with self._lock:
|
||||
self._refill()
|
||||
if self.tokens >= n:
|
||||
if not first_pass or total_slept > 0:
|
||||
logger.debug(
|
||||
"Signal scheduler: tokens sufficient for %d "
|
||||
"(remaining=%.1f, total_slept=%.1fs)",
|
||||
n, self.tokens, total_slept,
|
||||
)
|
||||
return total_slept
|
||||
deficit = n - self.tokens
|
||||
wait = deficit / self.refill_rate
|
||||
if first_pass:
|
||||
logger.info(
|
||||
"Signal scheduler: pausing %.1fs for %d tokens "
|
||||
"(available=%.1f, deficit=%.1f, refill=%.4f/s ≈ %.1fs/token)",
|
||||
wait, n, self.tokens, deficit,
|
||||
self.refill_rate, 1.0 / self.refill_rate,
|
||||
)
|
||||
first_pass = False
|
||||
await asyncio.sleep(wait)
|
||||
total_slept += wait
|
||||
|
||||
async def report_rpc_duration(self, rpc_duration: float, n_attachments: int) -> None:
|
||||
"""Record an attachment-send RPC that just completed.
|
||||
|
||||
Deducts ``n_attachments`` tokens without crediting refill during
|
||||
the upload window. Signal's server checks the bucket at RPC start
|
||||
and does *not* refill during request processing — refill resumes
|
||||
after the response. Crediting upload-time refill causes cumulative
|
||||
drift that eventually triggers 429s.
|
||||
|
||||
Advances ``last_refill`` so the next ``acquire`` / ``_refill``
|
||||
starts counting from this point.
|
||||
"""
|
||||
if n_attachments <= 0:
|
||||
return
|
||||
|
||||
async with self._lock:
|
||||
now = time.monotonic()
|
||||
token_before = self.tokens
|
||||
self.tokens = max(0.0, token_before - float(n_attachments))
|
||||
self.last_refill = now
|
||||
logger.log(
|
||||
logging.INFO if rpc_duration > 10 and n_attachments > 5 else logging.DEBUG,
|
||||
"Signal scheduler: RPC for %d att took %.1fs — "
|
||||
"tokens %.1f → %.1f (deducted=%d, no upload refill credited, refill=%.4fs⁻¹)",
|
||||
n_attachments, rpc_duration,
|
||||
token_before, self.tokens,
|
||||
n_attachments, self.refill_rate,
|
||||
)
|
||||
|
||||
def feedback(self, retry_after: Optional[float], n_attempted: int) -> None:
|
||||
"""Apply server feedback after a 429.
|
||||
|
||||
``retry_after`` is the per-*token* refill window the server
|
||||
reports (None when signal-cli is older than v0.14.3 and didn't
|
||||
surface it).
|
||||
|
||||
When present we calibrate ``refill_rate`` from it:
|
||||
the server is authoritative.
|
||||
"""
|
||||
if retry_after and retry_after > 0:
|
||||
new_rate = 1.0 / float(retry_after)
|
||||
if new_rate != self.refill_rate:
|
||||
logger.info(
|
||||
"Signal scheduler: calibrating refill_rate to %.4f tokens/sec "
|
||||
"(server retry_after=%.1fs per token)",
|
||||
new_rate, retry_after,
|
||||
)
|
||||
self.refill_rate = new_rate
|
||||
self.tokens = 0.0
|
||||
self.last_refill = time.monotonic()
|
||||
|
||||
def state(self) -> dict:
|
||||
"""Return current scheduler state for diagnostic logging (read-only).
|
||||
|
||||
Does not advance ``last_refill`` — safe to call from logging paths
|
||||
without perturbing the bucket.
|
||||
"""
|
||||
now = time.monotonic()
|
||||
elapsed = now - self.last_refill
|
||||
projected = self.tokens
|
||||
if elapsed > 0 and projected < self.capacity:
|
||||
projected = min(self.capacity, projected + elapsed * self.refill_rate)
|
||||
return {
|
||||
"tokens": round(projected, 1),
|
||||
"capacity": int(self.capacity),
|
||||
"refill_rate": round(self.refill_rate, 4),
|
||||
"refill_seconds_per_token": round(1.0 / self.refill_rate, 1) if self.refill_rate > 0 else float("inf"),
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Process-wide singleton
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_scheduler: Optional[SignalAttachmentScheduler] = None
|
||||
|
||||
|
||||
def get_scheduler() -> SignalAttachmentScheduler:
|
||||
"""Return the process-wide scheduler, creating it on first access."""
|
||||
global _scheduler
|
||||
if _scheduler is None:
|
||||
_scheduler = SignalAttachmentScheduler()
|
||||
logger.info(
|
||||
"Signal scheduler: created (capacity=%d tokens, refill=%.4f/s ≈ %.1fs/token)",
|
||||
int(_scheduler.capacity),
|
||||
_scheduler.refill_rate,
|
||||
1.0 / _scheduler.refill_rate,
|
||||
)
|
||||
return _scheduler
|
||||
|
||||
|
||||
def _reset_scheduler() -> None:
|
||||
"""Drop the cached scheduler so the next ``get_scheduler`` call
|
||||
builds a fresh one. Test-only — never call from production paths."""
|
||||
global _scheduler
|
||||
_scheduler = None
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,379 @@
|
||||
"""SMS (Twilio) platform adapter.
|
||||
|
||||
Connects to the Twilio REST API for outbound SMS and runs an aiohttp
|
||||
webhook server to receive inbound messages.
|
||||
|
||||
Shares credentials with the optional telephony skill — same env vars:
|
||||
- TWILIO_ACCOUNT_SID
|
||||
- TWILIO_AUTH_TOKEN
|
||||
- TWILIO_PHONE_NUMBER (E.164 from-number, e.g. +15551234567)
|
||||
|
||||
Gateway-specific env vars:
|
||||
- SMS_WEBHOOK_PORT (default 8080)
|
||||
- SMS_WEBHOOK_HOST (default 127.0.0.1)
|
||||
- SMS_WEBHOOK_URL (public URL for Twilio signature validation — required)
|
||||
- SMS_INSECURE_NO_SIGNATURE (true to disable signature validation — dev only)
|
||||
- SMS_ALLOWED_USERS (comma-separated E.164 phone numbers)
|
||||
- SMS_ALLOW_ALL_USERS (true/false)
|
||||
- SMS_HOME_CHANNEL (phone number for cron delivery)
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import hashlib
|
||||
import hmac
|
||||
import logging
|
||||
import os
|
||||
import urllib.parse
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
from gateway.platforms.base import (
|
||||
BasePlatformAdapter,
|
||||
MessageEvent,
|
||||
MessageType,
|
||||
SendResult,
|
||||
)
|
||||
from gateway.platforms.helpers import redact_phone, strip_markdown
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
TWILIO_API_BASE = "https://api.twilio.com/2010-04-01/Accounts"
|
||||
MAX_SMS_LENGTH = 1600 # ~10 SMS segments
|
||||
DEFAULT_WEBHOOK_PORT = 8080
|
||||
DEFAULT_WEBHOOK_HOST = "127.0.0.1"
|
||||
|
||||
|
||||
def check_sms_requirements() -> bool:
|
||||
"""Check if SMS adapter dependencies are available."""
|
||||
try:
|
||||
import aiohttp # noqa: F401
|
||||
except ImportError:
|
||||
return False
|
||||
return bool(os.getenv("TWILIO_ACCOUNT_SID") and os.getenv("TWILIO_AUTH_TOKEN"))
|
||||
|
||||
|
||||
class SmsAdapter(BasePlatformAdapter):
|
||||
"""
|
||||
Twilio SMS <-> Hermes gateway adapter.
|
||||
|
||||
Each inbound phone number gets its own Hermes session (multi-tenant).
|
||||
Replies are always sent from the configured TWILIO_PHONE_NUMBER.
|
||||
"""
|
||||
|
||||
MAX_MESSAGE_LENGTH = MAX_SMS_LENGTH
|
||||
|
||||
def __init__(self, config: PlatformConfig):
|
||||
super().__init__(config, Platform.SMS)
|
||||
self._account_sid: str = os.environ["TWILIO_ACCOUNT_SID"]
|
||||
self._auth_token: str = os.environ["TWILIO_AUTH_TOKEN"]
|
||||
self._from_number: str = os.getenv("TWILIO_PHONE_NUMBER", "")
|
||||
self._webhook_port: int = int(
|
||||
os.getenv("SMS_WEBHOOK_PORT", str(DEFAULT_WEBHOOK_PORT))
|
||||
)
|
||||
self._webhook_host: str = os.getenv("SMS_WEBHOOK_HOST", DEFAULT_WEBHOOK_HOST)
|
||||
self._webhook_url: str = os.getenv("SMS_WEBHOOK_URL", "").strip()
|
||||
self._runner = None
|
||||
self._http_session: Optional["aiohttp.ClientSession"] = None
|
||||
|
||||
def _basic_auth_header(self) -> str:
|
||||
"""Build HTTP Basic auth header value for Twilio."""
|
||||
creds = f"{self._account_sid}:{self._auth_token}"
|
||||
encoded = base64.b64encode(creds.encode("ascii")).decode("ascii")
|
||||
return f"Basic {encoded}"
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Required abstract methods
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def connect(self) -> bool:
|
||||
import aiohttp
|
||||
from aiohttp import web
|
||||
|
||||
if not self._from_number:
|
||||
msg = "[sms] TWILIO_PHONE_NUMBER not set — cannot send replies"
|
||||
logger.error(msg)
|
||||
self._set_fatal_error("sms_missing_phone_number", msg, retryable=False)
|
||||
return False
|
||||
|
||||
insecure_no_sig = os.getenv("SMS_INSECURE_NO_SIGNATURE", "").lower() == "true"
|
||||
|
||||
if not self._webhook_url and not insecure_no_sig:
|
||||
msg = (
|
||||
"[sms] Refusing to start: SMS_WEBHOOK_URL is required for Twilio "
|
||||
"signature validation. Set it to the public URL configured in your "
|
||||
"Twilio console (e.g. https://example.com/webhooks/twilio). "
|
||||
"For local development without validation, set "
|
||||
"SMS_INSECURE_NO_SIGNATURE=true (NOT recommended for production)."
|
||||
)
|
||||
logger.error(msg)
|
||||
self._set_fatal_error("sms_missing_webhook_url", msg, retryable=False)
|
||||
return False
|
||||
|
||||
if insecure_no_sig and not self._webhook_url:
|
||||
logger.warning(
|
||||
"[sms] SMS_INSECURE_NO_SIGNATURE=true — Twilio signature validation "
|
||||
"is DISABLED. Any client that can reach port %d can inject messages. "
|
||||
"Do NOT use this in production.",
|
||||
self._webhook_port,
|
||||
)
|
||||
|
||||
app = web.Application()
|
||||
app.router.add_post("/webhooks/twilio", self._handle_webhook)
|
||||
app.router.add_get("/health", lambda _: web.Response(text="ok"))
|
||||
|
||||
self._runner = web.AppRunner(app)
|
||||
await self._runner.setup()
|
||||
site = web.TCPSite(self._runner, self._webhook_host, self._webhook_port)
|
||||
await site.start()
|
||||
self._http_session = aiohttp.ClientSession(
|
||||
timeout=aiohttp.ClientTimeout(total=30),
|
||||
trust_env=True,
|
||||
)
|
||||
self._running = True
|
||||
|
||||
logger.info(
|
||||
"[sms] Twilio webhook server listening on %s:%d, from: %s",
|
||||
self._webhook_host,
|
||||
self._webhook_port,
|
||||
redact_phone(self._from_number),
|
||||
)
|
||||
return True
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
if self._http_session:
|
||||
await self._http_session.close()
|
||||
self._http_session = None
|
||||
if self._runner:
|
||||
await self._runner.cleanup()
|
||||
self._runner = None
|
||||
self._running = False
|
||||
logger.info("[sms] Disconnected")
|
||||
|
||||
async def send(
|
||||
self,
|
||||
chat_id: str,
|
||||
content: str,
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
import aiohttp
|
||||
|
||||
formatted = self.format_message(content)
|
||||
chunks = self.truncate_message(formatted)
|
||||
last_result = SendResult(success=True)
|
||||
|
||||
url = f"{TWILIO_API_BASE}/{self._account_sid}/Messages.json"
|
||||
headers = {
|
||||
"Authorization": self._basic_auth_header(),
|
||||
}
|
||||
|
||||
session = self._http_session or aiohttp.ClientSession(
|
||||
timeout=aiohttp.ClientTimeout(total=30),
|
||||
trust_env=True,
|
||||
)
|
||||
try:
|
||||
for chunk in chunks:
|
||||
form_data = aiohttp.FormData()
|
||||
form_data.add_field("From", self._from_number)
|
||||
form_data.add_field("To", chat_id)
|
||||
form_data.add_field("Body", chunk)
|
||||
|
||||
try:
|
||||
async with session.post(url, data=form_data, headers=headers) as resp:
|
||||
body = await resp.json()
|
||||
if resp.status >= 400:
|
||||
error_msg = body.get("message", str(body))
|
||||
logger.error(
|
||||
"[sms] send failed to %s: %s %s",
|
||||
redact_phone(chat_id),
|
||||
resp.status,
|
||||
error_msg,
|
||||
)
|
||||
return SendResult(
|
||||
success=False,
|
||||
error=f"Twilio {resp.status}: {error_msg}",
|
||||
)
|
||||
msg_sid = body.get("sid", "")
|
||||
last_result = SendResult(success=True, message_id=msg_sid)
|
||||
except Exception as e:
|
||||
logger.error("[sms] send error to %s: %s", redact_phone(chat_id), e)
|
||||
return SendResult(success=False, error=str(e))
|
||||
finally:
|
||||
# Close session only if we created a fallback (no persistent session)
|
||||
if not self._http_session and session:
|
||||
await session.close()
|
||||
|
||||
return last_result
|
||||
|
||||
async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
|
||||
return {"name": chat_id, "type": "dm"}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# SMS-specific formatting
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def format_message(self, content: str) -> str:
|
||||
"""Strip markdown — SMS renders it as literal characters."""
|
||||
return strip_markdown(content)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Twilio signature validation
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _validate_twilio_signature(
|
||||
self, url: str, post_params: dict, signature: str,
|
||||
) -> bool:
|
||||
"""Validate ``X-Twilio-Signature`` header (HMAC-SHA1, base64).
|
||||
|
||||
Tries both with and without the default port for the URL scheme,
|
||||
since Twilio may sign with either variant.
|
||||
|
||||
Algorithm: https://www.twilio.com/docs/usage/security#validating-requests
|
||||
"""
|
||||
if self._check_signature(url, post_params, signature):
|
||||
return True
|
||||
|
||||
variant = self._port_variant_url(url)
|
||||
if variant and self._check_signature(variant, post_params, signature):
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
def _check_signature(
|
||||
self, url: str, post_params: dict, signature: str,
|
||||
) -> bool:
|
||||
"""Compute and compare a single Twilio signature."""
|
||||
data_to_sign = url
|
||||
for key in sorted(post_params.keys()):
|
||||
data_to_sign += key + post_params[key]
|
||||
mac = hmac.new(
|
||||
self._auth_token.encode("utf-8"),
|
||||
data_to_sign.encode("utf-8"),
|
||||
hashlib.sha1,
|
||||
)
|
||||
computed = base64.b64encode(mac.digest()).decode("utf-8")
|
||||
return hmac.compare_digest(computed, signature)
|
||||
|
||||
@staticmethod
|
||||
def _port_variant_url(url: str) -> str | None:
|
||||
"""Return the URL with the default port toggled, or None.
|
||||
|
||||
Only toggles default ports (443 for https, 80 for http).
|
||||
Non-standard ports are never modified.
|
||||
"""
|
||||
parsed = urllib.parse.urlparse(url)
|
||||
default_ports = {"https": 443, "http": 80}
|
||||
default_port = default_ports.get(parsed.scheme)
|
||||
if default_port is None:
|
||||
return None
|
||||
|
||||
if parsed.port == default_port:
|
||||
# Has explicit default port → strip it
|
||||
return urllib.parse.urlunparse(
|
||||
(parsed.scheme, parsed.hostname, parsed.path,
|
||||
parsed.params, parsed.query, parsed.fragment)
|
||||
)
|
||||
elif parsed.port is None:
|
||||
# No port → add default
|
||||
netloc = f"{parsed.hostname}:{default_port}"
|
||||
return urllib.parse.urlunparse(
|
||||
(parsed.scheme, netloc, parsed.path,
|
||||
parsed.params, parsed.query, parsed.fragment)
|
||||
)
|
||||
|
||||
# Non-standard port — no variant
|
||||
return None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Twilio webhook handler
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _handle_webhook(self, request) -> "aiohttp.web.Response":
|
||||
from aiohttp import web
|
||||
|
||||
try:
|
||||
raw = await request.read()
|
||||
# Twilio sends form-encoded data, not JSON
|
||||
form = urllib.parse.parse_qs(raw.decode("utf-8"), keep_blank_values=True)
|
||||
except Exception as e:
|
||||
logger.error("[sms] webhook parse error: %s", e)
|
||||
return web.Response(
|
||||
text='<?xml version="1.0" encoding="UTF-8"?><Response></Response>',
|
||||
content_type="application/xml",
|
||||
status=400,
|
||||
)
|
||||
|
||||
# Validate Twilio request signature when SMS_WEBHOOK_URL is configured
|
||||
if self._webhook_url:
|
||||
twilio_sig = request.headers.get("X-Twilio-Signature", "")
|
||||
if not twilio_sig:
|
||||
logger.warning("[sms] Rejected: missing X-Twilio-Signature header")
|
||||
return web.Response(
|
||||
text='<?xml version="1.0" encoding="UTF-8"?><Response></Response>',
|
||||
content_type="application/xml",
|
||||
status=403,
|
||||
)
|
||||
flat_params = {k: v[0] for k, v in form.items() if v}
|
||||
if not self._validate_twilio_signature(
|
||||
self._webhook_url, flat_params, twilio_sig
|
||||
):
|
||||
logger.warning("[sms] Rejected: invalid Twilio signature")
|
||||
return web.Response(
|
||||
text='<?xml version="1.0" encoding="UTF-8"?><Response></Response>',
|
||||
content_type="application/xml",
|
||||
status=403,
|
||||
)
|
||||
|
||||
# Extract fields (parse_qs returns lists)
|
||||
from_number = (form.get("From", [""]))[0].strip()
|
||||
to_number = (form.get("To", [""]))[0].strip()
|
||||
text = (form.get("Body", [""]))[0].strip()
|
||||
message_sid = (form.get("MessageSid", [""]))[0].strip()
|
||||
|
||||
if not from_number or not text:
|
||||
return web.Response(
|
||||
text='<?xml version="1.0" encoding="UTF-8"?><Response></Response>',
|
||||
content_type="application/xml",
|
||||
)
|
||||
|
||||
# Ignore messages from our own number (echo prevention)
|
||||
if from_number == self._from_number:
|
||||
logger.debug("[sms] ignoring echo from own number %s", redact_phone(from_number))
|
||||
return web.Response(
|
||||
text='<?xml version="1.0" encoding="UTF-8"?><Response></Response>',
|
||||
content_type="application/xml",
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"[sms] inbound from %s -> %s: %s",
|
||||
redact_phone(from_number),
|
||||
redact_phone(to_number),
|
||||
text[:80],
|
||||
)
|
||||
|
||||
source = self.build_source(
|
||||
chat_id=from_number,
|
||||
chat_name=from_number,
|
||||
chat_type="dm",
|
||||
user_id=from_number,
|
||||
user_name=from_number,
|
||||
)
|
||||
event = MessageEvent(
|
||||
text=text,
|
||||
message_type=MessageType.TEXT,
|
||||
source=source,
|
||||
raw_message=form,
|
||||
message_id=message_sid,
|
||||
)
|
||||
|
||||
# Non-blocking: Twilio expects a fast response
|
||||
task = asyncio.create_task(self.handle_message(event))
|
||||
self._background_tasks.add(task)
|
||||
task.add_done_callback(self._background_tasks.discard)
|
||||
|
||||
# Return empty TwiML — we send replies via the REST API, not inline TwiML
|
||||
return web.Response(
|
||||
text='<?xml version="1.0" encoding="UTF-8"?><Response></Response>',
|
||||
content_type="application/xml",
|
||||
)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,259 @@
|
||||
"""Telegram-specific network helpers.
|
||||
|
||||
Provides a hostname-preserving fallback transport for networks where
|
||||
api.telegram.org resolves to an endpoint that is unreachable from the current
|
||||
host. The transport keeps the logical request host and TLS SNI as
|
||||
api.telegram.org while retrying the TCP connection against one or more fallback
|
||||
IPv4 addresses.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import ipaddress
|
||||
import logging
|
||||
import socket
|
||||
from typing import Iterable, Optional
|
||||
|
||||
import httpx
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_TELEGRAM_API_HOST = "api.telegram.org"
|
||||
|
||||
# DNS-over-HTTPS providers used to discover Telegram API IPs that may differ
|
||||
# from the (potentially unreachable) IP returned by the local system resolver.
|
||||
_DOH_TIMEOUT = 4.0 # seconds — bounded so connect() isn't noticeably delayed
|
||||
|
||||
_DOH_PROVIDERS: list[dict] = [
|
||||
{
|
||||
"url": "https://dns.google/resolve",
|
||||
"params": {"name": _TELEGRAM_API_HOST, "type": "A"},
|
||||
"headers": {},
|
||||
},
|
||||
{
|
||||
"url": "https://cloudflare-dns.com/dns-query",
|
||||
"params": {"name": _TELEGRAM_API_HOST, "type": "A"},
|
||||
"headers": {"Accept": "application/dns-json"},
|
||||
},
|
||||
]
|
||||
|
||||
# Last-resort IPs when DoH is also blocked. These are stable Telegram Bot API
|
||||
# endpoints in the 149.154.160.0/20 block (same seed used by OpenClaw).
|
||||
_SEED_FALLBACK_IPS: list[str] = ["149.154.167.220"]
|
||||
|
||||
|
||||
def _resolve_proxy_url(target_hosts=None) -> str | None:
|
||||
# Delegate to shared implementation (env vars + macOS system proxy detection)
|
||||
from gateway.platforms.base import resolve_proxy_url
|
||||
return resolve_proxy_url("TELEGRAM_PROXY", target_hosts=target_hosts)
|
||||
|
||||
|
||||
class TelegramFallbackTransport(httpx.AsyncBaseTransport):
|
||||
"""Retry Telegram Bot API requests via fallback IPs while preserving TLS/SNI.
|
||||
|
||||
Requests continue to target https://api.telegram.org/... logically, but on
|
||||
connect failures the underlying TCP connection is retried against a known
|
||||
reachable IP. This is effectively the programmatic equivalent of
|
||||
``curl --resolve api.telegram.org:443:<ip>``.
|
||||
"""
|
||||
|
||||
def __init__(self, fallback_ips: Iterable[str], **transport_kwargs):
|
||||
self._fallback_ips = list(dict.fromkeys(_normalize_fallback_ips(fallback_ips)))
|
||||
proxy_url = _resolve_proxy_url(target_hosts=[_TELEGRAM_API_HOST, *self._fallback_ips])
|
||||
if proxy_url and "proxy" not in transport_kwargs:
|
||||
transport_kwargs["proxy"] = proxy_url
|
||||
self._primary = httpx.AsyncHTTPTransport(**transport_kwargs)
|
||||
self._fallbacks = {
|
||||
ip: httpx.AsyncHTTPTransport(**transport_kwargs) for ip in self._fallback_ips
|
||||
}
|
||||
self._sticky_ip: Optional[str] = None
|
||||
self._sticky_lock = asyncio.Lock()
|
||||
|
||||
async def handle_async_request(self, request: httpx.Request) -> httpx.Response:
|
||||
if request.url.host != _TELEGRAM_API_HOST or not self._fallback_ips:
|
||||
return await self._primary.handle_async_request(request)
|
||||
|
||||
sticky_ip = self._sticky_ip
|
||||
attempt_order: list[Optional[str]] = [sticky_ip] if sticky_ip else [None]
|
||||
if sticky_ip:
|
||||
attempt_order.append(None) # retry primary DNS after sticky failure
|
||||
for ip in self._fallback_ips:
|
||||
if ip != sticky_ip:
|
||||
attempt_order.append(ip)
|
||||
|
||||
last_error: Exception | None = None
|
||||
for ip in attempt_order:
|
||||
candidate = request if ip is None else _rewrite_request_for_ip(request, ip)
|
||||
transport = self._primary if ip is None else self._fallbacks[ip]
|
||||
try:
|
||||
response = await transport.handle_async_request(candidate)
|
||||
if ip is not None and self._sticky_ip != ip:
|
||||
async with self._sticky_lock:
|
||||
if self._sticky_ip != ip:
|
||||
self._sticky_ip = ip
|
||||
logger.warning(
|
||||
"[Telegram] Primary api.telegram.org path unreachable; using sticky fallback IP %s",
|
||||
ip,
|
||||
)
|
||||
return response
|
||||
except Exception as exc:
|
||||
last_error = exc
|
||||
if not _is_retryable_connect_error(exc):
|
||||
raise
|
||||
if ip is not None and ip == self._sticky_ip:
|
||||
async with self._sticky_lock:
|
||||
if self._sticky_ip == ip:
|
||||
self._sticky_ip = None
|
||||
logger.warning(
|
||||
"[Telegram] Sticky fallback IP %s failed; resetting to primary DNS path",
|
||||
ip,
|
||||
)
|
||||
if ip is None:
|
||||
logger.warning(
|
||||
"[Telegram] Primary api.telegram.org connection failed (%s); trying fallback IPs %s",
|
||||
exc,
|
||||
", ".join(self._fallback_ips),
|
||||
)
|
||||
continue
|
||||
logger.warning("[Telegram] Fallback IP %s failed: %s", ip, exc)
|
||||
continue
|
||||
|
||||
if last_error is None:
|
||||
raise RuntimeError("All Telegram fallback IPs exhausted but no error was recorded")
|
||||
raise last_error
|
||||
|
||||
async def aclose(self) -> None:
|
||||
await self._primary.aclose()
|
||||
for transport in self._fallbacks.values():
|
||||
await transport.aclose()
|
||||
|
||||
|
||||
def _normalize_fallback_ips(values: Iterable[str]) -> list[str]:
|
||||
normalized: list[str] = []
|
||||
for value in values:
|
||||
raw = str(value).strip()
|
||||
if not raw:
|
||||
continue
|
||||
try:
|
||||
addr = ipaddress.ip_address(raw)
|
||||
except ValueError:
|
||||
logger.warning("Ignoring invalid Telegram fallback IP: %r", raw)
|
||||
continue
|
||||
if addr.version != 4:
|
||||
logger.warning("Ignoring non-IPv4 Telegram fallback IP: %s", raw)
|
||||
continue
|
||||
if addr.is_private or addr.is_loopback or addr.is_link_local or addr.is_unspecified:
|
||||
logger.warning("Ignoring private/internal Telegram fallback IP: %s", raw)
|
||||
continue
|
||||
normalized.append(str(addr))
|
||||
return normalized
|
||||
|
||||
|
||||
def parse_fallback_ip_env(value: str | None) -> list[str]:
|
||||
if not value:
|
||||
return []
|
||||
parts = [part.strip() for part in value.split(",")]
|
||||
return _normalize_fallback_ips(parts)
|
||||
|
||||
|
||||
def _resolve_system_dns() -> set[str]:
|
||||
"""Return the IPv4 addresses that the OS resolver gives for api.telegram.org."""
|
||||
try:
|
||||
results = socket.getaddrinfo(_TELEGRAM_API_HOST, 443, socket.AF_INET)
|
||||
return {addr[4][0] for addr in results}
|
||||
except Exception:
|
||||
return set()
|
||||
|
||||
|
||||
async def _query_doh_provider(
|
||||
client: httpx.AsyncClient, provider: dict
|
||||
) -> list[str]:
|
||||
"""Query one DoH provider and return A-record IPs."""
|
||||
try:
|
||||
resp = await client.get(
|
||||
provider["url"], params=provider["params"], headers=provider["headers"]
|
||||
)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
ips: list[str] = []
|
||||
for answer in data.get("Answer", []):
|
||||
if answer.get("type") != 1: # A record
|
||||
continue
|
||||
raw = answer.get("data", "").strip()
|
||||
try:
|
||||
ipaddress.ip_address(raw)
|
||||
ips.append(raw)
|
||||
except ValueError:
|
||||
continue
|
||||
return ips
|
||||
except Exception as exc:
|
||||
logger.debug("DoH query to %s failed: %s", provider["url"], exc)
|
||||
return []
|
||||
|
||||
|
||||
async def discover_fallback_ips() -> list[str]:
|
||||
"""Auto-discover Telegram API IPs via DNS-over-HTTPS.
|
||||
|
||||
Resolves api.telegram.org through Google and Cloudflare DoH and returns all
|
||||
unique A records. IPs that match the local system resolver are kept rather
|
||||
than excluded: in many networks the system-DNS IP is the most reliable path
|
||||
to api.telegram.org and a transient primary-path failure should be retried
|
||||
against the same address via the IP-rewrite path before the seed list is
|
||||
consulted (#14520). Falls back to a hardcoded seed list only when DoH
|
||||
yields no usable answers.
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=httpx.Timeout(_DOH_TIMEOUT)) as client:
|
||||
doh_tasks = [_query_doh_provider(client, p) for p in _DOH_PROVIDERS]
|
||||
system_dns_task = asyncio.to_thread(_resolve_system_dns)
|
||||
results = await asyncio.gather(system_dns_task, *doh_tasks, return_exceptions=True)
|
||||
|
||||
# results[0] = system DNS IPs (set), results[1:] = DoH IP lists
|
||||
system_ips: set[str] = results[0] if isinstance(results[0], set) else set()
|
||||
|
||||
doh_ips: list[str] = []
|
||||
for r in results[1:]:
|
||||
if isinstance(r, list):
|
||||
doh_ips.extend(r)
|
||||
|
||||
# Deduplicate preserving order
|
||||
seen: set[str] = set()
|
||||
candidates: list[str] = []
|
||||
for ip in doh_ips:
|
||||
if ip not in seen:
|
||||
seen.add(ip)
|
||||
candidates.append(ip)
|
||||
|
||||
# Validate through existing normalization
|
||||
validated = _normalize_fallback_ips(candidates)
|
||||
|
||||
if validated:
|
||||
logger.debug("Discovered Telegram fallback IPs via DoH: %s", ", ".join(validated))
|
||||
return validated
|
||||
|
||||
logger.info(
|
||||
"DoH discovery yielded no usable IPs (system DNS: %s); using seed fallback IPs %s",
|
||||
", ".join(system_ips) or "unknown",
|
||||
", ".join(_SEED_FALLBACK_IPS),
|
||||
)
|
||||
return list(_SEED_FALLBACK_IPS)
|
||||
|
||||
|
||||
def _rewrite_request_for_ip(request: httpx.Request, ip: str) -> httpx.Request:
|
||||
original_host = request.url.host or _TELEGRAM_API_HOST
|
||||
url = request.url.copy_with(host=ip)
|
||||
headers = request.headers.copy()
|
||||
headers["host"] = original_host
|
||||
extensions = dict(request.extensions)
|
||||
extensions["sni_hostname"] = original_host
|
||||
return httpx.Request(
|
||||
method=request.method,
|
||||
url=url,
|
||||
headers=headers,
|
||||
stream=request.stream,
|
||||
extensions=extensions,
|
||||
)
|
||||
|
||||
|
||||
def _is_retryable_connect_error(exc: Exception) -> bool:
|
||||
return isinstance(exc, (httpx.ConnectTimeout, httpx.ConnectError))
|
||||
@@ -0,0 +1,925 @@
|
||||
"""Generic webhook platform adapter.
|
||||
|
||||
Runs an aiohttp HTTP server that receives webhook POSTs from external
|
||||
services (GitHub, GitLab, JIRA, Stripe, etc.), validates HMAC signatures,
|
||||
transforms payloads into agent prompts, and routes responses back to the
|
||||
source or to another configured platform.
|
||||
|
||||
Configuration lives in config.yaml under platforms.webhook.extra.routes.
|
||||
Each route defines:
|
||||
- events: which event types to accept (header-based filtering)
|
||||
- secret: HMAC secret for signature validation (REQUIRED)
|
||||
- prompt: template string formatted with the webhook payload
|
||||
- skills: optional list of skills to load for the agent
|
||||
- deliver: where to send the response (github_comment, telegram, etc.)
|
||||
- deliver_extra: additional delivery config (repo, pr_number, chat_id)
|
||||
- deliver_only: if true, skip the agent — the rendered prompt IS the
|
||||
message that gets delivered. Use for external push notifications
|
||||
(Supabase, monitoring alerts, inter-agent pings) where zero LLM cost
|
||||
and sub-second delivery matter more than agent reasoning.
|
||||
|
||||
Security:
|
||||
- HMAC secret is required per route (validated at startup)
|
||||
- Rate limiting per route (fixed-window, configurable)
|
||||
- Idempotency cache prevents duplicate agent runs on webhook retries
|
||||
- Body size limits checked before reading payload
|
||||
- Set secret to "INSECURE_NO_AUTH" to skip validation (testing only)
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import binascii
|
||||
import hashlib
|
||||
import hmac
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import subprocess
|
||||
import time
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
try:
|
||||
from aiohttp import web
|
||||
|
||||
AIOHTTP_AVAILABLE = True
|
||||
except ImportError:
|
||||
AIOHTTP_AVAILABLE = False
|
||||
web = None # type: ignore[assignment]
|
||||
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
from gateway.platforms.base import (
|
||||
BasePlatformAdapter,
|
||||
MessageEvent,
|
||||
MessageType,
|
||||
SendResult,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_BUILTIN_DELIVER_PLATFORMS = {
|
||||
"telegram", "discord", "slack", "signal", "sms", "whatsapp",
|
||||
"matrix", "mattermost", "homeassistant", "email", "dingtalk",
|
||||
"feishu", "wecom", "wecom_callback", "weixin", "bluebubbles",
|
||||
"qqbot", "yuanbao",
|
||||
}
|
||||
|
||||
DEFAULT_HOST = "0.0.0.0"
|
||||
DEFAULT_PORT = 8644
|
||||
_INSECURE_NO_AUTH = "INSECURE_NO_AUTH"
|
||||
_DYNAMIC_ROUTES_FILENAME = "webhook_subscriptions.json"
|
||||
|
||||
# Hostnames/IP literals that only serve connections originating on the same
|
||||
# machine. Anything else is treated as a public bind for safety-rail purposes.
|
||||
_LOOPBACK_HOSTS = frozenset({
|
||||
"127.0.0.1",
|
||||
"localhost",
|
||||
"::1",
|
||||
"ip6-localhost",
|
||||
"ip6-loopback",
|
||||
})
|
||||
|
||||
|
||||
def _is_loopback_host(host: str) -> bool:
|
||||
"""True when `host` binds only to the local machine.
|
||||
|
||||
Covers IPv4 loopback, the standard `localhost` alias, IPv6 loopback in
|
||||
both bracketed and bare form, and the common Debian-style aliases. Any
|
||||
falsy value (empty string, None) is conservatively treated as non-loopback
|
||||
because an unset host usually means the platform-default public bind.
|
||||
"""
|
||||
if not host:
|
||||
return False
|
||||
return host.strip().lower() in _LOOPBACK_HOSTS
|
||||
|
||||
|
||||
def check_webhook_requirements() -> bool:
|
||||
"""Check if webhook adapter dependencies are available."""
|
||||
return AIOHTTP_AVAILABLE
|
||||
|
||||
|
||||
class WebhookAdapter(BasePlatformAdapter):
|
||||
"""Generic webhook receiver that triggers agent runs from HTTP POSTs."""
|
||||
|
||||
def __init__(self, config: PlatformConfig):
|
||||
super().__init__(config, Platform.WEBHOOK)
|
||||
self._host: str = config.extra.get("host", DEFAULT_HOST)
|
||||
self._port: int = int(config.extra.get("port", DEFAULT_PORT))
|
||||
self._global_secret: str = config.extra.get("secret", "")
|
||||
self._static_routes: Dict[str, dict] = config.extra.get("routes", {})
|
||||
self._dynamic_routes: Dict[str, dict] = {}
|
||||
self._dynamic_routes_mtime: float = 0.0
|
||||
self._routes: Dict[str, dict] = dict(self._static_routes)
|
||||
self._runner = None
|
||||
|
||||
# Delivery info keyed by session chat_id.
|
||||
#
|
||||
# Read by every send() invocation for the chat_id (status messages
|
||||
# AND the final response). Cleaned up via TTL on each POST so the
|
||||
# dict stays bounded — see _prune_delivery_info(). Do NOT pop on
|
||||
# send(), or interim status messages (e.g. fallback notifications,
|
||||
# context-pressure warnings) will consume the entry before the
|
||||
# final response arrives, causing the response to silently fall
|
||||
# back to the "log" deliver type.
|
||||
self._delivery_info: Dict[str, dict] = {}
|
||||
self._delivery_info_created: Dict[str, float] = {}
|
||||
|
||||
# Reference to gateway runner for cross-platform delivery (set externally)
|
||||
self.gateway_runner = None
|
||||
|
||||
# Idempotency: TTL cache of recently processed delivery IDs.
|
||||
# Prevents duplicate agent runs when webhook providers retry.
|
||||
self._seen_deliveries: Dict[str, float] = {}
|
||||
self._idempotency_ttl: int = 3600 # 1 hour
|
||||
|
||||
# Rate limiting: per-route timestamps in a fixed window.
|
||||
self._rate_counts: Dict[str, List[float]] = {}
|
||||
self._rate_limit: int = int(config.extra.get("rate_limit", 30)) # per minute
|
||||
|
||||
# Body size limit (auth-before-body pattern)
|
||||
self._max_body_bytes: int = int(
|
||||
config.extra.get("max_body_bytes", 1_048_576)
|
||||
) # 1MB
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Lifecycle
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def connect(self) -> bool:
|
||||
# Load agent-created subscriptions before validating
|
||||
self._reload_dynamic_routes()
|
||||
|
||||
# Validate routes at startup — secret is required per route
|
||||
for name, route in self._routes.items():
|
||||
secret = route.get("secret", self._global_secret)
|
||||
if not secret:
|
||||
raise ValueError(
|
||||
f"[webhook] Route '{name}' has no HMAC secret. "
|
||||
f"Set 'secret' on the route or globally. "
|
||||
f"For testing without auth, set secret to '{_INSECURE_NO_AUTH}'."
|
||||
)
|
||||
|
||||
# Safety rail: refuse to start if INSECURE_NO_AUTH is combined with a
|
||||
# non-loopback bind. The escape hatch is for local testing only;
|
||||
# serving an unauthenticated route on a public interface is a
|
||||
# deployment-grade footgun we'd rather crash early than ship.
|
||||
if secret == _INSECURE_NO_AUTH and not _is_loopback_host(self._host):
|
||||
raise ValueError(
|
||||
f"[webhook] Route '{name}' uses INSECURE_NO_AUTH secret "
|
||||
f"but is bound to non-loopback host '{self._host}'. "
|
||||
f"INSECURE_NO_AUTH is for local testing only. "
|
||||
f"Refusing to start to prevent accidental exposure."
|
||||
)
|
||||
# deliver_only routes bypass the agent — the POST body becomes a
|
||||
# direct push notification via the configured delivery target.
|
||||
# Validate up-front so misconfiguration surfaces at startup rather
|
||||
# than on the first webhook POST.
|
||||
if route.get("deliver_only"):
|
||||
deliver = route.get("deliver", "log")
|
||||
if not deliver or deliver == "log":
|
||||
raise ValueError(
|
||||
f"[webhook] Route '{name}' has deliver_only=true but "
|
||||
f"deliver is '{deliver}'. Direct delivery requires a "
|
||||
f"real target (telegram, discord, slack, github_comment, etc.)."
|
||||
)
|
||||
|
||||
app = web.Application()
|
||||
app.router.add_get("/health", self._handle_health)
|
||||
app.router.add_post("/webhooks/{route_name}", self._handle_webhook)
|
||||
|
||||
# Port conflict detection — fail fast if port is already in use
|
||||
import socket as _socket
|
||||
try:
|
||||
with _socket.socket(_socket.AF_INET, _socket.SOCK_STREAM) as _s:
|
||||
_s.settimeout(1)
|
||||
_s.connect(('127.0.0.1', self._port))
|
||||
logger.error('[webhook] Port %d already in use. Set a different port in config.yaml: platforms.webhook.port', self._port)
|
||||
return False
|
||||
except (ConnectionRefusedError, OSError):
|
||||
pass # port is free
|
||||
|
||||
self._runner = web.AppRunner(app)
|
||||
await self._runner.setup()
|
||||
site = web.TCPSite(self._runner, self._host, self._port)
|
||||
await site.start()
|
||||
self._mark_connected()
|
||||
|
||||
route_names = ", ".join(self._routes.keys()) or "(none configured)"
|
||||
logger.info(
|
||||
"[webhook] Listening on %s:%d — routes: %s",
|
||||
self._host,
|
||||
self._port,
|
||||
route_names,
|
||||
)
|
||||
return True
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
if self._runner:
|
||||
await self._runner.cleanup()
|
||||
self._runner = None
|
||||
self._mark_disconnected()
|
||||
logger.info("[webhook] Disconnected")
|
||||
|
||||
async def send(
|
||||
self,
|
||||
chat_id: str,
|
||||
content: str,
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
"""Deliver the agent's response to the configured destination.
|
||||
|
||||
chat_id is ``webhook:{route}:{delivery_id}``. The delivery info
|
||||
stored during webhook receipt is read with ``.get()`` (not popped)
|
||||
so that interim status messages emitted before the final response
|
||||
— fallback-model notifications, context-pressure warnings, etc. —
|
||||
do not consume the entry and silently downgrade the final response
|
||||
to the ``log`` deliver type. TTL cleanup happens on POST.
|
||||
"""
|
||||
delivery = self._delivery_info.get(chat_id, {})
|
||||
deliver_type = delivery.get("deliver", "log")
|
||||
|
||||
if deliver_type == "log":
|
||||
logger.info("[webhook] Response for %s: %s", chat_id, content[:200])
|
||||
return SendResult(success=True)
|
||||
|
||||
if deliver_type == "github_comment":
|
||||
return await self._deliver_github_comment(content, delivery)
|
||||
|
||||
# Cross-platform delivery — any platform with a gateway adapter.
|
||||
# Check both built-in names and plugin-registered platforms.
|
||||
_is_known_platform = deliver_type in _BUILTIN_DELIVER_PLATFORMS
|
||||
if not _is_known_platform:
|
||||
try:
|
||||
from gateway.platform_registry import platform_registry
|
||||
_is_known_platform = platform_registry.is_registered(deliver_type)
|
||||
except Exception:
|
||||
pass
|
||||
if self.gateway_runner and _is_known_platform:
|
||||
return await self._deliver_cross_platform(
|
||||
deliver_type, content, delivery
|
||||
)
|
||||
|
||||
logger.warning("[webhook] Unknown deliver type: %s", deliver_type)
|
||||
return SendResult(
|
||||
success=False, error=f"Unknown deliver type: {deliver_type}"
|
||||
)
|
||||
|
||||
def _prune_delivery_info(self, now: float) -> None:
|
||||
"""Drop delivery_info entries older than the idempotency TTL.
|
||||
|
||||
Mirrors the cleanup pattern used for ``_seen_deliveries``. Called
|
||||
on each POST so the dict size is bounded by ``rate_limit * TTL``
|
||||
even if many webhooks fire and never receive a final response.
|
||||
"""
|
||||
cutoff = now - self._idempotency_ttl
|
||||
stale = [
|
||||
k
|
||||
for k, t in self._delivery_info_created.items()
|
||||
if t < cutoff
|
||||
]
|
||||
for k in stale:
|
||||
self._delivery_info.pop(k, None)
|
||||
self._delivery_info_created.pop(k, None)
|
||||
|
||||
async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
|
||||
return {"name": chat_id, "type": "webhook"}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# HTTP handlers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _handle_health(self, request: "web.Request") -> "web.Response":
|
||||
"""GET /health — simple health check."""
|
||||
return web.json_response({"status": "ok", "platform": "webhook"})
|
||||
|
||||
def _reload_dynamic_routes(self) -> None:
|
||||
"""Reload agent-created subscriptions from disk if the file changed."""
|
||||
from hermes_constants import get_hermes_home
|
||||
hermes_home = get_hermes_home()
|
||||
subs_path = hermes_home / _DYNAMIC_ROUTES_FILENAME
|
||||
if not subs_path.exists():
|
||||
if self._dynamic_routes:
|
||||
self._dynamic_routes = {}
|
||||
self._routes = dict(self._static_routes)
|
||||
logger.debug("[webhook] Dynamic subscriptions file removed, cleared dynamic routes")
|
||||
return
|
||||
try:
|
||||
mtime = subs_path.stat().st_mtime
|
||||
if mtime <= self._dynamic_routes_mtime:
|
||||
return # No change
|
||||
data = json.loads(subs_path.read_text(encoding="utf-8"))
|
||||
if not isinstance(data, dict):
|
||||
return
|
||||
# Merge: static routes take precedence over dynamic ones.
|
||||
# Reject any dynamic route whose effective secret is empty —
|
||||
# an empty secret would cause _handle_webhook to skip HMAC
|
||||
# validation entirely, letting unauthenticated callers in.
|
||||
new_dynamic: Dict[str, dict] = {}
|
||||
for k, v in data.items():
|
||||
if k in self._static_routes:
|
||||
continue
|
||||
effective_secret = v.get("secret", self._global_secret)
|
||||
if not effective_secret:
|
||||
logger.warning(
|
||||
"[webhook] Dynamic route '%s' skipped: 'secret' is "
|
||||
"missing or empty. Set a valid HMAC secret, or use "
|
||||
"'%s' to explicitly disable auth (testing only).",
|
||||
k,
|
||||
_INSECURE_NO_AUTH,
|
||||
)
|
||||
continue
|
||||
if (
|
||||
effective_secret == _INSECURE_NO_AUTH
|
||||
and not _is_loopback_host(self._host)
|
||||
):
|
||||
logger.warning(
|
||||
"[webhook] Dynamic route '%s' skipped: INSECURE_NO_AUTH "
|
||||
"is only allowed on loopback hosts. Current host: '%s'.",
|
||||
k,
|
||||
self._host,
|
||||
)
|
||||
continue
|
||||
new_dynamic[k] = v
|
||||
self._dynamic_routes = new_dynamic
|
||||
self._routes = {**self._dynamic_routes, **self._static_routes}
|
||||
self._dynamic_routes_mtime = mtime
|
||||
logger.info(
|
||||
"[webhook] Reloaded %d dynamic route(s): %s",
|
||||
len(self._dynamic_routes),
|
||||
", ".join(self._dynamic_routes.keys()) or "(none)",
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error("[webhook] Failed to reload dynamic routes: %s", e)
|
||||
|
||||
async def _handle_webhook(self, request: "web.Request") -> "web.Response":
|
||||
"""POST /webhooks/{route_name} — receive and process a webhook event."""
|
||||
# Hot-reload dynamic subscriptions on each request (mtime-gated, cheap)
|
||||
self._reload_dynamic_routes()
|
||||
|
||||
route_name = request.match_info.get("route_name", "")
|
||||
route_config = self._routes.get(route_name)
|
||||
|
||||
if not route_config:
|
||||
return web.json_response(
|
||||
{"error": f"Unknown route: {route_name}"}, status=404
|
||||
)
|
||||
|
||||
# ── Auth-before-body ─────────────────────────────────────
|
||||
# Check Content-Length before reading the full payload.
|
||||
content_length = request.content_length or 0
|
||||
if content_length > self._max_body_bytes:
|
||||
return web.json_response(
|
||||
{"error": "Payload too large"}, status=413
|
||||
)
|
||||
|
||||
# Read body (must be done before any validation)
|
||||
try:
|
||||
raw_body = await request.read()
|
||||
except Exception as e:
|
||||
logger.error("[webhook] Failed to read body: %s", e)
|
||||
return web.json_response({"error": "Bad request"}, status=400)
|
||||
|
||||
# Validate HMAC signature FIRST (skip only for the explicit local-test
|
||||
# INSECURE_NO_AUTH mode). Missing/empty secrets must fail closed here,
|
||||
# not only during connect(), so direct handler reuse cannot turn a
|
||||
# network webhook route into an unauthenticated agent-dispatch surface.
|
||||
secret = route_config.get("secret", self._global_secret)
|
||||
if not secret:
|
||||
logger.error(
|
||||
"[webhook] Route %s has no HMAC secret; refusing request",
|
||||
route_name,
|
||||
)
|
||||
return web.json_response(
|
||||
{"error": "Webhook route is missing an HMAC secret"},
|
||||
status=403,
|
||||
)
|
||||
if secret != _INSECURE_NO_AUTH:
|
||||
if not self._validate_signature(request, raw_body, secret):
|
||||
logger.warning(
|
||||
"[webhook] Invalid signature for route %s", route_name
|
||||
)
|
||||
return web.json_response(
|
||||
{"error": "Invalid signature"}, status=401
|
||||
)
|
||||
|
||||
# ── Rate limiting (after auth) ───────────────────────────
|
||||
now = time.time()
|
||||
window = self._rate_counts.setdefault(route_name, [])
|
||||
window[:] = [t for t in window if now - t < 60]
|
||||
if len(window) >= self._rate_limit:
|
||||
return web.json_response(
|
||||
{"error": "Rate limit exceeded"}, status=429
|
||||
)
|
||||
window.append(now)
|
||||
|
||||
# Parse payload
|
||||
try:
|
||||
payload = json.loads(raw_body)
|
||||
except json.JSONDecodeError:
|
||||
# Try form-encoded as fallback
|
||||
try:
|
||||
import urllib.parse
|
||||
|
||||
payload = dict(
|
||||
urllib.parse.parse_qsl(raw_body.decode("utf-8"))
|
||||
)
|
||||
except Exception:
|
||||
return web.json_response(
|
||||
{"error": "Cannot parse body"}, status=400
|
||||
)
|
||||
|
||||
# Check event type filter
|
||||
event_type = (
|
||||
request.headers.get("X-GitHub-Event", "")
|
||||
or request.headers.get("X-GitLab-Event", "")
|
||||
or payload.get("event_type", "")
|
||||
or payload.get("type", "")
|
||||
or "unknown"
|
||||
)
|
||||
allowed_events = route_config.get("events", [])
|
||||
if allowed_events and event_type not in allowed_events:
|
||||
logger.debug(
|
||||
"[webhook] Ignoring event %s for route %s (allowed: %s)",
|
||||
event_type,
|
||||
route_name,
|
||||
allowed_events,
|
||||
)
|
||||
return web.json_response(
|
||||
{"status": "ignored", "event": event_type}
|
||||
)
|
||||
|
||||
# Format prompt from template
|
||||
prompt_template = route_config.get("prompt", "")
|
||||
prompt = self._render_prompt(
|
||||
prompt_template, payload, event_type, route_name
|
||||
)
|
||||
|
||||
# Inject skill content if configured.
|
||||
# We call build_skill_invocation_message() directly rather than
|
||||
# using /skill-name slash commands — the gateway's command parser
|
||||
# would intercept those and break the flow.
|
||||
skills = route_config.get("skills", [])
|
||||
if skills:
|
||||
try:
|
||||
from agent.skill_commands import (
|
||||
build_skill_invocation_message,
|
||||
get_skill_commands,
|
||||
)
|
||||
|
||||
skill_cmds = get_skill_commands()
|
||||
for skill_name in skills:
|
||||
cmd_key = f"/{skill_name}"
|
||||
if cmd_key in skill_cmds:
|
||||
skill_content = build_skill_invocation_message(
|
||||
cmd_key, user_instruction=prompt
|
||||
)
|
||||
if skill_content:
|
||||
prompt = skill_content
|
||||
break # Load the first matching skill
|
||||
else:
|
||||
logger.warning(
|
||||
"[webhook] Skill '%s' not found", skill_name
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("[webhook] Skill loading failed: %s", e)
|
||||
|
||||
# Build a unique delivery ID
|
||||
delivery_id = request.headers.get(
|
||||
"X-GitHub-Delivery",
|
||||
request.headers.get(
|
||||
"svix-id",
|
||||
request.headers.get("X-Request-ID", str(int(time.time() * 1000))),
|
||||
),
|
||||
)
|
||||
|
||||
# ── Idempotency ─────────────────────────────────────────
|
||||
# Skip duplicate deliveries (webhook retries).
|
||||
now = time.time()
|
||||
# Prune expired entries
|
||||
self._seen_deliveries = {
|
||||
k: v
|
||||
for k, v in self._seen_deliveries.items()
|
||||
if now - v < self._idempotency_ttl
|
||||
}
|
||||
if delivery_id in self._seen_deliveries:
|
||||
logger.info(
|
||||
"[webhook] Skipping duplicate delivery %s", delivery_id
|
||||
)
|
||||
return web.json_response(
|
||||
{"status": "duplicate", "delivery_id": delivery_id},
|
||||
status=200,
|
||||
)
|
||||
self._seen_deliveries[delivery_id] = now
|
||||
|
||||
# ── Direct delivery mode (deliver_only) ─────────────────
|
||||
# Skip the agent entirely — the rendered prompt IS the message we
|
||||
# deliver. Use case: external services (Supabase, monitoring,
|
||||
# cron jobs, other agents) that need to push a plain notification
|
||||
# to a user's chat with zero LLM cost. Reuses the same HMAC auth,
|
||||
# rate limiting, idempotency, and template rendering as agent mode.
|
||||
if route_config.get("deliver_only"):
|
||||
delivery = {
|
||||
"deliver": route_config.get("deliver", "log"),
|
||||
"deliver_extra": self._render_delivery_extra(
|
||||
route_config.get("deliver_extra", {}), payload
|
||||
),
|
||||
"payload": payload,
|
||||
}
|
||||
logger.info(
|
||||
"[webhook] direct-deliver event=%s route=%s target=%s msg_len=%d delivery=%s",
|
||||
event_type,
|
||||
route_name,
|
||||
delivery["deliver"],
|
||||
len(prompt),
|
||||
delivery_id,
|
||||
)
|
||||
try:
|
||||
result = await self._direct_deliver(prompt, delivery)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"[webhook] direct-deliver failed route=%s delivery=%s",
|
||||
route_name,
|
||||
delivery_id,
|
||||
)
|
||||
return web.json_response(
|
||||
{"status": "error", "error": "Delivery failed", "delivery_id": delivery_id},
|
||||
status=502,
|
||||
)
|
||||
|
||||
if result.success:
|
||||
return web.json_response(
|
||||
{
|
||||
"status": "delivered",
|
||||
"route": route_name,
|
||||
"target": delivery["deliver"],
|
||||
"delivery_id": delivery_id,
|
||||
},
|
||||
status=200,
|
||||
)
|
||||
# Delivery attempted but target rejected it — surface as 502
|
||||
# with a generic error (don't leak adapter-level detail).
|
||||
logger.warning(
|
||||
"[webhook] direct-deliver target rejected route=%s target=%s error=%s",
|
||||
route_name,
|
||||
delivery["deliver"],
|
||||
result.error,
|
||||
)
|
||||
return web.json_response(
|
||||
{"status": "error", "error": "Delivery failed", "delivery_id": delivery_id},
|
||||
status=502,
|
||||
)
|
||||
|
||||
# Use delivery_id in session key so concurrent webhooks on the
|
||||
# same route get independent agent runs (not queued/interrupted).
|
||||
session_chat_id = f"webhook:{route_name}:{delivery_id}"
|
||||
|
||||
# Store delivery info for send(). Read by every send() invocation
|
||||
# for this chat_id (interim status messages and the final response),
|
||||
# so we do NOT pop on send. TTL-based cleanup keeps the dict bounded.
|
||||
deliver_config = {
|
||||
"deliver": route_config.get("deliver", "log"),
|
||||
"deliver_extra": self._render_delivery_extra(
|
||||
route_config.get("deliver_extra", {}), payload
|
||||
),
|
||||
"payload": payload,
|
||||
}
|
||||
self._delivery_info[session_chat_id] = deliver_config
|
||||
self._delivery_info_created[session_chat_id] = now
|
||||
self._prune_delivery_info(now)
|
||||
|
||||
# Build source and event
|
||||
source = self.build_source(
|
||||
chat_id=session_chat_id,
|
||||
chat_name=f"webhook/{route_name}",
|
||||
chat_type="webhook",
|
||||
user_id=f"webhook:{route_name}",
|
||||
user_name=route_name,
|
||||
)
|
||||
event = MessageEvent(
|
||||
text=prompt,
|
||||
message_type=MessageType.TEXT,
|
||||
source=source,
|
||||
raw_message=payload,
|
||||
message_id=delivery_id,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"[webhook] %s event=%s route=%s prompt_len=%d delivery=%s",
|
||||
request.method,
|
||||
event_type,
|
||||
route_name,
|
||||
len(prompt),
|
||||
delivery_id,
|
||||
)
|
||||
|
||||
# Non-blocking — return 202 Accepted immediately
|
||||
task = asyncio.create_task(self.handle_message(event))
|
||||
self._background_tasks.add(task)
|
||||
task.add_done_callback(self._background_tasks.discard)
|
||||
|
||||
return web.json_response(
|
||||
{
|
||||
"status": "accepted",
|
||||
"route": route_name,
|
||||
"event": event_type,
|
||||
"delivery_id": delivery_id,
|
||||
},
|
||||
status=202,
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Signature validation
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _validate_signature(
|
||||
self, request: "web.Request", body: bytes, secret: str
|
||||
) -> bool:
|
||||
"""Validate webhook signature (GitHub, GitLab, Svix, generic HMAC-SHA256)."""
|
||||
def _header(name: str) -> str:
|
||||
return (
|
||||
request.headers.get(name, "")
|
||||
or request.headers.get(name.lower(), "")
|
||||
or request.headers.get(name.upper(), "")
|
||||
)
|
||||
|
||||
# Svix / AgentMail:
|
||||
# svix-id: msg_...
|
||||
# svix-timestamp: unix seconds
|
||||
# svix-signature: v1,<base64-hmac> [v1,<base64-hmac> ...]
|
||||
# Signed content is: "{id}.{timestamp}.{raw_body}". Svix secrets
|
||||
# usually start with "whsec_" and the remainder is base64-encoded.
|
||||
svix_id = _header("svix-id")
|
||||
svix_timestamp = _header("svix-timestamp")
|
||||
svix_signature = _header("svix-signature")
|
||||
if svix_id or svix_timestamp or svix_signature:
|
||||
return self._validate_svix_signature(
|
||||
body=body,
|
||||
secret=secret,
|
||||
msg_id=svix_id,
|
||||
timestamp=svix_timestamp,
|
||||
signature_header=svix_signature,
|
||||
)
|
||||
|
||||
# GitHub: X-Hub-Signature-256 = sha256=<hex>
|
||||
gh_sig = request.headers.get("X-Hub-Signature-256", "")
|
||||
if gh_sig:
|
||||
expected = "sha256=" + hmac.new(
|
||||
secret.encode(), body, hashlib.sha256
|
||||
).hexdigest()
|
||||
return hmac.compare_digest(gh_sig, expected)
|
||||
|
||||
# GitLab: X-Gitlab-Token = <plain secret>
|
||||
gl_token = request.headers.get("X-Gitlab-Token", "")
|
||||
if gl_token:
|
||||
return hmac.compare_digest(gl_token, secret)
|
||||
|
||||
# Generic: X-Webhook-Signature = <hex HMAC-SHA256>
|
||||
generic_sig = request.headers.get("X-Webhook-Signature", "")
|
||||
if generic_sig:
|
||||
expected = hmac.new(
|
||||
secret.encode(), body, hashlib.sha256
|
||||
).hexdigest()
|
||||
return hmac.compare_digest(generic_sig, expected)
|
||||
|
||||
# No recognised signature header but secret is configured → reject
|
||||
logger.debug(
|
||||
"[webhook] Secret configured but no signature header found"
|
||||
)
|
||||
return False
|
||||
|
||||
def _validate_svix_signature(
|
||||
self,
|
||||
body: bytes,
|
||||
secret: str,
|
||||
msg_id: str,
|
||||
timestamp: str,
|
||||
signature_header: str,
|
||||
tolerance_seconds: int = 300,
|
||||
) -> bool:
|
||||
"""Validate Svix-compatible signatures used by AgentMail webhooks."""
|
||||
if not (msg_id and timestamp and signature_header and secret):
|
||||
return False
|
||||
|
||||
try:
|
||||
ts = int(timestamp)
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
if abs(int(time.time()) - ts) > tolerance_seconds:
|
||||
logger.warning("[webhook] Svix signature timestamp outside replay window")
|
||||
return False
|
||||
|
||||
if secret.startswith("whsec_"):
|
||||
encoded_secret = secret.removeprefix("whsec_")
|
||||
try:
|
||||
key = base64.b64decode(encoded_secret, validate=True)
|
||||
except (binascii.Error, ValueError):
|
||||
logger.debug("[webhook] Invalid whsec_ Svix signing secret")
|
||||
return False
|
||||
else:
|
||||
# Be permissive for providers that document Svix-style headers but
|
||||
# hand out raw shared secrets rather than whsec_ base64 secrets.
|
||||
logger.debug("[webhook] Validating Svix-style signature with raw secret")
|
||||
key = secret.encode()
|
||||
|
||||
signed_content = msg_id.encode() + b"." + timestamp.encode() + b"." + body
|
||||
expected = base64.b64encode(
|
||||
hmac.new(key, signed_content, hashlib.sha256).digest()
|
||||
).decode()
|
||||
|
||||
# Svix can send multiple signatures separated by spaces during secret
|
||||
# rotation. Each entry is formatted as "vN,<base64>".
|
||||
for part in signature_header.split():
|
||||
try:
|
||||
version, signature = part.split(",", 1)
|
||||
except ValueError:
|
||||
continue
|
||||
if version == "v1" and hmac.compare_digest(signature, expected):
|
||||
return True
|
||||
return False
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Prompt rendering
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _render_prompt(
|
||||
self,
|
||||
template: str,
|
||||
payload: dict,
|
||||
event_type: str,
|
||||
route_name: str,
|
||||
) -> str:
|
||||
"""Render a prompt template with the webhook payload.
|
||||
|
||||
Supports dot-notation access into nested dicts:
|
||||
``{pull_request.title}`` → ``payload["pull_request"]["title"]``
|
||||
|
||||
Special token ``{__raw__}`` dumps the entire payload as indented
|
||||
JSON (truncated to 4000 chars). Useful for monitoring alerts or
|
||||
any webhook where the agent needs to see the full payload.
|
||||
"""
|
||||
if not template:
|
||||
truncated = json.dumps(payload, indent=2)[:4000]
|
||||
return (
|
||||
f"Webhook event '{event_type}' on route "
|
||||
f"'{route_name}':\n\n```json\n{truncated}\n```"
|
||||
)
|
||||
|
||||
def _resolve(match: re.Match) -> str:
|
||||
key = match.group(1)
|
||||
# Special token: dump the entire payload as JSON
|
||||
if key == "__raw__":
|
||||
return json.dumps(payload, indent=2)[:4000]
|
||||
value: Any = payload
|
||||
for part in key.split("."):
|
||||
if isinstance(value, dict):
|
||||
value = value.get(part, f"{{{key}}}")
|
||||
else:
|
||||
return f"{{{key}}}"
|
||||
if isinstance(value, (dict, list)):
|
||||
return json.dumps(value, indent=2)[:2000]
|
||||
return str(value)
|
||||
|
||||
return re.sub(r"\{([a-zA-Z0-9_.]+)\}", _resolve, template)
|
||||
|
||||
def _render_delivery_extra(
|
||||
self, extra: dict, payload: dict
|
||||
) -> dict:
|
||||
"""Render delivery_extra template values with payload data."""
|
||||
rendered: Dict[str, Any] = {}
|
||||
for key, value in extra.items():
|
||||
if isinstance(value, str):
|
||||
rendered[key] = self._render_prompt(value, payload, "", "")
|
||||
else:
|
||||
rendered[key] = value
|
||||
return rendered
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Response delivery
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _direct_deliver(
|
||||
self, content: str, delivery: dict
|
||||
) -> SendResult:
|
||||
"""Deliver *content* directly without invoking the agent.
|
||||
|
||||
Used by ``deliver_only`` routes: the rendered template becomes the
|
||||
literal message body, and we dispatch to the same delivery helpers
|
||||
that the agent-mode ``send()`` flow uses. All target types that
|
||||
work in agent mode work here — Telegram, Discord, Slack, GitHub
|
||||
PR comments, etc.
|
||||
"""
|
||||
deliver_type = delivery.get("deliver", "log")
|
||||
|
||||
if deliver_type == "log":
|
||||
# Shouldn't reach here — startup validation rejects deliver_only
|
||||
# with deliver=log — but guard defensively.
|
||||
logger.info("[webhook] direct-deliver log-only: %s", content[:200])
|
||||
return SendResult(success=True)
|
||||
|
||||
if deliver_type == "github_comment":
|
||||
return await self._deliver_github_comment(content, delivery)
|
||||
|
||||
# Fall through to the cross-platform dispatcher, which validates the
|
||||
# target name and routes via the gateway runner.
|
||||
return await self._deliver_cross_platform(
|
||||
deliver_type, content, delivery
|
||||
)
|
||||
|
||||
async def _deliver_github_comment(
|
||||
self, content: str, delivery: dict
|
||||
) -> SendResult:
|
||||
"""Post agent response as a GitHub PR/issue comment via ``gh`` CLI."""
|
||||
extra = delivery.get("deliver_extra", {})
|
||||
repo = extra.get("repo", "")
|
||||
pr_number = extra.get("pr_number", "")
|
||||
|
||||
if not repo or not pr_number:
|
||||
logger.error(
|
||||
"[webhook] github_comment delivery missing repo or pr_number"
|
||||
)
|
||||
return SendResult(
|
||||
success=False, error="Missing repo or pr_number"
|
||||
)
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[
|
||||
"gh",
|
||||
"pr",
|
||||
"comment",
|
||||
str(pr_number),
|
||||
"--repo",
|
||||
repo,
|
||||
"--body",
|
||||
content,
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
logger.info(
|
||||
"[webhook] Posted comment on %s#%s", repo, pr_number
|
||||
)
|
||||
return SendResult(success=True)
|
||||
else:
|
||||
logger.error(
|
||||
"[webhook] gh pr comment failed: %s", result.stderr
|
||||
)
|
||||
return SendResult(success=False, error=result.stderr)
|
||||
except FileNotFoundError:
|
||||
logger.error(
|
||||
"[webhook] 'gh' CLI not found — install GitHub CLI for "
|
||||
"github_comment delivery"
|
||||
)
|
||||
return SendResult(
|
||||
success=False, error="gh CLI not installed"
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error("[webhook] github_comment delivery error: %s", e)
|
||||
return SendResult(success=False, error=str(e))
|
||||
|
||||
async def _deliver_cross_platform(
|
||||
self, platform_name: str, content: str, delivery: dict
|
||||
) -> SendResult:
|
||||
"""Route response to another platform (telegram, discord, etc.)."""
|
||||
if not self.gateway_runner:
|
||||
return SendResult(
|
||||
success=False,
|
||||
error="No gateway runner for cross-platform delivery",
|
||||
)
|
||||
|
||||
try:
|
||||
target_platform = Platform(platform_name)
|
||||
except ValueError:
|
||||
return SendResult(
|
||||
success=False, error=f"Unknown platform: {platform_name}"
|
||||
)
|
||||
|
||||
adapter = self.gateway_runner.adapters.get(target_platform)
|
||||
if not adapter:
|
||||
return SendResult(
|
||||
success=False,
|
||||
error=f"Platform {platform_name} not connected",
|
||||
)
|
||||
|
||||
# Use home channel if no specific chat_id in deliver_extra
|
||||
extra = delivery.get("deliver_extra", {})
|
||||
chat_id = extra.get("chat_id", "")
|
||||
if not chat_id:
|
||||
home = self.gateway_runner.config.get_home_channel(target_platform)
|
||||
if home:
|
||||
chat_id = home.chat_id
|
||||
else:
|
||||
return SendResult(
|
||||
success=False,
|
||||
error=f"No chat_id or home channel for {platform_name}",
|
||||
)
|
||||
|
||||
# Pass thread_id from deliver_extra so Telegram forum topics work
|
||||
metadata = None
|
||||
thread_id = extra.get("message_thread_id") or extra.get("thread_id")
|
||||
if thread_id:
|
||||
metadata = {"thread_id": thread_id}
|
||||
|
||||
return await adapter.send(chat_id, content, metadata=metadata)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,425 @@
|
||||
"""WeCom callback-mode adapter for self-built enterprise applications.
|
||||
|
||||
Unlike the bot/websocket adapter in ``wecom.py``, this handles the standard
|
||||
WeCom callback flow: WeCom POSTs encrypted XML to an HTTP endpoint, the
|
||||
adapter decrypts it, queues the message for the agent, and immediately
|
||||
acknowledges. The agent's reply is delivered later via the proactive
|
||||
``message/send`` API using an access-token.
|
||||
|
||||
Supports multiple self-built apps under one gateway instance, scoped by
|
||||
``corp_id:user_id`` to avoid cross-corp collisions.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import socket as _socket
|
||||
import time
|
||||
from typing import Any, Dict, List, Optional
|
||||
# Security: parse untrusted, pre-auth request bodies (WeCom callbacks) with
|
||||
# defusedxml to block billion-laughs / entity-expansion (and XXE) DoS. The
|
||||
# parsing API (fromstring) is a drop-in for the stdlib calls used below;
|
||||
# response-building XML lives in wecom_crypto.py and is not parsed here.
|
||||
try:
|
||||
import defusedxml.ElementTree as ET
|
||||
|
||||
DEFUSEDXML_AVAILABLE = True
|
||||
except ImportError:
|
||||
ET = None # type: ignore[assignment]
|
||||
DEFUSEDXML_AVAILABLE = False
|
||||
|
||||
try:
|
||||
from aiohttp import web
|
||||
|
||||
AIOHTTP_AVAILABLE = True
|
||||
except ImportError:
|
||||
web = None # type: ignore[assignment]
|
||||
AIOHTTP_AVAILABLE = False
|
||||
|
||||
try:
|
||||
import httpx
|
||||
|
||||
HTTPX_AVAILABLE = True
|
||||
except ImportError:
|
||||
httpx = None # type: ignore[assignment]
|
||||
HTTPX_AVAILABLE = False
|
||||
|
||||
from gateway.config import Platform, PlatformConfig
|
||||
from gateway.platforms.base import BasePlatformAdapter, MessageEvent, MessageType, SendResult
|
||||
from gateway.platforms.wecom_crypto import WXBizMsgCrypt, WeComCryptoError
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
DEFAULT_HOST = "0.0.0.0"
|
||||
DEFAULT_PORT = 8645
|
||||
DEFAULT_PATH = "/wecom/callback"
|
||||
ACCESS_TOKEN_TTL_SECONDS = 7200
|
||||
MESSAGE_DEDUP_TTL_SECONDS = 300
|
||||
|
||||
|
||||
def check_wecom_callback_requirements() -> bool:
|
||||
return AIOHTTP_AVAILABLE and HTTPX_AVAILABLE and DEFUSEDXML_AVAILABLE
|
||||
|
||||
|
||||
class WecomCallbackAdapter(BasePlatformAdapter):
|
||||
def __init__(self, config: PlatformConfig):
|
||||
super().__init__(config, Platform.WECOM_CALLBACK)
|
||||
extra = config.extra or {}
|
||||
self._host = str(extra.get("host") or DEFAULT_HOST)
|
||||
self._port = int(extra.get("port") or DEFAULT_PORT)
|
||||
self._path = str(extra.get("path") or DEFAULT_PATH)
|
||||
self._apps: List[Dict[str, Any]] = self._normalize_apps(extra)
|
||||
self._runner: Optional[web.AppRunner] = None
|
||||
self._site: Optional[web.TCPSite] = None
|
||||
self._app: Optional[web.Application] = None
|
||||
self._http_client: Optional[httpx.AsyncClient] = None
|
||||
self._message_queue: asyncio.Queue[MessageEvent] = asyncio.Queue()
|
||||
self._poll_task: Optional[asyncio.Task] = None
|
||||
self._seen_messages: Dict[str, float] = {}
|
||||
self._user_app_map: Dict[str, str] = {}
|
||||
self._access_tokens: Dict[str, Dict[str, Any]] = {}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# App normalisation
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@staticmethod
|
||||
def _user_app_key(corp_id: str, user_id: str) -> str:
|
||||
return f"{corp_id}:{user_id}" if corp_id else user_id
|
||||
|
||||
@staticmethod
|
||||
def _normalize_apps(extra: Dict[str, Any]) -> List[Dict[str, Any]]:
|
||||
apps = extra.get("apps")
|
||||
if isinstance(apps, list) and apps:
|
||||
return [dict(app) for app in apps if isinstance(app, dict)]
|
||||
if extra.get("corp_id"):
|
||||
return [
|
||||
{
|
||||
"name": extra.get("name") or "default",
|
||||
"corp_id": extra.get("corp_id", ""),
|
||||
"corp_secret": extra.get("corp_secret", ""),
|
||||
"agent_id": str(extra.get("agent_id", "")),
|
||||
"token": extra.get("token", ""),
|
||||
"encoding_aes_key": extra.get("encoding_aes_key", ""),
|
||||
}
|
||||
]
|
||||
return []
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Lifecycle
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def connect(self) -> bool:
|
||||
if not self._apps:
|
||||
logger.warning("[WecomCallback] No callback apps configured")
|
||||
return False
|
||||
if not check_wecom_callback_requirements():
|
||||
logger.warning("[WecomCallback] aiohttp/httpx not installed")
|
||||
return False
|
||||
|
||||
# Quick port-in-use check.
|
||||
try:
|
||||
with _socket.socket(_socket.AF_INET, _socket.SOCK_STREAM) as sock:
|
||||
sock.settimeout(1)
|
||||
sock.connect(("127.0.0.1", self._port))
|
||||
logger.error("[WecomCallback] Port %d already in use", self._port)
|
||||
return False
|
||||
except (ConnectionRefusedError, OSError):
|
||||
pass
|
||||
|
||||
try:
|
||||
# Tighter keepalive so idle CLOSE_WAIT drains promptly (#18451).
|
||||
from gateway.platforms._http_client_limits import platform_httpx_limits
|
||||
self._http_client = httpx.AsyncClient(timeout=20.0, limits=platform_httpx_limits())
|
||||
self._app = web.Application()
|
||||
self._app.router.add_get("/health", self._handle_health)
|
||||
self._app.router.add_get(self._path, self._handle_verify)
|
||||
self._app.router.add_post(self._path, self._handle_callback)
|
||||
self._runner = web.AppRunner(self._app)
|
||||
await self._runner.setup()
|
||||
self._site = web.TCPSite(self._runner, self._host, self._port)
|
||||
await self._site.start()
|
||||
self._poll_task = asyncio.create_task(self._poll_loop())
|
||||
self._mark_connected()
|
||||
logger.info(
|
||||
"[WecomCallback] HTTP server listening on %s:%s%s",
|
||||
self._host, self._port, self._path,
|
||||
)
|
||||
for app in self._apps:
|
||||
try:
|
||||
await self._refresh_access_token(app)
|
||||
except Exception as exc:
|
||||
logger.warning(
|
||||
"[WecomCallback] Initial token refresh failed for app '%s': %s",
|
||||
app.get("name", "default"), exc,
|
||||
)
|
||||
return True
|
||||
except Exception:
|
||||
await self._cleanup()
|
||||
logger.exception("[WecomCallback] Failed to start")
|
||||
return False
|
||||
|
||||
async def disconnect(self) -> None:
|
||||
self._running = False
|
||||
if self._poll_task:
|
||||
self._poll_task.cancel()
|
||||
try:
|
||||
await self._poll_task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
self._poll_task = None
|
||||
await self._cleanup()
|
||||
self._mark_disconnected()
|
||||
logger.info("[WecomCallback] Disconnected")
|
||||
|
||||
async def _cleanup(self) -> None:
|
||||
self._site = None
|
||||
if self._runner:
|
||||
await self._runner.cleanup()
|
||||
self._runner = None
|
||||
self._app = None
|
||||
if self._http_client:
|
||||
await self._http_client.aclose()
|
||||
self._http_client = None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Outbound: proactive send via access-token API
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def send(
|
||||
self,
|
||||
chat_id: str,
|
||||
content: str,
|
||||
reply_to: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> SendResult:
|
||||
app = self._resolve_app_for_chat(chat_id)
|
||||
touser = chat_id.split(":", 1)[1] if ":" in chat_id else chat_id
|
||||
try:
|
||||
payload = {
|
||||
"touser": touser,
|
||||
"msgtype": "text",
|
||||
"agentid": int(str(app.get("agent_id") or 0)),
|
||||
"text": {"content": content[:2048]},
|
||||
"safe": 0,
|
||||
}
|
||||
for _attempt in range(2):
|
||||
token = await self._get_access_token(app)
|
||||
resp = await self._http_client.post(
|
||||
f"https://qyapi.weixin.qq.com/cgi-bin/message/send?access_token={token}",
|
||||
json=payload,
|
||||
)
|
||||
data = resp.json()
|
||||
errcode = data.get("errcode")
|
||||
if errcode in {40001, 42001} and _attempt == 0:
|
||||
# WeCom rejected the token — evict the cached entry so
|
||||
# the next _get_access_token call forces a fresh fetch.
|
||||
logger.warning(
|
||||
"[WecomCallback] Token rejected for app '%s' (errcode=%s), refreshing",
|
||||
app.get("name", "default"), errcode,
|
||||
)
|
||||
self._access_tokens.pop(app["name"], None)
|
||||
continue
|
||||
if errcode != 0:
|
||||
return SendResult(success=False, error=str(data))
|
||||
return SendResult(
|
||||
success=True,
|
||||
message_id=str(data.get("msgid", "")),
|
||||
raw_response=data,
|
||||
)
|
||||
return SendResult(success=False, error="send failed after token refresh")
|
||||
except Exception as exc:
|
||||
return SendResult(success=False, error=str(exc))
|
||||
|
||||
def _resolve_app_for_chat(self, chat_id: str) -> Dict[str, Any]:
|
||||
"""Pick the app associated with *chat_id*, falling back sensibly."""
|
||||
app_name = self._user_app_map.get(chat_id)
|
||||
if not app_name and ":" not in chat_id:
|
||||
# Legacy bare user_id — try to find a unique match.
|
||||
matching = [k for k in self._user_app_map if k.endswith(f":{chat_id}")]
|
||||
if len(matching) == 1:
|
||||
app_name = self._user_app_map.get(matching[0])
|
||||
app = self._get_app_by_name(app_name) if app_name else None
|
||||
return app or self._apps[0]
|
||||
|
||||
async def get_chat_info(self, chat_id: str) -> Dict[str, Any]:
|
||||
return {"name": chat_id, "type": "dm"}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Inbound: HTTP callback handlers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _handle_health(self, request: web.Request) -> web.Response:
|
||||
return web.json_response({"status": "ok", "platform": "wecom_callback"})
|
||||
|
||||
async def _handle_verify(self, request: web.Request) -> web.Response:
|
||||
"""GET endpoint — WeCom URL verification handshake."""
|
||||
msg_signature = request.query.get("msg_signature", "")
|
||||
timestamp = request.query.get("timestamp", "")
|
||||
nonce = request.query.get("nonce", "")
|
||||
echostr = request.query.get("echostr", "")
|
||||
for app in self._apps:
|
||||
try:
|
||||
crypt = self._crypt_for_app(app)
|
||||
plain = crypt.verify_url(msg_signature, timestamp, nonce, echostr)
|
||||
return web.Response(text=plain, content_type="text/plain")
|
||||
except Exception:
|
||||
continue
|
||||
return web.Response(status=403, text="signature verification failed")
|
||||
|
||||
async def _handle_callback(self, request: web.Request) -> web.Response:
|
||||
"""POST endpoint — receive an encrypted message callback."""
|
||||
msg_signature = request.query.get("msg_signature", "")
|
||||
timestamp = request.query.get("timestamp", "")
|
||||
nonce = request.query.get("nonce", "")
|
||||
body = await request.text()
|
||||
|
||||
for app in self._apps:
|
||||
try:
|
||||
decrypted = self._decrypt_request(
|
||||
app, body, msg_signature, timestamp, nonce,
|
||||
)
|
||||
event = self._build_event(app, decrypted)
|
||||
if event is not None:
|
||||
# Deduplicate: WeCom retries callbacks on timeout,
|
||||
# producing duplicate inbound messages (#10305).
|
||||
if event.message_id:
|
||||
now = time.time()
|
||||
if event.message_id in self._seen_messages:
|
||||
if now - self._seen_messages[event.message_id] < MESSAGE_DEDUP_TTL_SECONDS:
|
||||
logger.debug("[WecomCallback] Duplicate MsgId %s, skipping", event.message_id)
|
||||
return web.Response(text="success", content_type="text/plain")
|
||||
del self._seen_messages[event.message_id]
|
||||
self._seen_messages[event.message_id] = now
|
||||
# Prune expired entries when cache grows large
|
||||
if len(self._seen_messages) > 2000:
|
||||
cutoff = now - MESSAGE_DEDUP_TTL_SECONDS
|
||||
self._seen_messages = {k: v for k, v in self._seen_messages.items() if v > cutoff}
|
||||
# Record which app this user belongs to.
|
||||
if event.source and event.source.user_id:
|
||||
map_key = self._user_app_key(
|
||||
str(app.get("corp_id") or ""), event.source.user_id,
|
||||
)
|
||||
self._user_app_map[map_key] = app["name"]
|
||||
await self._message_queue.put(event)
|
||||
# Immediately acknowledge — the agent's reply will arrive
|
||||
# later via the proactive message/send API.
|
||||
return web.Response(text="success", content_type="text/plain")
|
||||
except WeComCryptoError:
|
||||
continue
|
||||
except Exception:
|
||||
logger.exception("[WecomCallback] Error handling message")
|
||||
break
|
||||
return web.Response(status=400, text="invalid callback payload")
|
||||
|
||||
async def _poll_loop(self) -> None:
|
||||
"""Drain the message queue and dispatch to the gateway runner."""
|
||||
while True:
|
||||
event = await self._message_queue.get()
|
||||
try:
|
||||
task = asyncio.create_task(self.handle_message(event))
|
||||
self._background_tasks.add(task)
|
||||
task.add_done_callback(self._background_tasks.discard)
|
||||
except Exception:
|
||||
logger.exception("[WecomCallback] Failed to enqueue event")
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# XML / crypto helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _decrypt_request(
|
||||
self, app: Dict[str, Any], body: str,
|
||||
msg_signature: str, timestamp: str, nonce: str,
|
||||
) -> str:
|
||||
root = ET.fromstring(body)
|
||||
encrypt = root.findtext("Encrypt", default="")
|
||||
crypt = self._crypt_for_app(app)
|
||||
return crypt.decrypt(msg_signature, timestamp, nonce, encrypt).decode("utf-8")
|
||||
|
||||
def _build_event(self, app: Dict[str, Any], xml_text: str) -> Optional[MessageEvent]:
|
||||
root = ET.fromstring(xml_text)
|
||||
msg_type = (root.findtext("MsgType") or "").lower()
|
||||
# Silently acknowledge lifecycle events.
|
||||
if msg_type == "event":
|
||||
event_name = (root.findtext("Event") or "").lower()
|
||||
if event_name in {"enter_agent", "subscribe"}:
|
||||
return None
|
||||
if msg_type not in {"text", "event"}:
|
||||
return None
|
||||
|
||||
user_id = root.findtext("FromUserName", default="")
|
||||
corp_id = root.findtext("ToUserName", default=app.get("corp_id", ""))
|
||||
scoped_chat_id = self._user_app_key(corp_id, user_id)
|
||||
content = root.findtext("Content", default="").strip()
|
||||
if not content and msg_type == "event":
|
||||
content = "/start"
|
||||
msg_id = (
|
||||
root.findtext("MsgId")
|
||||
or f"{user_id}:{root.findtext('CreateTime', default='0')}"
|
||||
)
|
||||
source = self.build_source(
|
||||
chat_id=scoped_chat_id,
|
||||
chat_name=user_id,
|
||||
chat_type="dm",
|
||||
user_id=user_id,
|
||||
user_name=user_id,
|
||||
)
|
||||
return MessageEvent(
|
||||
text=content,
|
||||
message_type=MessageType.TEXT,
|
||||
source=source,
|
||||
raw_message=xml_text,
|
||||
message_id=msg_id,
|
||||
)
|
||||
|
||||
def _crypt_for_app(self, app: Dict[str, Any]) -> WXBizMsgCrypt:
|
||||
return WXBizMsgCrypt(
|
||||
token=str(app.get("token") or ""),
|
||||
encoding_aes_key=str(app.get("encoding_aes_key") or ""),
|
||||
receive_id=str(app.get("corp_id") or ""),
|
||||
)
|
||||
|
||||
def _get_app_by_name(self, name: Optional[str]) -> Optional[Dict[str, Any]]:
|
||||
if not name:
|
||||
return None
|
||||
for app in self._apps:
|
||||
if app.get("name") == name:
|
||||
return app
|
||||
return None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Access-token management
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def _get_access_token(self, app: Dict[str, Any]) -> str:
|
||||
cached = self._access_tokens.get(app["name"])
|
||||
now = time.time()
|
||||
if cached and cached.get("expires_at", 0) > now + 60:
|
||||
return cached["token"]
|
||||
return await self._refresh_access_token(app)
|
||||
|
||||
async def _refresh_access_token(self, app: Dict[str, Any]) -> str:
|
||||
resp = await self._http_client.get(
|
||||
"https://qyapi.weixin.qq.com/cgi-bin/gettoken",
|
||||
params={
|
||||
"corpid": app.get("corp_id"),
|
||||
"corpsecret": app.get("corp_secret"),
|
||||
},
|
||||
)
|
||||
data = resp.json()
|
||||
if data.get("errcode") != 0:
|
||||
raise RuntimeError(f"WeCom token refresh failed: {data}")
|
||||
token = data["access_token"]
|
||||
expires_in = int(data.get("expires_in", ACCESS_TOKEN_TTL_SECONDS))
|
||||
self._access_tokens[app["name"]] = {
|
||||
"token": token,
|
||||
"expires_at": time.time() + expires_in,
|
||||
}
|
||||
logger.info(
|
||||
"[WecomCallback] Token refreshed for app '%s' (corp=%s), expires in %ss",
|
||||
app.get("name", "default"),
|
||||
app.get("corp_id", ""),
|
||||
expires_in,
|
||||
)
|
||||
return token
|
||||
@@ -0,0 +1,142 @@
|
||||
"""WeCom BizMsgCrypt-compatible AES-CBC encryption for callback mode.
|
||||
|
||||
Implements the same wire format as Tencent's official ``WXBizMsgCrypt``
|
||||
SDK so that WeCom can verify, encrypt, and decrypt callback payloads.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import os
|
||||
import secrets
|
||||
import socket
|
||||
import struct
|
||||
from typing import Optional
|
||||
from xml.etree import ElementTree as ET
|
||||
|
||||
from cryptography.hazmat.backends import default_backend
|
||||
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
|
||||
|
||||
|
||||
class WeComCryptoError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class SignatureError(WeComCryptoError):
|
||||
pass
|
||||
|
||||
|
||||
class DecryptError(WeComCryptoError):
|
||||
pass
|
||||
|
||||
|
||||
class EncryptError(WeComCryptoError):
|
||||
pass
|
||||
|
||||
|
||||
class PKCS7Encoder:
|
||||
block_size = 32
|
||||
|
||||
@classmethod
|
||||
def encode(cls, text: bytes) -> bytes:
|
||||
amount_to_pad = cls.block_size - (len(text) % cls.block_size)
|
||||
if amount_to_pad == 0:
|
||||
amount_to_pad = cls.block_size
|
||||
pad = bytes([amount_to_pad]) * amount_to_pad
|
||||
return text + pad
|
||||
|
||||
@classmethod
|
||||
def decode(cls, decrypted: bytes) -> bytes:
|
||||
if not decrypted:
|
||||
raise DecryptError("empty decrypted payload")
|
||||
pad = decrypted[-1]
|
||||
if pad < 1 or pad > cls.block_size:
|
||||
raise DecryptError("invalid PKCS7 padding")
|
||||
if decrypted[-pad:] != bytes([pad]) * pad:
|
||||
raise DecryptError("malformed PKCS7 padding")
|
||||
return decrypted[:-pad]
|
||||
|
||||
|
||||
def _sha1_signature(token: str, timestamp: str, nonce: str, encrypt: str) -> str:
|
||||
parts = sorted([token, timestamp, nonce, encrypt])
|
||||
return hashlib.sha1("".join(parts).encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
class WXBizMsgCrypt:
|
||||
"""Minimal WeCom callback crypto helper compatible with BizMsgCrypt semantics."""
|
||||
|
||||
def __init__(self, token: str, encoding_aes_key: str, receive_id: str):
|
||||
if not token:
|
||||
raise ValueError("token is required")
|
||||
if not encoding_aes_key:
|
||||
raise ValueError("encoding_aes_key is required")
|
||||
if len(encoding_aes_key) != 43:
|
||||
raise ValueError("encoding_aes_key must be 43 chars")
|
||||
if not receive_id:
|
||||
raise ValueError("receive_id is required")
|
||||
|
||||
self.token = token
|
||||
self.receive_id = receive_id
|
||||
self.key = base64.b64decode(encoding_aes_key + "=")
|
||||
self.iv = self.key[:16]
|
||||
|
||||
def verify_url(self, msg_signature: str, timestamp: str, nonce: str, echostr: str) -> str:
|
||||
plain = self.decrypt(msg_signature, timestamp, nonce, echostr)
|
||||
return plain.decode("utf-8")
|
||||
|
||||
def decrypt(self, msg_signature: str, timestamp: str, nonce: str, encrypt: str) -> bytes:
|
||||
expected = _sha1_signature(self.token, timestamp, nonce, encrypt)
|
||||
if expected != msg_signature:
|
||||
raise SignatureError("signature mismatch")
|
||||
try:
|
||||
cipher_text = base64.b64decode(encrypt)
|
||||
except Exception as exc:
|
||||
raise DecryptError(f"invalid base64 payload: {exc}") from exc
|
||||
try:
|
||||
cipher = Cipher(algorithms.AES(self.key), modes.CBC(self.iv), backend=default_backend())
|
||||
decryptor = cipher.decryptor()
|
||||
padded = decryptor.update(cipher_text) + decryptor.finalize()
|
||||
plain = PKCS7Encoder.decode(padded)
|
||||
content = plain[16:] # skip 16-byte random prefix
|
||||
xml_length = socket.ntohl(struct.unpack("I", content[:4])[0])
|
||||
xml_content = content[4:4 + xml_length]
|
||||
receive_id = content[4 + xml_length:].decode("utf-8")
|
||||
except WeComCryptoError:
|
||||
raise
|
||||
except Exception as exc:
|
||||
raise DecryptError(f"decrypt failed: {exc}") from exc
|
||||
|
||||
if receive_id != self.receive_id:
|
||||
raise DecryptError("receive_id mismatch")
|
||||
return xml_content
|
||||
|
||||
def encrypt(self, plaintext: str, nonce: Optional[str] = None, timestamp: Optional[str] = None) -> str:
|
||||
nonce = nonce or self._random_nonce()
|
||||
timestamp = timestamp or str(int(__import__("time").time()))
|
||||
encrypt = self._encrypt_bytes(plaintext.encode("utf-8"))
|
||||
signature = _sha1_signature(self.token, timestamp, nonce, encrypt)
|
||||
root = ET.Element("xml")
|
||||
ET.SubElement(root, "Encrypt").text = encrypt
|
||||
ET.SubElement(root, "MsgSignature").text = signature
|
||||
ET.SubElement(root, "TimeStamp").text = timestamp
|
||||
ET.SubElement(root, "Nonce").text = nonce
|
||||
return ET.tostring(root, encoding="unicode")
|
||||
|
||||
def _encrypt_bytes(self, raw: bytes) -> str:
|
||||
try:
|
||||
random_prefix = os.urandom(16)
|
||||
msg_len = struct.pack("I", socket.htonl(len(raw)))
|
||||
payload = random_prefix + msg_len + raw + self.receive_id.encode("utf-8")
|
||||
padded = PKCS7Encoder.encode(payload)
|
||||
cipher = Cipher(algorithms.AES(self.key), modes.CBC(self.iv), backend=default_backend())
|
||||
encryptor = cipher.encryptor()
|
||||
encrypted = encryptor.update(padded) + encryptor.finalize()
|
||||
return base64.b64encode(encrypted).decode("utf-8")
|
||||
except Exception as exc:
|
||||
raise EncryptError(f"encrypt failed: {exc}") from exc
|
||||
|
||||
@staticmethod
|
||||
def _random_nonce(length: int = 10) -> str:
|
||||
alphabet = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
|
||||
return "".join(secrets.choice(alphabet) for _ in range(length))
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,645 @@
|
||||
"""
|
||||
yuanbao_media.py — 元宝平台媒体处理模块
|
||||
|
||||
提供 COS 上传、文件下载、TIM 媒体消息构建等功能。
|
||||
移植自 TypeScript 版 media.ts(yuanbao-openclaw-plugin),
|
||||
使用 httpx 替代 cos-nodejs-sdk-v5,避免引入额外 SDK 依赖。
|
||||
|
||||
COS 上传流程:
|
||||
1. 调用 genUploadInfo 获取临时凭证(tmpSecretId/tmpSecretKey/sessionToken)
|
||||
2. 用临时凭证通过 HMAC-SHA1 签名构建 Authorization 头
|
||||
3. HTTP PUT 上传到 COS
|
||||
|
||||
TIM 消息体构建:
|
||||
- buildImageMsgBody() → TIMImageElem
|
||||
- buildFileMsgBody() → TIMFileElem
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import hmac
|
||||
import logging
|
||||
import os
|
||||
import secrets
|
||||
import struct
|
||||
import time
|
||||
import urllib.parse
|
||||
from typing import Optional, Any
|
||||
|
||||
import httpx
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ============ 常量 ============
|
||||
|
||||
UPLOAD_INFO_PATH = "/api/resource/genUploadInfo"
|
||||
DEFAULT_API_DOMAIN = "yuanbao.tencent.com"
|
||||
DEFAULT_MAX_SIZE_MB = 50
|
||||
|
||||
# COS 加速域名后缀(优先使用全球加速)
|
||||
COS_USE_ACCELERATE = True
|
||||
|
||||
# ============ 类型映射 ============
|
||||
|
||||
# MIME → image_format 数字(TIM 协议字段)
|
||||
_MIME_TO_IMAGE_FORMAT: dict[str, int] = {
|
||||
"image/jpeg": 1,
|
||||
"image/jpg": 1,
|
||||
"image/gif": 2,
|
||||
"image/png": 3,
|
||||
"image/bmp": 4,
|
||||
"image/webp": 255,
|
||||
"image/heic": 255,
|
||||
"image/tiff": 255,
|
||||
}
|
||||
|
||||
# 文件扩展名 → MIME
|
||||
_EXT_TO_MIME: dict[str, str] = {
|
||||
".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg",
|
||||
".png": "image/png",
|
||||
".gif": "image/gif",
|
||||
".webp": "image/webp",
|
||||
".bmp": "image/bmp",
|
||||
".heic": "image/heic",
|
||||
".tiff": "image/tiff",
|
||||
".ico": "image/x-icon",
|
||||
".pdf": "application/pdf",
|
||||
".doc": "application/msword",
|
||||
".docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
|
||||
".xls": "application/vnd.ms-excel",
|
||||
".xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
|
||||
".ppt": "application/vnd.ms-powerpoint",
|
||||
".pptx": "application/vnd.openxmlformats-officedocument.presentationml.presentation",
|
||||
".txt": "text/plain",
|
||||
".zip": "application/zip",
|
||||
".tar": "application/x-tar",
|
||||
".gz": "application/gzip",
|
||||
".mp3": "audio/mpeg",
|
||||
".mp4": "video/mp4",
|
||||
".wav": "audio/wav",
|
||||
".ogg": "audio/ogg",
|
||||
".webm": "video/webm",
|
||||
}
|
||||
|
||||
|
||||
# ============ 工具函数 ============
|
||||
|
||||
def guess_mime_type(filename: str) -> str:
|
||||
"""根据文件扩展名猜测 MIME 类型。"""
|
||||
ext = os.path.splitext(filename)[-1].lower()
|
||||
return _EXT_TO_MIME.get(ext, "application/octet-stream")
|
||||
|
||||
|
||||
def is_image(filename: str, mime_type: str = "") -> bool:
|
||||
"""判断是否为图片类型。"""
|
||||
if mime_type.startswith("image/"):
|
||||
return True
|
||||
ext = os.path.splitext(filename)[-1].lower()
|
||||
return ext in {".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp", ".heic", ".tiff", ".ico"}
|
||||
|
||||
|
||||
def get_image_format(mime_type: str) -> int:
|
||||
"""获取 TIM 图片格式编号。"""
|
||||
return _MIME_TO_IMAGE_FORMAT.get(mime_type.lower(), 255)
|
||||
|
||||
|
||||
def md5_hex(data: bytes) -> str:
|
||||
"""计算 MD5 十六进制摘要。"""
|
||||
return hashlib.md5(data).hexdigest()
|
||||
|
||||
|
||||
def generate_file_id() -> str:
|
||||
"""生成随机文件 ID(32 位 hex)。"""
|
||||
return secrets.token_hex(16)
|
||||
|
||||
|
||||
|
||||
# ============ 图片尺寸解析(纯 Python,无需 Pillow) ============
|
||||
|
||||
def parse_image_size(data: bytes) -> Optional[dict[str, int]]:
|
||||
"""
|
||||
解析图片宽高(支持 JPEG/PNG/GIF/WebP),无需第三方依赖。
|
||||
返回 {"width": w, "height": h} 或 None(无法识别)。
|
||||
"""
|
||||
return (
|
||||
_parse_png_size(data)
|
||||
or _parse_jpeg_size(data)
|
||||
or _parse_gif_size(data)
|
||||
or _parse_webp_size(data)
|
||||
)
|
||||
|
||||
|
||||
def _parse_png_size(buf: bytes) -> Optional[dict[str, int]]:
|
||||
if len(buf) < 24:
|
||||
return None
|
||||
if buf[:4] != b"\x89PNG":
|
||||
return None
|
||||
w = struct.unpack(">I", buf[16:20])[0]
|
||||
h = struct.unpack(">I", buf[20:24])[0]
|
||||
return {"width": w, "height": h}
|
||||
|
||||
|
||||
def _parse_jpeg_size(buf: bytes) -> Optional[dict[str, int]]:
|
||||
if len(buf) < 4 or buf[0] != 0xFF or buf[1] != 0xD8:
|
||||
return None
|
||||
i = 2
|
||||
while i < len(buf) - 9:
|
||||
if buf[i] != 0xFF:
|
||||
i += 1
|
||||
continue
|
||||
marker = buf[i + 1]
|
||||
if marker in {0xC0, 0xC2}:
|
||||
h = struct.unpack(">H", buf[i + 5: i + 7])[0]
|
||||
w = struct.unpack(">H", buf[i + 7: i + 9])[0]
|
||||
return {"width": w, "height": h}
|
||||
if i + 3 < len(buf):
|
||||
i += 2 + struct.unpack(">H", buf[i + 2: i + 4])[0]
|
||||
else:
|
||||
break
|
||||
return None
|
||||
|
||||
|
||||
def _parse_gif_size(buf: bytes) -> Optional[dict[str, int]]:
|
||||
if len(buf) < 10:
|
||||
return None
|
||||
sig = buf[:6].decode("ascii", errors="replace")
|
||||
if sig not in {"GIF87a", "GIF89a"}:
|
||||
return None
|
||||
w = struct.unpack("<H", buf[6:8])[0]
|
||||
h = struct.unpack("<H", buf[8:10])[0]
|
||||
return {"width": w, "height": h}
|
||||
|
||||
|
||||
def _parse_webp_size(buf: bytes) -> Optional[dict[str, int]]:
|
||||
if len(buf) < 16:
|
||||
return None
|
||||
if buf[:4] != b"RIFF" or buf[8:12] != b"WEBP":
|
||||
return None
|
||||
chunk = buf[12:16].decode("ascii", errors="replace")
|
||||
if chunk == "VP8 ":
|
||||
if len(buf) >= 30 and buf[23] == 0x9D and buf[24] == 0x01 and buf[25] == 0x2A:
|
||||
w = struct.unpack("<H", buf[26:28])[0] & 0x3FFF
|
||||
h = struct.unpack("<H", buf[28:30])[0] & 0x3FFF
|
||||
return {"width": w, "height": h}
|
||||
elif chunk == "VP8L":
|
||||
if len(buf) >= 25 and buf[20] == 0x2F:
|
||||
bits = struct.unpack("<I", buf[21:25])[0]
|
||||
w = (bits & 0x3FFF) + 1
|
||||
h = ((bits >> 14) & 0x3FFF) + 1
|
||||
return {"width": w, "height": h}
|
||||
elif chunk == "VP8X":
|
||||
if len(buf) >= 30:
|
||||
w = (buf[24] | (buf[25] << 8) | (buf[26] << 16)) + 1
|
||||
h = (buf[27] | (buf[28] << 8) | (buf[29] << 16)) + 1
|
||||
return {"width": w, "height": h}
|
||||
return None
|
||||
|
||||
|
||||
# ============ URL 下载 ============
|
||||
|
||||
async def download_url(
|
||||
url: str,
|
||||
max_size_mb: int = DEFAULT_MAX_SIZE_MB,
|
||||
) -> tuple[bytes, str]:
|
||||
"""
|
||||
下载 URL 内容,返回 (bytes, content_type)。
|
||||
|
||||
Args:
|
||||
url: HTTP(S) URL
|
||||
max_size_mb: 最大允许大小(MB),超过则抛出异常
|
||||
|
||||
Returns:
|
||||
(data_bytes, content_type_string)
|
||||
|
||||
Raises:
|
||||
ValueError: 内容超过大小限制
|
||||
httpx.HTTPError: 网络/HTTP 错误
|
||||
"""
|
||||
max_bytes = max_size_mb * 1024 * 1024
|
||||
async with httpx.AsyncClient(timeout=30.0, follow_redirects=True) as client:
|
||||
# 先 HEAD 检查大小
|
||||
try:
|
||||
head = await client.head(url)
|
||||
content_length = int(head.headers.get("content-length", 0) or 0)
|
||||
if content_length > 0 and content_length > max_bytes:
|
||||
raise ValueError(
|
||||
f"文件过大: {content_length / 1024 / 1024:.1f} MB > {max_size_mb} MB"
|
||||
)
|
||||
except httpx.HTTPStatusError:
|
||||
pass # 部分服务器不支持 HEAD,忽略
|
||||
|
||||
# GET 下载(流式读取,防止超限)
|
||||
async with client.stream("GET", url) as resp:
|
||||
resp.raise_for_status()
|
||||
|
||||
content_type = resp.headers.get("content-type", "").split(";")[0].strip()
|
||||
|
||||
chunks: list[bytes] = []
|
||||
downloaded = 0
|
||||
async for chunk in resp.aiter_bytes(65536):
|
||||
downloaded += len(chunk)
|
||||
if downloaded > max_bytes:
|
||||
raise ValueError(
|
||||
f"文件过大: 已超过 {max_size_mb} MB 限制"
|
||||
)
|
||||
chunks.append(chunk)
|
||||
|
||||
data = b"".join(chunks)
|
||||
return data, content_type
|
||||
|
||||
|
||||
# ============ COS 鉴权(HMAC-SHA1) ============
|
||||
|
||||
def _cos_sign(
|
||||
method: str,
|
||||
path: str,
|
||||
params: dict[str, str],
|
||||
headers: dict[str, str],
|
||||
secret_id: str,
|
||||
secret_key: str,
|
||||
start_time: Optional[int] = None,
|
||||
expire_seconds: int = 3600,
|
||||
) -> str:
|
||||
"""
|
||||
构建 COS 请求签名(q-sign-algorithm=sha1 方案)。
|
||||
参考:https://cloud.tencent.com/document/product/436/7778
|
||||
|
||||
Args:
|
||||
method: HTTP 方法(小写,如 "put")
|
||||
path: URL 路径(URL encode 后的小写)
|
||||
params: URL 查询参数 dict(用于签名)
|
||||
headers: 参与签名的请求头 dict(key 需小写)
|
||||
secret_id: 临时 SecretId(tmpSecretId)
|
||||
secret_key: 临时 SecretKey(tmpSecretKey)
|
||||
start_time: 签名起始 Unix 时间戳(默认 now)
|
||||
expire_seconds: 签名有效期(秒,默认 3600)
|
||||
|
||||
Returns:
|
||||
Authorization header 值(完整字符串)
|
||||
"""
|
||||
now = int(time.time())
|
||||
q_sign_time = f"{start_time or now};{(start_time or now) + expire_seconds}"
|
||||
|
||||
# Step 1: SignKey = HMAC-SHA1(SecretKey, q-sign-time)
|
||||
sign_key = hmac.new(
|
||||
secret_key.encode("utf-8"),
|
||||
q_sign_time.encode("utf-8"),
|
||||
hashlib.sha1,
|
||||
).hexdigest()
|
||||
|
||||
# Step 2: HttpString
|
||||
# 参数和头部需按字典序排列,key 小写
|
||||
sorted_params = sorted((k.lower(), urllib.parse.quote(str(v), safe="") ) for k, v in params.items())
|
||||
sorted_headers = sorted((k.lower(), urllib.parse.quote(str(v), safe="") ) for k, v in headers.items())
|
||||
|
||||
url_param_list = ";".join(k for k, _ in sorted_params)
|
||||
url_params = "&".join(f"{k}={v}" for k, v in sorted_params)
|
||||
header_list = ";".join(k for k, _ in sorted_headers)
|
||||
header_str = "&".join(f"{k}={v}" for k, v in sorted_headers)
|
||||
|
||||
http_string = "\n".join([
|
||||
method.lower(),
|
||||
path,
|
||||
url_params,
|
||||
header_str,
|
||||
"",
|
||||
])
|
||||
|
||||
# Step 3: StringToSign = sha1 hash of HttpString
|
||||
sha1_of_http = hashlib.sha1(http_string.encode("utf-8")).hexdigest()
|
||||
string_to_sign = "\n".join([
|
||||
"sha1",
|
||||
q_sign_time,
|
||||
sha1_of_http,
|
||||
"",
|
||||
])
|
||||
|
||||
# Step 4: Signature = HMAC-SHA1(SignKey, StringToSign)
|
||||
signature = hmac.new(
|
||||
sign_key.encode("utf-8"),
|
||||
string_to_sign.encode("utf-8"),
|
||||
hashlib.sha1,
|
||||
).hexdigest()
|
||||
|
||||
return (
|
||||
f"q-sign-algorithm=sha1"
|
||||
f"&q-ak={secret_id}"
|
||||
f"&q-sign-time={q_sign_time}"
|
||||
f"&q-key-time={q_sign_time}"
|
||||
f"&q-header-list={header_list}"
|
||||
f"&q-url-param-list={url_param_list}"
|
||||
f"&q-signature={signature}"
|
||||
)
|
||||
|
||||
|
||||
# ============ 主要公开 API ============
|
||||
|
||||
async def get_cos_credentials(
|
||||
app_key: str,
|
||||
api_domain: str,
|
||||
token: str,
|
||||
filename: str = "file",
|
||||
file_id: Optional[str] = None,
|
||||
bot_id: str = "",
|
||||
route_env: str = "",
|
||||
) -> dict:
|
||||
"""
|
||||
调用 genUploadInfo 接口获取 COS 临时密钥及上传配置。
|
||||
|
||||
Args:
|
||||
app_key: 应用 Key(用于 X-ID 头)
|
||||
api_domain: API 域名(如 https://bot.yuanbao.tencent.com)
|
||||
token: 当前有效的签票 token(X-Token 头)
|
||||
filename: 待上传的文件名(含扩展名)
|
||||
file_id: 客户端生成的唯一文件 ID(不传则自动生成)
|
||||
bot_id: Bot 账号 ID(用于 X-ID 头)
|
||||
|
||||
Returns:
|
||||
COS 上传配置 dict,包含以下字段:
|
||||
bucketName (str) — COS Bucket 名称
|
||||
region (str) — COS 地域
|
||||
location (str) — 上传 Key(对象路径)
|
||||
encryptTmpSecretId (str) — 临时 SecretId
|
||||
encryptTmpSecretKey(str) — 临时 SecretKey
|
||||
encryptToken (str) — SessionToken
|
||||
startTime (int) — 凭证起始时间戳(Unix)
|
||||
expiredTime (int) — 凭证过期时间戳(Unix)
|
||||
resourceUrl (str) — 上传后的公网访问 URL
|
||||
resourceID (str) — 资源 ID(可选)
|
||||
|
||||
Raises:
|
||||
RuntimeError: 接口返回非 0 code 或字段缺失
|
||||
"""
|
||||
if file_id is None:
|
||||
file_id = generate_file_id()
|
||||
|
||||
upload_url = f"{api_domain.rstrip('/')}{UPLOAD_INFO_PATH}"
|
||||
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
"X-Token": token,
|
||||
"X-ID": bot_id or app_key,
|
||||
"X-Source": "web",
|
||||
}
|
||||
if route_env:
|
||||
headers["X-Route-Env"] = route_env
|
||||
body = {
|
||||
"fileName": filename,
|
||||
"fileId": file_id,
|
||||
"docFrom": "localDoc",
|
||||
"docOpenId": "",
|
||||
}
|
||||
|
||||
async with httpx.AsyncClient(timeout=15.0) as client:
|
||||
resp = await client.post(upload_url, json=body, headers=headers)
|
||||
resp.raise_for_status()
|
||||
result: dict[str, Any] = resp.json()
|
||||
|
||||
code = result.get("code")
|
||||
if code != 0 and code is not None:
|
||||
raise RuntimeError(
|
||||
f"genUploadInfo 失败: code={code}, msg={result.get('msg', '')}"
|
||||
)
|
||||
|
||||
data = result.get("data") or result
|
||||
required_fields = ["bucketName", "location"]
|
||||
missing = [f for f in required_fields if not data.get(f)]
|
||||
if missing:
|
||||
raise RuntimeError(
|
||||
f"genUploadInfo 返回字段不完整: 缺少字段 {missing}"
|
||||
)
|
||||
|
||||
return data
|
||||
|
||||
|
||||
async def upload_to_cos(
|
||||
file_bytes: bytes,
|
||||
filename: str,
|
||||
content_type: str,
|
||||
credentials: dict,
|
||||
bucket: str,
|
||||
region: str,
|
||||
) -> dict:
|
||||
"""
|
||||
通过 httpx PUT 请求将文件上传到 COS。
|
||||
使用临时凭证(tmpSecretId/tmpSecretKey/sessionToken)构建 HMAC-SHA1 签名。
|
||||
|
||||
Args:
|
||||
file_bytes: 文件二进制内容
|
||||
filename: 文件名(用于辅助计算 MIME、UUID)
|
||||
content_type: MIME 类型(如 "image/jpeg")
|
||||
credentials: get_cos_credentials() 返回的 dict,包含:
|
||||
encryptTmpSecretId → tmpSecretId
|
||||
encryptTmpSecretKey → tmpSecretKey
|
||||
encryptToken → sessionToken
|
||||
location → COS key(对象路径)
|
||||
resourceUrl → 上传后公网 URL
|
||||
startTime → 凭证起始时间(Unix)
|
||||
expiredTime → 凭证过期时间(Unix)
|
||||
bucket: COS Bucket 名称(如 chatbot-1234567890)
|
||||
region: COS 地域(如 ap-guangzhou)
|
||||
|
||||
Returns:
|
||||
上传结果 dict,包含:
|
||||
url (str) — COS 公网访问 URL
|
||||
uuid (str) — 文件内容 MD5
|
||||
size (int) — 文件大小(字节)
|
||||
width (int, optional) — 图片宽度(仅图片)
|
||||
height (int, optional) — 图片高度(仅图片)
|
||||
|
||||
Raises:
|
||||
httpx.HTTPStatusError: COS 返回非 2xx 状态
|
||||
RuntimeError: credentials 字段缺失
|
||||
"""
|
||||
secret_id: str = credentials.get("encryptTmpSecretId", "")
|
||||
secret_key: str = credentials.get("encryptTmpSecretKey", "")
|
||||
session_token: str = credentials.get("encryptToken", "")
|
||||
cos_key: str = credentials.get("location", "")
|
||||
resource_url: str = credentials.get("resourceUrl", "")
|
||||
start_time: Optional[int] = credentials.get("startTime")
|
||||
expired_time: Optional[int] = credentials.get("expiredTime")
|
||||
|
||||
if not secret_id or not secret_key or not cos_key:
|
||||
raise RuntimeError(
|
||||
f"COS credentials 不完整: secretId={bool(secret_id)}, "
|
||||
f"secretKey={bool(secret_key)}, location={bool(cos_key)}"
|
||||
)
|
||||
|
||||
# 构建 COS 上传 URL(优先使用全球加速域名)
|
||||
if COS_USE_ACCELERATE:
|
||||
cos_host = f"{bucket}.cos.accelerate.myqcloud.com"
|
||||
else:
|
||||
cos_host = f"{bucket}.cos.{region}.myqcloud.com"
|
||||
|
||||
# URL encode cos_key(保留 /)
|
||||
encoded_key = urllib.parse.quote(cos_key, safe="/")
|
||||
cos_url = f"https://{cos_host}/{encoded_key.lstrip('/')}"
|
||||
|
||||
# 确定 Content-Type
|
||||
if not content_type or content_type == "application/octet-stream":
|
||||
if is_image(filename):
|
||||
content_type = guess_mime_type(filename)
|
||||
else:
|
||||
content_type = "application/octet-stream"
|
||||
|
||||
# 计算文件 MD5 + size
|
||||
file_uuid = md5_hex(file_bytes)
|
||||
file_size = len(file_bytes)
|
||||
|
||||
# 参与签名的请求头
|
||||
sign_headers = {
|
||||
"host": cos_host,
|
||||
"content-type": content_type,
|
||||
"x-cos-security-token": session_token,
|
||||
}
|
||||
|
||||
# 计算签名有效期
|
||||
now = int(time.time())
|
||||
sign_start = start_time if start_time else now
|
||||
sign_expire = (expired_time - now) if expired_time and expired_time > now else 3600
|
||||
|
||||
authorization = _cos_sign(
|
||||
method="put",
|
||||
path=f"/{encoded_key.lstrip('/')}",
|
||||
params={},
|
||||
headers=sign_headers,
|
||||
secret_id=secret_id,
|
||||
secret_key=secret_key,
|
||||
start_time=sign_start,
|
||||
expire_seconds=sign_expire,
|
||||
)
|
||||
|
||||
put_headers = {
|
||||
"Authorization": authorization,
|
||||
"Content-Type": content_type,
|
||||
"x-cos-security-token": session_token,
|
||||
}
|
||||
|
||||
logger.info(
|
||||
"COS PUT: bucket=%s region=%s key=%s size=%d mime=%s",
|
||||
bucket, region, cos_key, file_size, content_type,
|
||||
)
|
||||
|
||||
async with httpx.AsyncClient(timeout=120.0) as client:
|
||||
resp = await client.put(
|
||||
cos_url,
|
||||
content=file_bytes,
|
||||
headers=put_headers,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
|
||||
# 解析图片尺寸(仅图片类型)
|
||||
result: dict[str, Any] = {
|
||||
"url": resource_url or cos_url,
|
||||
"uuid": file_uuid,
|
||||
"size": file_size,
|
||||
}
|
||||
|
||||
if content_type.startswith("image/"):
|
||||
size_info = parse_image_size(file_bytes)
|
||||
if size_info:
|
||||
result["width"] = size_info["width"]
|
||||
result["height"] = size_info["height"]
|
||||
|
||||
logger.info(
|
||||
"COS 上传成功: url=%s size=%d",
|
||||
result["url"], file_size,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
# ============ TIM 媒体消息构建 ============
|
||||
|
||||
def build_image_msg_body(
|
||||
url: str,
|
||||
uuid: Optional[str] = None,
|
||||
filename: Optional[str] = None,
|
||||
size: int = 0,
|
||||
width: int = 0,
|
||||
height: int = 0,
|
||||
mime_type: str = "",
|
||||
) -> list[dict]:
|
||||
"""
|
||||
构建腾讯 IM TIMImageElem 消息体。
|
||||
参考:https://cloud.tencent.com/document/product/269/2720
|
||||
|
||||
Args:
|
||||
url: 图片公网访问 URL(COS resourceUrl)
|
||||
uuid: 文件 UUID(MD5 或其他唯一标识)
|
||||
filename: 文件名(uuid 为空时作为备用)
|
||||
size: 文件大小(字节)
|
||||
width: 图片宽度(像素)
|
||||
height: 图片高度(像素)
|
||||
mime_type: MIME 类型(用于确定 image_format)
|
||||
|
||||
Returns:
|
||||
TIMImageElem 消息体列表(适合直接放入 msg_body)
|
||||
"""
|
||||
_uuid = uuid or filename or _basename_from_url(url) or "image"
|
||||
image_format = get_image_format(mime_type) if mime_type else 255
|
||||
|
||||
return [
|
||||
{
|
||||
"msg_type": "TIMImageElem",
|
||||
"msg_content": {
|
||||
"uuid": _uuid,
|
||||
"image_format": image_format,
|
||||
"image_info_array": [
|
||||
{
|
||||
"type": 1, # 1 = 原图
|
||||
"size": size,
|
||||
"width": width,
|
||||
"height": height,
|
||||
"url": url,
|
||||
}
|
||||
],
|
||||
},
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def build_file_msg_body(
|
||||
url: str,
|
||||
filename: str,
|
||||
uuid: Optional[str] = None,
|
||||
size: int = 0,
|
||||
) -> list[dict]:
|
||||
"""
|
||||
构建腾讯 IM TIMFileElem 消息体。
|
||||
参考:https://cloud.tencent.com/document/product/269/2720
|
||||
|
||||
Args:
|
||||
url: 文件公网访问 URL(COS resourceUrl)
|
||||
filename: 文件名(含扩展名)
|
||||
uuid: 文件 UUID(MD5 或其他唯一标识,不传则使用 filename)
|
||||
size: 文件大小(字节)
|
||||
|
||||
Returns:
|
||||
TIMFileElem 消息体列表(适合直接放入 msg_body)
|
||||
"""
|
||||
_uuid = uuid or filename
|
||||
|
||||
return [
|
||||
{
|
||||
"msg_type": "TIMFileElem",
|
||||
"msg_content": {
|
||||
"uuid": _uuid,
|
||||
"file_name": filename,
|
||||
"file_size": size,
|
||||
"url": url,
|
||||
},
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
# ============ 内部工具 ============
|
||||
|
||||
def _basename_from_url(url: str) -> str:
|
||||
"""从 URL 提取文件名。"""
|
||||
try:
|
||||
parsed = urllib.parse.urlparse(url)
|
||||
return os.path.basename(parsed.path)
|
||||
except Exception:
|
||||
return ""
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,558 @@
|
||||
"""
|
||||
Yuanbao sticker (TIMFaceElem) support.
|
||||
|
||||
Ported from yuanbao-openclaw-plugin/src/sticker/.
|
||||
|
||||
TIMFaceElem wire format:
|
||||
{
|
||||
"msg_type": "TIMFaceElem",
|
||||
"msg_content": {
|
||||
"index": 0, # always 0 per Yuanbao convention
|
||||
"data": "<json>", # serialised sticker metadata
|
||||
}
|
||||
}
|
||||
|
||||
The `data` field carries a JSON string with the sticker's metadata so the
|
||||
receiver can look up the correct asset in the emoji pack.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import random
|
||||
import re
|
||||
import unicodedata
|
||||
from typing import Optional
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Sticker catalogue – ported from builtin-stickers.json
|
||||
# Key : canonical name (Chinese)
|
||||
# Value : {sticker_id, package_id, name, description, width, height, formats}
|
||||
# ---------------------------------------------------------------------------
|
||||
STICKER_MAP: dict[str, dict] = {
|
||||
"六六六": {
|
||||
"sticker_id": "278", "package_id": "1003", "name": "六六六",
|
||||
"description": "666 厉害 牛 棒 绝了 好强 awesome",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"我想开了": {
|
||||
"sticker_id": "262", "package_id": "1003", "name": "我想开了",
|
||||
"description": "想开 佛系 释怀 顿悟 看淡了 无所谓",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"害羞": {
|
||||
"sticker_id": "130", "package_id": "1003", "name": "害羞",
|
||||
"description": "腼腆 不好意思 脸红 娇羞 羞涩 捂脸",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"比心": {
|
||||
"sticker_id": "252", "package_id": "1003", "name": "比心",
|
||||
"description": "笔芯 爱你 爱心手势 love heart 喜欢你",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"委屈": {
|
||||
"sticker_id": "125", "package_id": "1003", "name": "委屈",
|
||||
"description": "难过 想哭 可怜巴巴 瘪嘴 受伤 被欺负",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"亲亲": {
|
||||
"sticker_id": "146", "package_id": "1003", "name": "亲亲",
|
||||
"description": "么么 mua 亲一下 kiss 飞吻 啵",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"酷": {
|
||||
"sticker_id": "131", "package_id": "1003", "name": "酷",
|
||||
"description": "帅 墨镜 cool 高冷 有型 swagger",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"睡": {
|
||||
"sticker_id": "145", "package_id": "1003", "name": "睡",
|
||||
"description": "睡觉 困 zzZ 打盹 躺平 休眠 sleepy",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"发呆": {
|
||||
"sticker_id": "152", "package_id": "1003", "name": "发呆",
|
||||
"description": "懵 愣住 放空 呆滞 出神 脑子空白",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"可怜": {
|
||||
"sticker_id": "157", "package_id": "1003", "name": "可怜",
|
||||
"description": "卖萌 求饶 委屈巴巴 弱小 拜托 眼巴巴",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"摊手": {
|
||||
"sticker_id": "200", "package_id": "1003", "name": "摊手",
|
||||
"description": "无奈 没办法 耸肩 随便 那咋整 whatever",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"头大": {
|
||||
"sticker_id": "213", "package_id": "1003", "name": "头大",
|
||||
"description": "头疼 烦恼 郁闷 难搞 崩溃 一团乱",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"吓": {
|
||||
"sticker_id": "256", "package_id": "1003", "name": "吓",
|
||||
"description": "害怕 惊恐 震惊 吓一跳 恐怖 怂",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"吐血": {
|
||||
"sticker_id": "203", "package_id": "1003", "name": "吐血",
|
||||
"description": "无语 崩溃 被雷 内伤 一口老血 屮",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"哼": {
|
||||
"sticker_id": "185", "package_id": "1003", "name": "哼",
|
||||
"description": "傲娇 生气 不满 撇嘴 不理 赌气",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"嘿嘿": {
|
||||
"sticker_id": "220", "package_id": "1003", "name": "嘿嘿",
|
||||
"description": "坏笑 猥琐笑 偷笑 憨笑 得意 你懂的",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"头秃": {
|
||||
"sticker_id": "218", "package_id": "1003", "name": "头秃",
|
||||
"description": "程序员 加班 焦虑 没头发 秃了 肝爆",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"暗中观察": {
|
||||
"sticker_id": "221", "package_id": "1003", "name": "暗中观察",
|
||||
"description": "窥屏 潜水 偷偷看 角落 围观 屏住呼吸",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"我酸了": {
|
||||
"sticker_id": "224", "package_id": "1003", "name": "我酸了",
|
||||
"description": "嫉妒 柠檬精 羡慕 吃柠檬 眼红 恰柠檬",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"打call": {
|
||||
"sticker_id": "246", "package_id": "1003", "name": "打call",
|
||||
"description": "应援 加油 支持 喝彩 助威 call",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"庆祝": {
|
||||
"sticker_id": "251", "package_id": "1003", "name": "庆祝",
|
||||
"description": "祝贺 开心 耶 party 胜利 干杯",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"奋斗": {
|
||||
"sticker_id": "151", "package_id": "1003", "name": "奋斗",
|
||||
"description": "努力 加油 拼搏 冲 干劲 卷起来",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"惊讶": {
|
||||
"sticker_id": "143", "package_id": "1003", "name": "惊讶",
|
||||
"description": "震惊 哇 不敢相信 OMG 居然 这么离谱",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"疑问": {
|
||||
"sticker_id": "144", "package_id": "1003", "name": "疑问",
|
||||
"description": "问号 不懂 啥 为什么 啥情况 懵逼问",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"仔细分析": {
|
||||
"sticker_id": "248", "package_id": "1003", "name": "仔细分析",
|
||||
"description": "思考 推敲 认真 研究 琢磨 让我想想",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"撅嘴": {
|
||||
"sticker_id": "184", "package_id": "1003", "name": "撅嘴",
|
||||
"description": "嘟嘴 卖萌 不高兴 撒娇 嘴翘",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"泪奔": {
|
||||
"sticker_id": "199", "package_id": "1003", "name": "泪奔",
|
||||
"description": "大哭 伤心 破防 感动哭 泪流满面 呜呜",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"尊嘟假嘟": {
|
||||
"sticker_id": "276", "package_id": "1003", "name": "尊嘟假嘟",
|
||||
"description": "真的假的 真假 可爱问 你骗我 是不是",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"略略略": {
|
||||
"sticker_id": "113", "package_id": "1003", "name": "略略略",
|
||||
"description": "调皮 吐舌 不服 略 气死你 鬼脸",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"困": {
|
||||
"sticker_id": "180", "package_id": "1003", "name": "困",
|
||||
"description": "想睡 倦 打哈欠 睁不开眼 好困啊 sleepy",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"折磨": {
|
||||
"sticker_id": "181", "package_id": "1003", "name": "折磨",
|
||||
"description": "难受 痛苦 煎熬 蚌埠住了 受不了 要命",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"抠鼻": {
|
||||
"sticker_id": "182", "package_id": "1003", "name": "抠鼻",
|
||||
"description": "不屑 无聊 淡定 无所谓 鄙视 挖鼻",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"鼓掌": {
|
||||
"sticker_id": "183", "package_id": "1003", "name": "鼓掌",
|
||||
"description": "拍手 叫好 赞同 666 喝彩 掌声",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"斜眼笑": {
|
||||
"sticker_id": "204", "package_id": "1003", "name": "斜眼笑",
|
||||
"description": "滑稽 坏笑 doge 意味深长 阴阳怪气 嘿嘿嘿",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"辣眼睛": {
|
||||
"sticker_id": "216", "package_id": "1003", "name": "辣眼睛",
|
||||
"description": "看不下去 cringe 毁三观 太丑了 瞎了",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"哦哟": {
|
||||
"sticker_id": "217", "package_id": "1003", "name": "哦哟",
|
||||
"description": "惊讶 起哄 哇哦 有戏 不简单 哟",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"吃瓜": {
|
||||
"sticker_id": "222", "package_id": "1003", "name": "吃瓜",
|
||||
"description": "围观 看戏 八卦 路人 看热闹 板凳",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"狗头": {
|
||||
"sticker_id": "225", "package_id": "1003", "name": "狗头",
|
||||
"description": "doge 保命 开玩笑 滑稽 反讽 懂的都懂",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"敬礼": {
|
||||
"sticker_id": "227", "package_id": "1003", "name": "敬礼",
|
||||
"description": "salute 尊重 收到 遵命 致敬 报告",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"哦": {
|
||||
"sticker_id": "231", "package_id": "1003", "name": "哦",
|
||||
"description": "知道了 明白 敷衍 嗯 这样啊 收到",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"拿到红包": {
|
||||
"sticker_id": "236", "package_id": "1003", "name": "拿到红包",
|
||||
"description": "红包 谢谢老板 发财 开心 抢到了 欧气",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"牛吖": {
|
||||
"sticker_id": "239", "package_id": "1003", "name": "牛吖",
|
||||
"description": "牛 厉害 强 666 佩服 大佬",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"贴贴": {
|
||||
"sticker_id": "272", "package_id": "1003", "name": "贴贴",
|
||||
"description": "抱抱 亲昵 蹭蹭 亲密 靠靠 撒娇贴",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"爱心": {
|
||||
"sticker_id": "138", "package_id": "1003", "name": "爱心",
|
||||
"description": "心 love 喜欢你 红心 示爱 么么哒",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"晚安": {
|
||||
"sticker_id": "170", "package_id": "1003", "name": "晚安",
|
||||
"description": "好梦 睡了 night 早点休息 安啦 moon",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"太阳": {
|
||||
"sticker_id": "176", "package_id": "1003", "name": "太阳",
|
||||
"description": "晴天 早上好 阳光 morning 好天气 日",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"柠檬": {
|
||||
"sticker_id": "266", "package_id": "1003", "name": "柠檬",
|
||||
"description": "酸 嫉妒 柠檬精 羡慕 我酸 恰柠檬",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"大冤种": {
|
||||
"sticker_id": "267", "package_id": "1003", "name": "大冤种",
|
||||
"description": "倒霉 吃亏 自嘲 好心没好报 背锅 工具人",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"吐了": {
|
||||
"sticker_id": "132", "package_id": "1003", "name": "吐了",
|
||||
"description": "恶心 yue 受不了 嫌弃 想吐 生理不适",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"怒": {
|
||||
"sticker_id": "134", "package_id": "1003", "name": "怒",
|
||||
"description": "生气 愤怒 火大 暴躁 气炸 怼",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"玫瑰": {
|
||||
"sticker_id": "165", "package_id": "1003", "name": "玫瑰",
|
||||
"description": "花 示爱 表白 浪漫 送你花 情人节",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"凋谢": {
|
||||
"sticker_id": "119", "package_id": "1003", "name": "凋谢",
|
||||
"description": "花谢 失恋 难过 枯萎 心碎 凉了",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"点赞": {
|
||||
"sticker_id": "159", "package_id": "1003", "name": "点赞",
|
||||
"description": "赞 认同 好棒 good like 大拇指 顶",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"握手": {
|
||||
"sticker_id": "164", "package_id": "1003", "name": "握手",
|
||||
"description": "合作 你好 商务 hello deal 成交 友好",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"抱拳": {
|
||||
"sticker_id": "163", "package_id": "1003", "name": "抱拳",
|
||||
"description": "谢谢 失敬 江湖 承让 拜托 有礼",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"ok": {
|
||||
"sticker_id": "169", "package_id": "1003", "name": "ok",
|
||||
"description": "好的 收到 没问题 okay 行 可以 懂了",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"拳头": {
|
||||
"sticker_id": "174", "package_id": "1003", "name": "拳头",
|
||||
"description": "加油 干 冲 fight 力量 击拳 硬气",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"鞭炮": {
|
||||
"sticker_id": "191", "package_id": "1003", "name": "鞭炮",
|
||||
"description": "过年 喜庆 爆竹 春节 噼里啪啦 红",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
"烟花": {
|
||||
"sticker_id": "258", "package_id": "1003", "name": "烟花",
|
||||
"description": "庆典 漂亮 新年 嘭 绽放 节日快乐",
|
||||
"width": 128, "height": 128, "formats": "png",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def get_sticker_by_name(name: str) -> Optional[dict]:
|
||||
"""
|
||||
按名称查找贴纸,支持模糊匹配。
|
||||
|
||||
匹配优先级:
|
||||
1. 完全相等(name)
|
||||
2. name 包含查询词(前缀/子串)
|
||||
3. description 包含查询词(同义词搜索)
|
||||
4. 通用模糊评分(与 sticker-search 同算法),命中即返回得分最高的一条
|
||||
|
||||
返回 sticker dict,找不到返回 None。
|
||||
"""
|
||||
if not name:
|
||||
return None
|
||||
|
||||
query = name.strip()
|
||||
|
||||
if query in STICKER_MAP:
|
||||
return STICKER_MAP[query]
|
||||
|
||||
for key, sticker in STICKER_MAP.items():
|
||||
if query in key or key in query:
|
||||
return sticker
|
||||
|
||||
for sticker in STICKER_MAP.values():
|
||||
desc = sticker.get("description", "")
|
||||
if query in desc:
|
||||
return sticker
|
||||
|
||||
matches = search_stickers(query, limit=1)
|
||||
return matches[0] if matches else None
|
||||
|
||||
|
||||
def get_random_sticker(category: str = None) -> dict:
|
||||
"""
|
||||
随机返回一个贴纸。
|
||||
|
||||
若指定 category,则在 description 中含有该关键词的贴纸里随机选取;
|
||||
category 为 None 时从全表随机。
|
||||
"""
|
||||
if category:
|
||||
candidates = [
|
||||
s for s in STICKER_MAP.values()
|
||||
if category in s.get("description", "") or category in s.get("name", "")
|
||||
]
|
||||
if candidates:
|
||||
return random.choice(candidates)
|
||||
return random.choice(list(STICKER_MAP.values()))
|
||||
|
||||
|
||||
def get_sticker_by_id(sticker_id: str) -> Optional[dict]:
|
||||
"""按 sticker_id 精确查找贴纸。"""
|
||||
if not sticker_id:
|
||||
return None
|
||||
sid = str(sticker_id).strip()
|
||||
for sticker in STICKER_MAP.values():
|
||||
if sticker.get("sticker_id") == sid:
|
||||
return sticker
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 模糊搜索(对齐 chatbot-web yuanbao-openclaw-plugin/sticker-cache.ts.searchStickers)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_PUNCT_RE = re.compile(r"[\s\u3000\-_·.,,。!!??\"“”'‘’、/\\]+")
|
||||
|
||||
|
||||
def _normalize_text(raw: str) -> str:
|
||||
return unicodedata.normalize("NFKC", str(raw or "")).strip().lower()
|
||||
|
||||
|
||||
def _compact_text(raw: str) -> str:
|
||||
return _PUNCT_RE.sub("", _normalize_text(raw))
|
||||
|
||||
|
||||
def _multiset_char_hit_ratio(needle: str, haystack: str) -> float:
|
||||
if not needle:
|
||||
return 0.0
|
||||
bag: dict[str, int] = {}
|
||||
for ch in haystack:
|
||||
bag[ch] = bag.get(ch, 0) + 1
|
||||
hits = 0
|
||||
for ch in needle:
|
||||
n = bag.get(ch, 0)
|
||||
if n > 0:
|
||||
hits += 1
|
||||
bag[ch] = n - 1
|
||||
return hits / len(needle)
|
||||
|
||||
|
||||
def _bigram_jaccard(a: str, b: str) -> float:
|
||||
if len(a) < 2 or len(b) < 2:
|
||||
return 0.0
|
||||
A = {a[i:i + 2] for i in range(len(a) - 1)}
|
||||
B = {b[i:i + 2] for i in range(len(b) - 1)}
|
||||
inter = len(A & B)
|
||||
union = len(A) + len(B) - inter
|
||||
return inter / union if union else 0.0
|
||||
|
||||
|
||||
def _longest_subsequence_ratio(needle: str, haystack: str) -> float:
|
||||
if not needle:
|
||||
return 0.0
|
||||
j = 0
|
||||
for ch in haystack:
|
||||
if j >= len(needle):
|
||||
break
|
||||
if ch == needle[j]:
|
||||
j += 1
|
||||
return j / len(needle)
|
||||
|
||||
|
||||
def _score_field(haystack: str, query: str) -> float:
|
||||
hay = _normalize_text(haystack)
|
||||
q = _normalize_text(query)
|
||||
if not hay or not q:
|
||||
return 0.0
|
||||
hay_c = _compact_text(haystack)
|
||||
q_c = _compact_text(query)
|
||||
best = 0.0
|
||||
if hay == q:
|
||||
best = max(best, 100.0)
|
||||
if q in hay:
|
||||
best = max(best, 92 + min(6, len(q)))
|
||||
if len(q) >= 2 and hay.startswith(q):
|
||||
best = max(best, 88.0)
|
||||
if q_c and q_c in hay_c:
|
||||
best = max(best, 86.0)
|
||||
best = max(best, _multiset_char_hit_ratio(q_c, hay_c) * 62)
|
||||
best = max(best, _bigram_jaccard(q_c, hay_c) * 58)
|
||||
best = max(best, _longest_subsequence_ratio(q_c, hay_c) * 52)
|
||||
if len(q) == 1 and q in hay:
|
||||
best = max(best, 68.0)
|
||||
return best
|
||||
|
||||
|
||||
def search_stickers(query: str, limit: int = 10) -> list[dict]:
|
||||
"""
|
||||
在内置贴纸表中按模糊匹配排序返回前 N 条结果。
|
||||
|
||||
评分综合 name/description 字段的子串、字符多重集覆盖、bigram Jaccard、子序列比例。
|
||||
name 权重略高于 description(×0.88)。空 query 时按字典顺序返回前 N 条。
|
||||
"""
|
||||
safe_limit = max(1, min(500, int(limit) if limit else 10))
|
||||
if not query or not _normalize_text(query):
|
||||
return list(STICKER_MAP.values())[:safe_limit]
|
||||
|
||||
scored: list[tuple[float, dict]] = []
|
||||
for sticker in STICKER_MAP.values():
|
||||
name_s = _score_field(sticker.get("name", ""), query)
|
||||
desc_s = _score_field(sticker.get("description", ""), query) * 0.88
|
||||
sid = str(sticker.get("sticker_id", "")).strip()
|
||||
q_norm = _normalize_text(query)
|
||||
id_s = 0.0
|
||||
if sid and q_norm:
|
||||
sid_norm = _normalize_text(sid)
|
||||
if sid_norm == q_norm:
|
||||
id_s = 100.0
|
||||
elif q_norm in sid_norm:
|
||||
id_s = 84.0
|
||||
scored.append((max(name_s, desc_s, id_s), sticker))
|
||||
|
||||
scored.sort(key=lambda x: x[0], reverse=True)
|
||||
top = scored[0][0] if scored else 0
|
||||
if top <= 0:
|
||||
return [s for _, s in scored[:safe_limit]]
|
||||
|
||||
if top >= 22:
|
||||
floor = 18.0
|
||||
elif top >= 12:
|
||||
floor = max(10.0, top * 0.5)
|
||||
else:
|
||||
floor = max(6.0, top * 0.35)
|
||||
|
||||
filtered = [pair for pair in scored if pair[0] >= floor]
|
||||
out = filtered if filtered else scored
|
||||
return [s for _, s in out[:safe_limit]]
|
||||
|
||||
|
||||
def build_face_msg_body(
|
||||
face_index: int,
|
||||
face_type: int = 1,
|
||||
data: Optional[str] = None,
|
||||
) -> list:
|
||||
"""
|
||||
构造 TIMFaceElem 消息体。
|
||||
|
||||
Yuanbao 约定:
|
||||
- index 固定传 0(服务端通过 data 字段识别具体表情)
|
||||
- data 为 JSON 字符串,包含 sticker_id / package_id 等字段
|
||||
|
||||
Args:
|
||||
face_index: 保留字段,暂时不影响 wire format(Yuanbao 固定 index=0)。
|
||||
当 face_index > 0 时视为旧版 QQ 表情 ID,直接放入 index。
|
||||
face_type: 保留字段(兼容旧接口,当前未使用)。
|
||||
data: 已序列化的 JSON 字符串;为 None 时仅传 index。
|
||||
|
||||
Returns:
|
||||
符合 Yuanbao TIM 协议的 msg_body list,如::
|
||||
|
||||
[{"msg_type": "TIMFaceElem", "msg_content": {"index": 0, "data": "..."}}]
|
||||
"""
|
||||
msg_content: dict = {"index": face_index}
|
||||
if data is not None:
|
||||
msg_content["data"] = data
|
||||
return [{"msg_type": "TIMFaceElem", "msg_content": msg_content}]
|
||||
|
||||
|
||||
def build_sticker_msg_body(sticker: dict) -> list:
|
||||
"""
|
||||
从 STICKER_MAP 中的 sticker dict 直接构造 TIMFaceElem 消息体。
|
||||
|
||||
这是 send_sticker() 的内部辅助,确保 data 字段与原始 JS 插件一致。
|
||||
"""
|
||||
data_payload = json.dumps(
|
||||
{
|
||||
"sticker_id": sticker["sticker_id"],
|
||||
"package_id": sticker["package_id"],
|
||||
"width": sticker.get("width", 128),
|
||||
"height": sticker.get("height", 128),
|
||||
"formats": sticker.get("formats", "png"),
|
||||
"name": sticker["name"],
|
||||
},
|
||||
ensure_ascii=False,
|
||||
separators=(",", ":"),
|
||||
)
|
||||
return build_face_msg_body(face_index=0, data=data_payload)
|
||||
Reference in New Issue
Block a user