Compare commits
32 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| f65526cbaa | |||
| 5e6fc45785 | |||
| e570d08772 | |||
| cc90e10b19 | |||
| fc24fb699b | |||
| 380cfd565d | |||
| 2c766af477 | |||
| d7551bf19d | |||
| 3e5f9df5b8 | |||
| 0e8dd3f412 | |||
| 762824c590 | |||
| 447dc4acc4 | |||
| 373704a683 | |||
| 97e9e63f42 | |||
| 853e5190e4 | |||
| 7015b08063 | |||
| b0e010c6cc | |||
| 4ca570a39f | |||
| 9045fee67e | |||
| 5b07d95e7e | |||
| a827452881 | |||
| 781496540b | |||
| bdf692c391 | |||
| 40963eb6b7 | |||
| 6bc92a0ac7 | |||
| 5dfd29eafd | |||
| dd1af8fc17 | |||
| 7e264cbde6 | |||
| c2911cd4a2 | |||
| 84b474c79b | |||
| b6241e8e67 | |||
| 607f0d2c56 |
@@ -1,7 +0,0 @@
|
||||
{
|
||||
"permissions": {
|
||||
"allow": [
|
||||
"WebFetch(domain:community.groq.com)"
|
||||
]
|
||||
}
|
||||
}
|
||||
@@ -13,3 +13,6 @@
|
||||
|
||||
# Groq Whisper (optional, for video transcription) — https://console.groq.com
|
||||
# GROQ_API_KEY=gsk_your_key_here
|
||||
|
||||
# OpenAI Whisper (optional fallback when Groq is rate-limited) — https://platform.openai.com
|
||||
# OPENAI_API_KEY=sk-your_key_here
|
||||
|
||||
@@ -28,3 +28,43 @@ jobs:
|
||||
- name: Run tests
|
||||
run: |
|
||||
pytest -q
|
||||
|
||||
# Editable installs (-e) never exercise wheel packaging, so a broken wheel
|
||||
# can pass tests and still fail every real `pip install` from source.
|
||||
# This job builds the actual wheel and installs it into a clean venv.
|
||||
wheel-gate:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Build wheel
|
||||
run: |
|
||||
python -m pip install --upgrade pip build
|
||||
python -m build
|
||||
|
||||
- name: Verify wheel has no duplicate entries and ships data files
|
||||
run: |
|
||||
python - <<'PY'
|
||||
import glob, zipfile, collections
|
||||
whl = glob.glob("dist/*.whl")[0]
|
||||
names = zipfile.ZipFile(whl).namelist()
|
||||
dupes = [n for n, c in collections.Counter(names).items() if c > 1]
|
||||
assert not dupes, f"duplicate entries in wheel: {dupes}"
|
||||
assert "agent_reach/skill/SKILL.md" in names, "SKILL.md missing from wheel"
|
||||
for prefix in ("agent_reach/guides/", "agent_reach/scripts/", "agent_reach/skill/references/"):
|
||||
assert any(n.startswith(prefix) for n in names), f"{prefix} missing from wheel"
|
||||
print(f"wheel OK: {len(names)} entries, no duplicates, data files present")
|
||||
PY
|
||||
|
||||
- name: Smoke-install wheel into clean venv
|
||||
run: |
|
||||
python -m venv /tmp/smoke
|
||||
/tmp/smoke/bin/pip install --quiet dist/*.whl
|
||||
/tmp/smoke/bin/agent-reach version
|
||||
cd /tmp && /tmp/smoke/bin/python -c "import agent_reach; from importlib.resources import files; assert (files('agent_reach')/'skill'/'SKILL.md').is_file(); print('SKILL.md ships in site-packages OK')"
|
||||
|
||||
@@ -8,3 +8,7 @@ build/
|
||||
.env
|
||||
.agent-reach/
|
||||
*.log
|
||||
|
||||
# Claude Code personal permission settings — local only, never commit
|
||||
.claude/settings.local.json
|
||||
uv.lock
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
# CLAUDE.md
|
||||
|
||||
## Project
|
||||
Agent Reach — Python CLI + library that gives AI agents read/search access to 14+ internet platforms.
|
||||
Agent Reach — Python CLI + library that gives AI agents read/search access to 13 internet platforms.
|
||||
Positioning: installer + doctor + config tool. NOT a wrapper — after install, agents call upstream tools directly.
|
||||
Repo: github.com/Panniantong/Agent-Reach | License: MIT | Version: 1.3.0
|
||||
Repo: github.com/Panniantong/Agent-Reach | License: MIT | Version: 1.5.0
|
||||
|
||||
## Commands
|
||||
- `pip install -e .` — Dev install
|
||||
|
||||
@@ -4,9 +4,13 @@
|
||||
<strong>给你的 AI Agent 一键装上互联网能力</strong>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
当下最稳的接入方式,替你选好、装好、体检好——接入方式会换代,你不用操心
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a>
|
||||
<a href="https://www.python.org/"><img src="https://img.shields.io/badge/Python-3.10+-green.svg?style=for-the-badge&logo=python&logoColor=white" alt="Python 3.8+"></a>
|
||||
<a href="https://www.python.org/"><img src="https://img.shields.io/badge/Python-3.10+-green.svg?style=for-the-badge&logo=python&logoColor=white" alt="Python 3.10+"></a>
|
||||
<a href="https://github.com/Panniantong/agent-reach/stargazers"><img src="https://img.shields.io/github/stars/Panniantong/agent-reach?style=for-the-badge" alt="GitHub Stars"></a>
|
||||
</p>
|
||||
|
||||
@@ -24,7 +28,7 @@ AI Agent 已经能帮你写代码、改文档、管项目——但你让它去
|
||||
- 🐦 "帮我搜一下推特上大家怎么评价这个产品" → **搜不了**,Twitter API 要付费
|
||||
- 📖 "去 Reddit 上看看有没有人遇到过同样的 bug" → **403 被封**,服务器 IP 被拒
|
||||
- 📕 "帮我看看小红书上这个品的口碑" → **打不开**,必须登录才能看
|
||||
- 📺 "B站上有个技术视频,帮我总结一下" → **连不上**,海外/服务器 IP 被屏蔽
|
||||
- 📺 "B站上有个技术视频,帮我总结一下" → **拿不到**,通用下载工具被 B站风控全面拦截
|
||||
- 🔍 "帮我在网上搜一下最新的 LLM 框架对比" → **没有好用的搜索**,要么付费要么质量差
|
||||
- 🌐 "帮我看看这个网页写了啥" → **抓回来一堆 HTML 标签**,根本没法读
|
||||
- 📦 "这个 GitHub 仓库是干嘛的?Issue 里说了什么?" → 能用,但认证配置很麻烦
|
||||
@@ -56,7 +60,7 @@ AI Agent 已经能帮你写代码、改文档、管项目——但你让它去
|
||||
|---|---|
|
||||
| 💰 **完全免费** | 所有工具开源、所有 API 免费。唯一可能花钱的是服务器代理($1/月),本地电脑不需要 |
|
||||
| 🔒 **隐私安全** | Cookie 只存在你本地,不上传不外传。代码完全开源,随时可审查 |
|
||||
| 🔄 **持续更新** | 底层工具(yt-dlp、twitter-cli、rdt-cli、Jina Reader 等)定期追踪更新到最新版,你不用自己盯 |
|
||||
| 🔄 **持续换代** | 每个平台都是「首选 + 备选」多后端路由。某个接入方式失效了,我们换下一个,你无感(2026-06 实例:yt-dlp 被 B站风控封死 → 已切换 bili-cli,用户零操作) |
|
||||
| 🤖 **兼容所有 Agent** | Claude Code、OpenClaw、Cursor、Windsurf……任何能跑命令行的 Agent 都能用 |
|
||||
| 🩺 **自带诊断** | `agent-reach doctor` 一条命令告诉你哪个通、哪个不通、怎么修 |
|
||||
|
||||
@@ -71,14 +75,11 @@ AI Agent 已经能帮你写代码、改文档、管项目——但你让它去
|
||||
| 📡 **RSS** | 阅读任意 RSS/Atom 源 | — | 无需配置 |
|
||||
| 🔍 **全网搜索** | — | 全网语义搜索 | 自动配置(MCP 接入,免费无需 Key) |
|
||||
| 📦 **GitHub** | 读公开仓库 + 搜索 | 私有仓库、提 Issue/PR、Fork | 告诉 Agent「帮我登录 GitHub」 |
|
||||
| 🐦 **Twitter/X** | 读单条推文 | 搜索推文、浏览时间线、发推 | 告诉 Agent「帮我配 Twitter」 |
|
||||
| 📺 **B站** | 本地:字幕提取 + 搜索 | 服务器也能用 | 告诉 Agent「帮我配代理」 |
|
||||
| 📖 **Reddit** | 搜索 + 读帖子和评论(通过 rdt-cli) | Cookie | 需要登录认证(`rdt login`),详见 [rdt-cli](https://github.com/public-clis/rdt-cli) |
|
||||
| 📕 **小红书** | — | 阅读、搜索、发帖、评论、点赞 | 告诉 Agent「帮我配小红书」 |
|
||||
| 🎵 **抖音** | — | 视频解析、无水印下载链接获取 | 告诉 Agent「帮我配抖音」 |
|
||||
| 🐦 **Twitter/X** | 读单条推文 | 搜索推文、浏览时间线、读长文 | 告诉 Agent「帮我配 Twitter」 |
|
||||
| 📺 **B站** | 搜索 + 视频详情(bili-cli,无需登录) | 字幕(OpenCLI) | 告诉 Agent「帮我配 B站」 |
|
||||
| 📖 **Reddit** | —(没有零配置路径:匿名接口已被封) | 搜索 + 读帖子和评论 | 桌面装 OpenCLI 用浏览器登录态;或 rdt-cli + Cookie |
|
||||
| 📕 **小红书** | — | 搜索、阅读、评论 | 桌面装 OpenCLI(刷过小红书即可用);服务器用 xiaohongshu-mcp 扫码 |
|
||||
| 💼 **LinkedIn** | Jina Reader 读公开页面 | Profile 详情、公司页面、职位搜索 | 告诉 Agent「帮我配 LinkedIn」 |
|
||||
| 💬 **微信公众号** | 搜索 + 阅读公众号文章(全文 Markdown) | — | 无需配置 |
|
||||
| 📰 **微博** | 热搜、搜索内容/用户/话题、用户动态、评论 | — | 无需配置 |
|
||||
| 💻 **V2EX** | 热门帖子、节点帖子、帖子详情+回复、用户信息 | — | 无需配置 |
|
||||
| 📈 **雪球** | 股票行情、搜索股票、热门帖子、热门股票排行 | — | 告诉 Agent「帮我配雪球」 |
|
||||
| 🎙️ **小宇宙播客** | — | 播客音频转文字(Whisper 转录,免费 Key) | 告诉 Agent「帮我配小宇宙播客」 |
|
||||
@@ -126,13 +127,14 @@ AI Agent 已经能帮你写代码、改文档、管项目——但你让它去
|
||||
<details>
|
||||
<summary>它会做什么?(点击展开)</summary>
|
||||
|
||||
1. **安装 CLI 工具** — `pip install` 装好 `agent-reach` 命令行
|
||||
2. **安装系统依赖** — 自动检测并安装 Node.js、gh CLI、mcporter、twitter-cli、rdt-cli 等
|
||||
1. **安装 CLI 工具** — `pip install` 装好 `agent-reach` 命令行(自带 yt-dlp、feedparser)
|
||||
2. **安装系统基建** — 自动检测并安装 Node.js、gh CLI、mcporter
|
||||
3. **配置搜索引擎** — 通过 MCP 接入 Exa(免费,无需 API Key)
|
||||
4. **检测环境** — 判断是本地电脑还是服务器,给出对应的配置建议
|
||||
5. **注册 SKILL.md** — 在 Agent 的 skills 目录安装使用指南,以后 Agent 遇到"搜推特"、"看视频"这类需求,会自动知道该调哪个上游工具
|
||||
5. **注册 SKILL.md** — 在 Agent 的 skills 目录安装使用指南,以后 Agent 遇到"全网调研"、"搜推特"、"看视频"这类需求,会自动知道该调哪个上游工具
|
||||
6. **问你要不要更多** — 默认只激活 6 个零配置渠道;小红书、Twitter、Reddit 这些需要登录的,Agent 会列菜单问你要哪些,点名才装
|
||||
|
||||
安装完之后,`agent-reach doctor` 一条命令告诉你每个渠道的状态。
|
||||
安装完之后,`agent-reach doctor` 一条命令告诉你每个渠道的状态、当前走哪条路。
|
||||
</details>
|
||||
|
||||
---
|
||||
@@ -143,92 +145,60 @@ AI Agent 已经能帮你写代码、改文档、管项目——但你让它去
|
||||
|
||||
- "帮我看看这个链接" → `curl https://r.jina.ai/URL` 读任意网页
|
||||
- "这个 GitHub 仓库是做什么的" → `gh repo view owner/repo`
|
||||
- "这个视频讲了什么" → `yt-dlp --dump-json URL` 提取字幕
|
||||
- "帮我看看这条推文" → `twitter tweet URL`
|
||||
- "这个 YouTube 视频讲了什么" → `yt-dlp` 提取字幕
|
||||
- "B站搜一下 AI 教程" → `bili search`(无需登录)
|
||||
- "全网搜一下 LLM 框架对比" → Exa 语义搜索
|
||||
- "订阅这个 RSS" → `feedparser` 解析
|
||||
- "搜一下 GitHub 上有什么 LLM 框架" → `gh search repos "LLM framework"`
|
||||
|
||||
**不需要记命令。** Agent 读了 SKILL.md 之后自己知道该调什么。
|
||||
**不需要记命令。** Agent 读了 SKILL.md 之后自己知道该调什么。需要登录的平台(小红书、Twitter、Reddit),告诉 Agent「帮我配 XXX」即可解锁。
|
||||
|
||||
---
|
||||
|
||||
## 设计理念
|
||||
|
||||
**Agent Reach 是一个脚手架(scaffolding),不是框架。**
|
||||
**Agent Reach 是一个能力层(capability layer),不是又一个工具。**
|
||||
|
||||
你给一个新 Agent 装环境的时候,总要花时间去找工具、装依赖、调配置——Twitter 用什么读?Reddit 怎么绕封?YouTube 字幕怎么提取?每次都要重新踩一遍。
|
||||
它比任何具体实现高一层——负责**选型、安装、体检、路由**,不负责底层读取本身。读取由 Agent 直接调用上游工具完成,没有包装层。
|
||||
|
||||
Agent Reach 做的事情很简单:**帮你把这些选型和配置的活儿做完了。**
|
||||
你给一个新 Agent 装环境的时候,总要花时间去找工具、装依赖、调配置——Twitter 用什么读?Reddit 怎么登录?小红书的 CLI 停更了换什么?每次都要重新踩一遍。Agent Reach 做的事情很简单:**当下最稳的接入方式,我们替你选好、装好、体检好。接入方式会换代(2026 年 3 月一批单平台 CLI 集体停更,我们换了路由),你不用操心。**
|
||||
|
||||
安装完成后,Agent 直接调用上游工具(twitter-cli、rdt-cli、xhs-cli、yt-dlp、mcporter、gh CLI 等),不需要经过 Agent Reach 的包装层。
|
||||
### 🔌 每个平台 = 首选 + 备选的有序后端列表
|
||||
|
||||
### 🔌 每个渠道都是可插拔的
|
||||
|
||||
每个平台背后是一个独立的上游工具。**不满意?换掉就行。**
|
||||
换接入方式 = 调整列表顺序,不是重写代码。`agent-reach doctor` 会告诉你每个平台**当前在用哪个后端**。
|
||||
|
||||
```
|
||||
channels/
|
||||
├── web.py → Jina Reader ← 可以换成 Firecrawl、Crawl4AI……
|
||||
├── twitter.py → twitter-cli ← 可以换成官方 API……
|
||||
├── youtube.py → yt-dlp ← 可以换成 YouTube API、Whisper……
|
||||
├── github.py → gh CLI ← 可以换成 REST API、PyGithub……
|
||||
├── bilibili.py → yt-dlp ← 可以换成 bilibili-api……
|
||||
├── reddit.py → rdt-cli ← 搜索+阅读,需 Cookie 认证
|
||||
├── xiaohongshu.py → mcporter MCP ← 可以换成其他 XHS 工具……
|
||||
├── douyin.py → mcporter MCP ← 可以换成其他抖音工具……
|
||||
├── linkedin.py → linkedin-mcp ← 可以换成 LinkedIn API……
|
||||
├── wechat.py → Exa (+ Camoufox) ← 搜索+阅读微信公众号文章
|
||||
├── rss.py → feedparser ← 可以换成 atoma……
|
||||
├── exa_search.py → mcporter MCP ← 可以换成 Tavily、SerpAPI……
|
||||
├── web.py → Jina Reader
|
||||
├── twitter.py → twitter-cli ▸ OpenCLI ▸ bird
|
||||
├── youtube.py → yt-dlp
|
||||
├── github.py → gh CLI
|
||||
├── bilibili.py → bili-cli ▸ OpenCLI ▸ 搜索 API(yt-dlp 已被 B站风控封死,退役)
|
||||
├── reddit.py → OpenCLI ▸ rdt-cli(无零配置路径,必须登录态)
|
||||
├── xiaohongshu.py → OpenCLI ▸ xiaohongshu-mcp ▸ xhs-cli
|
||||
├── linkedin.py → linkedin-mcp ▸ Jina Reader
|
||||
├── rss.py → feedparser
|
||||
├── exa_search.py → Exa via mcporter
|
||||
└── __init__.py → 渠道注册(doctor 检测用)
|
||||
```
|
||||
|
||||
每个渠道文件只负责检测对应上游工具是否可用(`check()` 方法),给 `agent-reach doctor` 提供状态信息。实际的读取和搜索由 Agent 直接调用上游工具完成。
|
||||
每个渠道文件按序**真实探测**各候选后端(不只是看命令存不存在),第一个完整可用的当选;坏掉的会给出修复处方。实际的读取和搜索由 Agent 直接调用上游工具完成。
|
||||
|
||||
### 当前选型
|
||||
|
||||
| 场景 | 选型 | 为什么选它 |
|
||||
|------|------|-----------|
|
||||
| 读网页 | [Jina Reader](https://github.com/jina-ai/reader) | 9.8K Star,免费,不需要 API Key |
|
||||
| 读推特 | [twitter-cli](https://github.com/public-clis/twitter-cli) | 2.1K Star,Cookie 登录,搜索/读推文/时间线/长文 |
|
||||
| Reddit | [rdt-cli](https://github.com/public-clis/rdt-cli) | 304 Star,Cookie 认证,搜索+全文+评论 |
|
||||
| 视频字幕 + 搜索 | [yt-dlp](https://github.com/yt-dlp/yt-dlp) | 154K Star,YouTube + B站 + 1800 站通吃 |
|
||||
| B站增强 | [bili-cli](https://github.com/public-clis/bilibili-cli) | 590 Star,热门/排行/搜索/动态 |
|
||||
| 搜全网 | [Exa](https://exa.ai) via [mcporter](https://github.com/nicobailon/mcporter) | AI 语义搜索,MCP 接入免 Key |
|
||||
| GitHub | [gh CLI](https://cli.github.com) | 官方工具,认证后完整 API 能力 |
|
||||
| 读 RSS | [feedparser](https://github.com/kurtmckee/feedparser) | Python 生态标准选择,2.3K Star |
|
||||
| 小红书 | [xhs-cli](https://github.com/jackwener/xiaohongshu-cli) | 1.5K Star,pipx 一行安装,搜索/阅读/评论/发帖 |
|
||||
| 抖音 | [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) | MCP 服务,无需登录,视频解析 + 无水印下载 |
|
||||
| LinkedIn | [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) | ⭐1.2K,MCP 服务,浏览器自动化 |
|
||||
| 微信公众号 | [Exa](https://exa.ai)(搜索+阅读)+ [Camoufox](https://github.com/daijro/camoufox)(可选) | 零配置搜索+全文阅读,Camoufox 可选增强 |
|
||||
| 场景 | 首选 | 备选 | 为什么这么选 |
|
||||
|------|------|------|-----------|
|
||||
| 读网页 | [Jina Reader](https://github.com/jina-ai/reader) | — | 免费,不需要 API Key |
|
||||
| 读推特 | [twitter-cli](https://github.com/public-clis/twitter-cli) | [OpenCLI](https://github.com/jackwener/opencli) | 实测搜索稳定;OpenCLI 走浏览器登录态兜底 |
|
||||
| Reddit | [OpenCLI](https://github.com/jackwener/opencli)(桌面) | [rdt-cli](https://github.com/public-clis/rdt-cli) | 匿名接口已被封、官方 API 审批制——只剩登录态路线 |
|
||||
| YouTube 字幕 + 搜索 | [yt-dlp](https://github.com/yt-dlp/yt-dlp) | — | 154K Star,YouTube 仍是最佳(注意:不再用于 B站) |
|
||||
| B站 | [bili-cli](https://github.com/public-clis/bilibili-cli) | OpenCLI ▸ 搜索 API | yt-dlp 被 B站风控 412 封死(2026-06 实测),bili-cli 无登录可搜可读 |
|
||||
| 搜全网 | [Exa](https://exa.ai) via [mcporter](https://github.com/nicobailon/mcporter) | — | AI 语义搜索,MCP 接入免 Key |
|
||||
| GitHub | [gh CLI](https://cli.github.com) | — | 官方工具,认证后完整 API 能力 |
|
||||
| 读 RSS | [feedparser](https://github.com/kurtmckee/feedparser) | — | Python 生态标准选择 |
|
||||
| 小红书 | [OpenCLI](https://github.com/jackwener/opencli)(桌面) | [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp)(服务器)▸ xhs-cli | xhs-cli 作者已转投 OpenCLI(24K Star);浏览器登录态零摩擦 |
|
||||
| LinkedIn | [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) | Jina Reader | MCP 服务,浏览器自动化 |
|
||||
|
||||
> 📌 这些都是「当前选型」。不满意?换掉对应文件就行。这正是脚手架的意义。
|
||||
|
||||
### 抖音 / 小红书脚本提取的可选实现
|
||||
|
||||
如果你不只是想“解析抖音视频信息”,还想统一处理:
|
||||
|
||||
- 抖音视频脚本提取
|
||||
- 小红书视频笔记脚本提取
|
||||
- 小红书图文笔记正文 + 图片文字提取
|
||||
- 固定输出 `script.md` 和 `info.json`
|
||||
|
||||
可以把 `douyin` 这个 mcporter alias 指向另一个兼容实现:
|
||||
|
||||
- [social-post-extractor-mcp](https://github.com/JNHFlow21/social-post-extractor-mcp)
|
||||
|
||||
这个实现保留了旧工具名兼容性:
|
||||
|
||||
- `parse_douyin_video_info`
|
||||
- `get_douyin_download_link`
|
||||
- `extract_douyin_text`
|
||||
|
||||
同时新增统一工具:
|
||||
|
||||
- `parse_social_post_info`
|
||||
- `extract_social_post_script`
|
||||
|
||||
所以从 Agent Reach 的视角看,它依然只是一个 `mcporter` 里的 `douyin` server,只是能力更完整。
|
||||
> 📌 这些都是「当前选型」,基于真机实测定期复核。某条路失效了我们换下一条——`agent-reach doctor` 永远告诉你现在走的是哪条。
|
||||
|
||||
---
|
||||
|
||||
@@ -323,7 +293,7 @@ Agent Reach uses twitter-cli with cookie auth — zero API fees. Install with `p
|
||||
<details>
|
||||
<summary><strong>Reddit 返回 403 怎么办?</strong></summary>
|
||||
|
||||
Agent Reach 使用 [rdt-cli](https://github.com/public-clis/rdt-cli) 访问 Reddit。Reddit 自 2024 年起要求认证,安装后需运行 `rdt login` 登录。安装:`pipx install rdt-cli`,然后 `rdt login`(自动从浏览器提取 Cookie)。之后 Agent 可以用 `rdt search "关键词"` 搜索、`rdt read POST_ID` 读帖子全文和评论。
|
||||
Reddit 所有访问都需要登录态(匿名接口已被全面封锁,官方 API 需人工审批)。桌面首选 **OpenCLI**:浏览器里登录过 reddit.com 即可直接 `opencli reddit search "关键词"`。备选 [rdt-cli](https://github.com/public-clis/rdt-cli):`pipx install 'git+https://github.com/public-clis/rdt-cli.git@5e4fb3720d5c174e976cd425ccc3b879d52cac66'`(与代码同一钉定版本,PyPI 落后),然后 `rdt login`。中国大陆网络访问 Reddit 需要代理。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
@@ -335,13 +305,7 @@ Agent Reach 使用 [rdt-cli](https://github.com/public-clis/rdt-cli) 访问 Redd
|
||||
<details>
|
||||
<summary><strong>怎么让 AI Agent 读小红书?</strong></summary>
|
||||
|
||||
安装 `pipx install xiaohongshu-cli`,然后 `xhs login`(自动从浏览器提取 Cookie)。之后 Agent 就能用 `xhs search "关键词"` 搜索笔记、`xhs read NOTE_ID` 阅读详情、`xhs comments NOTE_ID` 查看评论了。不需要 Docker。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>怎么让 AI Agent 解析抖音视频?</strong></summary>
|
||||
|
||||
安装 douyin-mcp-server 后,Agent 就能用 `mcporter call 'douyin.parse_douyin_video_info(share_link: "分享链接")'` 解析视频信息、获取无水印下载链接。不需要登录,把抖音分享链接发给 Agent 就行。详见 https://github.com/yzfly/douyin-mcp-server
|
||||
桌面电脑首选 **OpenCLI**(`agent-reach install --channels opencli`)——它复用你浏览器里的登录态,平时刷过小红书就直接能用,零配置;装完在 Chrome 商店点一次「添加扩展」即可。之后 Agent 用 `opencli xiaohongshu search "关键词"` 搜索、`opencli xiaohongshu note URL` 读笔记。服务器上用 [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp)(自带无头浏览器,扫码登录)。已装过 xhs-cli 的老用户不受影响,它仍是备选后端(上游 2026-03 起停更,不推荐新装)。
|
||||
</details>
|
||||
|
||||
<details>
|
||||
@@ -355,14 +319,14 @@ Yes! Agent Reach is an installer + configuration tool — any AI coding agent th
|
||||
<details>
|
||||
<summary><strong>Is this free? Any API costs?</strong></summary>
|
||||
|
||||
100% free. All backends are open-source tools (twitter-cli, rdt-cli, xhs-cli, yt-dlp, Jina Reader, Exa, etc.) that don't require paid API keys. The only optional cost is a residential proxy (~$1/month) if you need Bilibili access from an overseas server.
|
||||
100% free. All backends are open-source tools (OpenCLI, twitter-cli, bili-cli, rdt-cli, yt-dlp, Jina Reader, Exa, xiaohongshu-mcp, etc.) that don't require paid API keys. The only optional cost is a residential proxy (~$1/month) if your network blocks Reddit/Twitter (e.g. mainland China).
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## 致谢
|
||||
|
||||
[twitter-cli](https://github.com/public-clis/twitter-cli) · [rdt-cli](https://github.com/public-clis/rdt-cli) · [xhs-cli](https://github.com/jackwener/xiaohongshu-cli) · [bili-cli](https://github.com/public-clis/bilibili-cli) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [Jina Reader](https://github.com/jina-ai/reader) · [Exa](https://exa.ai) · [mcporter](https://github.com/nicobailon/mcporter) · [feedparser](https://github.com/kurtmckee/feedparser) · [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server)
|
||||
[OpenCLI](https://github.com/jackwener/opencli) · [twitter-cli](https://github.com/public-clis/twitter-cli) · [rdt-cli](https://github.com/public-clis/rdt-cli) · [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) · [xhs-cli](https://github.com/jackwener/xiaohongshu-cli) · [bili-cli](https://github.com/public-clis/bilibili-cli) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [Jina Reader](https://github.com/jina-ai/reader) · [Exa](https://exa.ai) · [mcporter](https://github.com/nicobailon/mcporter) · [feedparser](https://github.com/kurtmckee/feedparser) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server)
|
||||
|
||||
## 联系
|
||||
|
||||
@@ -383,10 +347,6 @@ Yes! Agent Reach is an installer + configuration tool — any AI coding agent th
|
||||
|
||||
## 友情链接
|
||||
|
||||
[FluxNode](https://fluxnode.org) — 低价 AI API 中转站,官方一折,可按量或按套餐付费。可用于 OpenClaw、Claude Code 等一切 Agent。
|
||||
|
||||
[OpenClaw for Enterprise](https://github.com/littleben/openclaw-for-enterprise) — 企业级 OpenClaw 多用户部署方案,飞书里直接用 AI,容器隔离,一条命令管理。
|
||||
|
||||
[腾讯云 OpenClaw](https://www.tencentcloud.com/act/pro/intl-openclaw?referral_code=G76Y819A&lang=zh&pg=) — 在腾讯云Lighthouse秒级部署OpenClaw全能助手,可通过对话丝滑接入Agent Reach,给你的OpenClaw一键装上互联网能力。
|
||||
|
||||
## Star History
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Agent Reach — Give your AI Agent eyes to see the entire internet."""
|
||||
|
||||
__version__ = "1.4.0"
|
||||
__version__ = "1.5.0"
|
||||
__author__ = "Neo Reid"
|
||||
|
||||
from agent_reach.core import AgentReach
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Cross-channel backends.
|
||||
|
||||
A backend here is an upstream runtime that serves MULTIPLE channels
|
||||
(e.g. OpenCLI covers xiaohongshu/reddit/bilibili/twitter through one
|
||||
browser session), as opposed to the per-platform tools probed inside
|
||||
each channel file.
|
||||
"""
|
||||
|
||||
from .opencli import ( # noqa: F401
|
||||
OPENCLI_EXTENSION_URL,
|
||||
OPENCLI_PACKAGE,
|
||||
OpenCLIStatus,
|
||||
opencli_status,
|
||||
opencli_summary,
|
||||
)
|
||||
@@ -0,0 +1,136 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""OpenCLI backend probing.
|
||||
|
||||
OpenCLI (github.com/jackwener/opencli) drives the user's real Chrome via a
|
||||
browser-bridge extension + local daemon, reusing existing login sessions —
|
||||
zero per-platform configuration, desktop-only (no headless).
|
||||
|
||||
Probing notes (verified live):
|
||||
- `opencli doctor` AUTO-STARTS the daemon — a side effect, so health
|
||||
checks must use `opencli daemon status` (pure query) instead.
|
||||
- Exit codes are always 0; status must be parsed from text output.
|
||||
- "Extension: disconnected" does NOT mean unusable: the extension's
|
||||
service worker sleeps and any real opencli command wakes it up
|
||||
(verified: status flips disconnected→connected after one call).
|
||||
Since daemon status can't tell "sleeping" from "never installed",
|
||||
we check Chrome's Extensions directory on disk to disambiguate.
|
||||
"""
|
||||
|
||||
import glob
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
OPENCLI_PACKAGE = "@jackwener/opencli"
|
||||
OPENCLI_EXTENSION_ID = "ildkmabpimmkaediidaifkhjpohdnifk"
|
||||
OPENCLI_EXTENSION_URL = (
|
||||
f"https://chromewebstore.google.com/detail/opencli/{OPENCLI_EXTENSION_ID}"
|
||||
)
|
||||
|
||||
#: Chrome-family profile roots that contain <Profile>/Extensions/<id>/
|
||||
_CHROME_PROFILE_ROOTS = (
|
||||
"~/Library/Application Support/Google/Chrome", # macOS Chrome
|
||||
"~/Library/Application Support/Chromium", # macOS Chromium
|
||||
"~/.config/google-chrome", # Linux Chrome
|
||||
"~/.config/chromium", # Linux Chromium
|
||||
)
|
||||
|
||||
|
||||
def _extension_installed_on_disk() -> bool:
|
||||
"""True if the OpenCLI extension exists in any Chrome profile.
|
||||
|
||||
Store-installed extensions always live under
|
||||
<profile>/Extensions/<extension id>/ — this disambiguates a sleeping
|
||||
service worker from a never-installed extension. Dev installs via
|
||||
"Load unpacked" are not covered (those users can read `opencli doctor`).
|
||||
"""
|
||||
roots = [os.path.expanduser(p) for p in _CHROME_PROFILE_ROOTS]
|
||||
local_app_data = os.environ.get("LOCALAPPDATA")
|
||||
if local_app_data: # Windows
|
||||
roots.append(os.path.join(local_app_data, "Google", "Chrome", "User Data"))
|
||||
for root in roots:
|
||||
if glob.glob(os.path.join(root, "*", "Extensions", OPENCLI_EXTENSION_ID)):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
@dataclass
|
||||
class OpenCLIStatus:
|
||||
installed: bool = False
|
||||
broken: bool = False
|
||||
daemon_running: bool = False
|
||||
extension_connected: bool = False
|
||||
extension_installed: bool = False
|
||||
version: str = ""
|
||||
hint: str = ""
|
||||
|
||||
@property
|
||||
def ready(self) -> bool:
|
||||
"""Usable now or on first call.
|
||||
|
||||
A live connection counts, and so does an installed-but-sleeping
|
||||
extension: its service worker wakes on the first real command.
|
||||
"""
|
||||
return self.installed and not self.broken and (
|
||||
self.extension_connected or self.extension_installed
|
||||
)
|
||||
|
||||
|
||||
def opencli_status(timeout: int = 10) -> OpenCLIStatus:
|
||||
"""Probe OpenCLI install + daemon/extension state without side effects."""
|
||||
version_probe = probe_command(
|
||||
"opencli", ["--version"], timeout=timeout, package=OPENCLI_PACKAGE
|
||||
)
|
||||
if version_probe.status == "missing":
|
||||
return OpenCLIStatus(installed=False)
|
||||
if not version_probe.ok:
|
||||
return OpenCLIStatus(
|
||||
installed=True,
|
||||
broken=True,
|
||||
hint=(
|
||||
"opencli 命令存在但无法执行(node 环境损坏),重装:\n"
|
||||
f" npm install -g {OPENCLI_PACKAGE}"
|
||||
),
|
||||
)
|
||||
|
||||
st = OpenCLIStatus(installed=True, version=version_probe.output.strip())
|
||||
|
||||
daemon_probe = probe_command(
|
||||
"opencli", ["daemon", "status"], timeout=timeout, package=OPENCLI_PACKAGE
|
||||
)
|
||||
output = daemon_probe.output if daemon_probe.ok else ""
|
||||
# `opencli daemon status` prints lines like:
|
||||
# Daemon: running (PID 37389) / Daemon: not running
|
||||
# Extension: connected / Extension: disconnected
|
||||
for line in output.splitlines():
|
||||
line = line.strip().lower()
|
||||
if line.startswith("daemon:"):
|
||||
st.daemon_running = "not running" not in line and "running" in line
|
||||
elif line.startswith("extension:"):
|
||||
st.extension_connected = "disconnected" not in line and "connected" in line
|
||||
|
||||
if not st.extension_connected:
|
||||
st.extension_installed = _extension_installed_on_disk()
|
||||
if not st.extension_installed:
|
||||
st.hint = (
|
||||
"OpenCLI 已安装,但 Chrome 扩展未安装。\n"
|
||||
f" 1. 安装扩展(需手动点一次):{OPENCLI_EXTENSION_URL}\n"
|
||||
" 2. 保持 Chrome 打开,运行 `opencli doctor` 验证"
|
||||
)
|
||||
return st
|
||||
|
||||
|
||||
def opencli_summary(st: OpenCLIStatus) -> str:
|
||||
"""One-line state description for channel messages / install output."""
|
||||
if not st.installed:
|
||||
return "OpenCLI 未安装"
|
||||
if st.broken:
|
||||
return "OpenCLI 无法执行(node 环境损坏)"
|
||||
if st.extension_connected:
|
||||
return f"OpenCLI 可用(浏览器登录态,v{st.version})"
|
||||
if st.ready:
|
||||
return "OpenCLI 可用(扩展睡眠中,调用时自动唤醒)"
|
||||
if st.daemon_running:
|
||||
return "OpenCLI 已安装,等待 Chrome 扩展安装"
|
||||
return "OpenCLI 已安装(daemon 未运行,使用时自动启动;需 Chrome 扩展)"
|
||||
@@ -16,10 +16,7 @@ from .rss import RSSChannel
|
||||
from .bilibili import BilibiliChannel
|
||||
from .exa_search import ExaSearchChannel
|
||||
from .xiaohongshu import XiaoHongShuChannel
|
||||
from .douyin import DouyinChannel
|
||||
from .linkedin import LinkedInChannel
|
||||
from .wechat import WeChatChannel
|
||||
from .weibo import WeiboChannel
|
||||
from .xiaoyuzhou import XiaoyuzhouChannel
|
||||
from .v2ex import V2EXChannel
|
||||
from .xueqiu import XueqiuChannel
|
||||
@@ -32,10 +29,7 @@ ALL_CHANNELS: List[Channel] = [
|
||||
RedditChannel(),
|
||||
BilibiliChannel(),
|
||||
XiaoHongShuChannel(),
|
||||
DouyinChannel(),
|
||||
LinkedInChannel(),
|
||||
WeChatChannel(),
|
||||
WeiboChannel(),
|
||||
XiaoyuzhouChannel(),
|
||||
V2EXChannel(),
|
||||
XueqiuChannel(),
|
||||
|
||||
@@ -8,11 +8,22 @@ and provides:
|
||||
- check(config) → is the upstream tool installed and configured?
|
||||
|
||||
After installation, agents call upstream tools directly.
|
||||
|
||||
Backend routing semantics:
|
||||
- `backends` is an ORDERED candidate list: backends[0] is the preferred
|
||||
backend, the rest are fallbacks. "Switching backends" for a platform
|
||||
means reordering this list (or a user override) — not rewriting code.
|
||||
- check() must set `self.active_backend` to the backend that is actually
|
||||
serving the channel right now (None when nothing usable is found).
|
||||
shutil.which() alone is NOT proof of health — a stale venv shim passes
|
||||
which() but cannot execute (see agent_reach.probe). Channels should
|
||||
really execute a lightweight command before claiming a backend active.
|
||||
- Users can force a backend with config key `<channel>_backend`
|
||||
(or env var `<CHANNEL>_BACKEND`); ordered_backends() applies it.
|
||||
"""
|
||||
|
||||
import shutil
|
||||
from abc import ABC, abstractmethod
|
||||
from typing import List, Tuple
|
||||
from typing import List, Optional, Tuple
|
||||
|
||||
|
||||
class Channel(ABC):
|
||||
@@ -20,17 +31,40 @@ class Channel(ABC):
|
||||
|
||||
name: str = "" # e.g. "youtube"
|
||||
description: str = "" # e.g. "YouTube 视频和字幕"
|
||||
backends: List[str] = [] # e.g. ["yt-dlp"] — what upstream tool is used
|
||||
backends: List[str] = [] # ordered candidates — backends[0] = preferred
|
||||
tier: int = 0 # 0=zero-config, 1=needs free key, 2=needs setup
|
||||
|
||||
#: Backend currently serving this channel; set by check(), None = unavailable.
|
||||
active_backend: Optional[str] = None
|
||||
|
||||
@abstractmethod
|
||||
def can_handle(self, url: str) -> bool:
|
||||
"""Check if this channel can handle this URL."""
|
||||
...
|
||||
|
||||
def ordered_backends(self, config=None) -> List[str]:
|
||||
"""Candidate backends in probe order, honoring the user override.
|
||||
|
||||
The config key `<channel>_backend` (env `<CHANNEL>_BACKEND`) moves the
|
||||
named backend to the front of the list; unknown values are ignored so
|
||||
a stale override can never hide working backends.
|
||||
"""
|
||||
candidates = list(self.backends)
|
||||
override = config.get(f"{self.name}_backend") if config else None
|
||||
if override:
|
||||
for i, b in enumerate(candidates):
|
||||
if b == override or b.startswith(override):
|
||||
candidates.insert(0, candidates.pop(i))
|
||||
break
|
||||
return candidates
|
||||
|
||||
def check(self, config=None) -> Tuple[str, str]:
|
||||
"""
|
||||
Check if this channel's upstream tool is available.
|
||||
Returns (status, message) where status is 'ok'/'warn'/'off'/'error'.
|
||||
|
||||
Subclasses with external backends must really probe them (see
|
||||
agent_reach.probe.probe_command) and set self.active_backend.
|
||||
"""
|
||||
self.active_backend = self.backends[0] if self.backends else "内置"
|
||||
return "ok", f"{'、'.join(self.backends) if self.backends else '内置'}"
|
||||
|
||||
@@ -1,11 +1,19 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Bilibili — video via yt-dlp, search/browse via bili-cli or API."""
|
||||
"""Bilibili — multi-backend: bili-cli / OpenCLI / search API.
|
||||
|
||||
yt-dlp was REMOVED from this channel (live-verified 2026-06): bilibili's
|
||||
risk control 412-blocks yt-dlp's requests in every configuration we
|
||||
tried — latest version, direct, proxied, with warmed cookies — while
|
||||
bili-cli keeps working (search/hot/video detail without login) and
|
||||
OpenCLI covers subtitles through the browser session. yt-dlp remains the
|
||||
YouTube backend; it just no longer serves bilibili.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import urllib.request
|
||||
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
from .base import Channel
|
||||
|
||||
_UA = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36"
|
||||
@@ -27,7 +35,7 @@ def _search_api_ok() -> bool:
|
||||
class BilibiliChannel(Channel):
|
||||
name = "bilibili"
|
||||
description = "B站视频、字幕和搜索"
|
||||
backends = ["yt-dlp", "bili-cli (可选)", "B站搜索 API"]
|
||||
backends = ["bili-cli", "OpenCLI", "B站搜索 API"]
|
||||
tier = 1
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
@@ -36,31 +44,76 @@ class BilibiliChannel(Channel):
|
||||
return "bilibili.com" in d or "b23.tv" in d
|
||||
|
||||
def check(self, config=None):
|
||||
if not shutil.which("yt-dlp"):
|
||||
return "off", "yt-dlp 未安装。安装:pip install yt-dlp"
|
||||
"""Probe candidates in order; first fully-usable backend wins."""
|
||||
self.active_backend = None
|
||||
findings = []
|
||||
|
||||
proxy = (config.get("bilibili_proxy") if config else None) or os.environ.get("BILIBILI_PROXY")
|
||||
has_bili_cli = bool(shutil.which("bili"))
|
||||
|
||||
parts = []
|
||||
|
||||
# 视频读取状态
|
||||
if proxy:
|
||||
parts.append("视频读取:yt-dlp(代理已配置)")
|
||||
else:
|
||||
parts.append("视频读取:yt-dlp")
|
||||
|
||||
# bili-cli 增强
|
||||
if has_bili_cli:
|
||||
parts.append("搜索/热门/排行:bili-cli 可用")
|
||||
else:
|
||||
# 检测搜索 API 连通性
|
||||
api_ok = _search_api_ok()
|
||||
if api_ok:
|
||||
parts.append("搜索:B站 API 可用")
|
||||
for backend in self.ordered_backends(config):
|
||||
if backend == "bili-cli":
|
||||
result = self._check_bili_cli()
|
||||
elif backend == "OpenCLI":
|
||||
result = self._check_opencli()
|
||||
else:
|
||||
parts.append("搜索:B站 API 不可达")
|
||||
parts.append("提示:安装 bili-cli 可解锁热门/排行/动态:pipx install bilibili-cli")
|
||||
result = self._check_search_api()
|
||||
if result is None:
|
||||
continue
|
||||
findings.append((backend, *result))
|
||||
|
||||
status = "ok" if has_bili_cli or _search_api_ok() else "warn"
|
||||
return status, "。".join(parts)
|
||||
# 有后端断链时,即使别的候选兜底成功也要把处方带出来
|
||||
broken_notes = [m for _, s, m in findings if s == "error"]
|
||||
|
||||
for wanted in ("ok", "warn"):
|
||||
for backend, status, message in findings:
|
||||
if status == wanted:
|
||||
self.active_backend = backend
|
||||
if broken_notes:
|
||||
message += "\n[备选后端异常] " + ";".join(broken_notes)
|
||||
return status, message
|
||||
|
||||
if findings:
|
||||
return "error", "\n".join(m for _, _, m in findings)
|
||||
|
||||
return "off", (
|
||||
"没有可用的 B站后端(搜索 API 也不可达,可能是网络问题)。推荐:\n"
|
||||
" pipx install bilibili-cli(搜索/热门/视频详情,无需登录)\n"
|
||||
" 或桌面装 OpenCLI(额外解锁字幕):agent-reach install --channels opencli"
|
||||
)
|
||||
|
||||
def _check_bili_cli(self):
|
||||
"""bili-cli candidate. None = not installed."""
|
||||
probe = probe_command("bili", ["--version"], timeout=10, package="bilibili-cli")
|
||||
if probe.status == "missing":
|
||||
return None
|
||||
if probe.status == "broken":
|
||||
return "error", "bili 命令存在但无法执行\n" + probe.hint
|
||||
if not probe.ok:
|
||||
return "warn", f"bili-cli 探测失败({probe.status}),运行 `bili status` 查看详情"
|
||||
return "ok", (
|
||||
"bili-cli 可用(搜索/热门/排行/视频详情/音频,无需登录;"
|
||||
"字幕需 OpenCLI。上游 2026-03 起停更)"
|
||||
)
|
||||
|
||||
def _check_opencli(self):
|
||||
"""OpenCLI candidate. None = not installed."""
|
||||
from agent_reach.backends import opencli_status
|
||||
|
||||
st = opencli_status()
|
||||
if not st.installed:
|
||||
return None
|
||||
if st.broken:
|
||||
return "error", st.hint
|
||||
if st.ready:
|
||||
return "ok", (
|
||||
"OpenCLI 可用(复用浏览器登录态)。用法:"
|
||||
"opencli bilibili search/video/subtitle/ranking -f yaml"
|
||||
)
|
||||
return "warn", st.hint
|
||||
|
||||
def _check_search_api(self):
|
||||
"""Zero-dependency search API fallback. None = unreachable."""
|
||||
if not _search_api_ok():
|
||||
return None
|
||||
return "ok", (
|
||||
"B站搜索 API 可达(仅搜索,curl 直连)。"
|
||||
"完整功能建议安装 bili-cli:pipx install bilibili-cli"
|
||||
)
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Douyin (抖音) — check if mcporter + douyin-mcp-server is available."""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from .base import Channel
|
||||
|
||||
|
||||
class DouyinChannel(Channel):
|
||||
name = "douyin"
|
||||
description = "抖音短视频"
|
||||
backends = ["douyin-mcp-server"]
|
||||
tier = 2
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
d = urlparse(url).netloc.lower()
|
||||
return "douyin.com" in d or "iesdouyin.com" in d
|
||||
|
||||
def check(self, config=None):
|
||||
mcporter = shutil.which("mcporter")
|
||||
if not mcporter:
|
||||
return "off", (
|
||||
"需要 mcporter + douyin-mcp-server。安装步骤:\n"
|
||||
" 1. npm install -g mcporter\n"
|
||||
" 2. pip install douyin-mcp-server\n"
|
||||
" 3. 启动服务(见下方说明)\n"
|
||||
" 4. mcporter config add douyin http://localhost:18070/mcp\n"
|
||||
" 详见 https://github.com/yzfly/douyin-mcp-server"
|
||||
)
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[mcporter, "config", "list"], capture_output=True,
|
||||
encoding="utf-8", errors="replace", timeout=5
|
||||
)
|
||||
if "douyin" not in r.stdout:
|
||||
return "off", (
|
||||
"mcporter 已装但抖音 MCP 未配置。运行:\n"
|
||||
" pip install douyin-mcp-server\n"
|
||||
" # 启动服务后:\n"
|
||||
" mcporter config add douyin http://localhost:18070/mcp"
|
||||
)
|
||||
except Exception:
|
||||
return "off", "mcporter 连接异常"
|
||||
# Verify MCP connectivity by listing available tools instead of
|
||||
# calling with a hardcoded (invalid) share link that always fails.
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[mcporter, "list", "douyin"],
|
||||
capture_output=True, encoding="utf-8", errors="replace", timeout=15
|
||||
)
|
||||
if r.returncode == 0 and r.stdout.strip():
|
||||
return "ok", "完整可用(视频解析、下载链接获取)"
|
||||
return "warn", "MCP 已连接但工具列表为空,检查 douyin-mcp-server 服务是否在运行"
|
||||
except Exception:
|
||||
return "warn", "MCP 连接异常,检查 douyin-mcp-server 服务是否在运行"
|
||||
@@ -1,10 +1,13 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Exa Search — check if mcporter + Exa MCP is available."""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
from .base import Channel
|
||||
|
||||
#: mcporter 是 npm 包,断链处方与默认的 pipx/uv 不同
|
||||
_MCPORTER_BROKEN_HINT = "mcporter 无法执行(node 环境损坏),重装:\n npm install -g mcporter"
|
||||
|
||||
|
||||
class ExaSearchChannel(Channel):
|
||||
name = "exa_search"
|
||||
@@ -16,23 +19,22 @@ class ExaSearchChannel(Channel):
|
||||
return False # Search-only channel
|
||||
|
||||
def check(self, config=None):
|
||||
mcporter = shutil.which("mcporter")
|
||||
if not mcporter:
|
||||
self.active_backend = None
|
||||
probe = probe_command("mcporter", ["config", "list"], timeout=10, package="mcporter")
|
||||
if probe.status == "missing":
|
||||
return "off", (
|
||||
"需要 mcporter + Exa MCP。安装:\n"
|
||||
" npm install -g mcporter\n"
|
||||
" mcporter config add exa https://mcp.exa.ai/mcp"
|
||||
)
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[mcporter, "config", "list"], capture_output=True,
|
||||
encoding="utf-8", errors="replace", timeout=5
|
||||
)
|
||||
if "exa" in r.stdout.lower():
|
||||
return "ok", "全网语义搜索可用(免费,无需 API Key)"
|
||||
return "off", (
|
||||
"mcporter 已装但 Exa 未配置。运行:\n"
|
||||
" mcporter config add exa https://mcp.exa.ai/mcp"
|
||||
)
|
||||
except Exception:
|
||||
return "off", "mcporter 连接异常"
|
||||
if probe.status == "broken":
|
||||
return "error", _MCPORTER_BROKEN_HINT
|
||||
if not probe.ok: # timeout / error
|
||||
return "error", f"mcporter 执行异常:{probe.hint or probe.output or probe.status}"
|
||||
if "exa" in probe.output.lower():
|
||||
self.active_backend = self.backends[0]
|
||||
return "ok", "全网语义搜索可用(免费,无需 API Key)"
|
||||
return "off", (
|
||||
"mcporter 已装但 Exa 未配置。运行:\n"
|
||||
" mcporter config add exa https://mcp.exa.ai/mcp"
|
||||
)
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""GitHub — check if gh CLI is available."""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
from .base import Channel
|
||||
|
||||
|
||||
@@ -17,16 +17,26 @@ class GitHubChannel(Channel):
|
||||
return "github.com" in urlparse(url).netloc.lower()
|
||||
|
||||
def check(self, config=None):
|
||||
gh = shutil.which("gh")
|
||||
if not gh:
|
||||
# 真跑 gh auth status 探活。注意:未登录时 rc!=0 是正常业务态(warn),不是 error。
|
||||
probe = probe_command("gh", ["auth", "status"], timeout=10, package="gh")
|
||||
if probe.status == "missing":
|
||||
self.active_backend = None
|
||||
return "warn", "gh CLI 未安装。安装:https://cli.github.com"
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[gh, "auth", "status"],
|
||||
capture_output=True, encoding="utf-8", errors="replace", timeout=5
|
||||
if probe.status == "broken":
|
||||
# gh 是二进制安装(brew/官方包),不是 pip 包——处方不用 pipx/uv 文案
|
||||
self.active_backend = None
|
||||
return "error", (
|
||||
"gh 命令存在但无法执行——安装已损坏。重装即可修复:\n"
|
||||
" brew reinstall gh\n"
|
||||
"或从 https://cli.github.com 重新安装 gh CLI"
|
||||
)
|
||||
if r.returncode == 0:
|
||||
return "ok", "完整可用(读取、搜索、Fork、Issue、PR 等)"
|
||||
return "warn", "gh CLI 已安装但未认证。运行 gh auth login 可解锁完整功能"
|
||||
except Exception:
|
||||
return "warn", "gh CLI 状态检查失败,运行 gh auth status 查看详情"
|
||||
if probe.status == "timeout":
|
||||
# gh 本体能启动(工具是活的),只是状态检查超时
|
||||
self.active_backend = "gh CLI"
|
||||
return "warn", "gh CLI 状态检查超时,运行 gh auth status 查看详情"
|
||||
if probe.ok:
|
||||
self.active_backend = "gh CLI"
|
||||
return "ok", "完整可用(读取、搜索、Fork、Issue、PR 等)"
|
||||
# rc != 0:gh 活着但未认证(gh auth status 的正常业务态)
|
||||
self.active_backend = "gh CLI"
|
||||
return "warn", "gh CLI 已安装但未认证。运行 gh auth login 可解锁完整功能"
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""LinkedIn — check if linkedin-scraper-mcp is available."""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
from .base import Channel
|
||||
|
||||
#: mcporter 是 npm 包,断链处方与默认的 pipx/uv 不同
|
||||
_MCPORTER_BROKEN_HINT = "mcporter 无法执行(node 环境损坏),重装:\n npm install -g mcporter"
|
||||
|
||||
|
||||
class LinkedInChannel(Channel):
|
||||
name = "linkedin"
|
||||
@@ -17,23 +20,22 @@ class LinkedInChannel(Channel):
|
||||
return "linkedin.com" in urlparse(url).netloc.lower()
|
||||
|
||||
def check(self, config=None):
|
||||
mcporter = shutil.which("mcporter")
|
||||
if not mcporter:
|
||||
self.active_backend = None
|
||||
probe = probe_command("mcporter", ["config", "list"], timeout=10, package="mcporter")
|
||||
if probe.status == "missing":
|
||||
return "off", (
|
||||
"基本内容可通过 Jina Reader 读取。完整功能需要:\n"
|
||||
" pip install linkedin-scraper-mcp\n"
|
||||
" mcporter config add linkedin http://localhost:3000/mcp\n"
|
||||
" 详见 https://github.com/stickerdaniel/linkedin-mcp-server"
|
||||
)
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[mcporter, "config", "list"], capture_output=True,
|
||||
encoding="utf-8", errors="replace", timeout=5
|
||||
)
|
||||
if "linkedin" in r.stdout.lower():
|
||||
return "ok", "完整可用(Profile、公司、职位搜索)"
|
||||
except Exception:
|
||||
pass
|
||||
if probe.status == "broken":
|
||||
return "error", _MCPORTER_BROKEN_HINT
|
||||
if not probe.ok: # timeout / error
|
||||
return "error", f"mcporter 执行异常:{probe.hint or probe.output or probe.status}"
|
||||
if "linkedin" in probe.output.lower():
|
||||
self.active_backend = "linkedin-scraper-mcp"
|
||||
return "ok", "完整可用(Profile、公司、职位搜索)"
|
||||
return "off", (
|
||||
"mcporter 已装但 LinkedIn MCP 未配置。运行:\n"
|
||||
" pip install linkedin-scraper-mcp\n"
|
||||
|
||||
+126
-37
@@ -1,25 +1,42 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Reddit — search and read via rdt-cli (public-clis/rdt-cli).
|
||||
"""Reddit — multi-backend: OpenCLI / rdt-cli. Login is mandatory.
|
||||
|
||||
NOTE: Reddit requires authentication since 2024. All API requests
|
||||
(including public subreddit reads) return HTTP 403 without a valid
|
||||
session cookie. Run `rdt login` after installation to authenticate.
|
||||
Honest tiering (live-verified 2026-06): there is NO zero-config path.
|
||||
Anonymous .json endpoints are blocked (403 anti-bot, all variants), and
|
||||
the official API closed self-service registration in 2025-11 (manual
|
||||
approval, individual scripts rarely granted — PRAW is only an option for
|
||||
users who already hold credentials). Every working backend rides a
|
||||
logged-in session: OpenCLI reuses the browser's, rdt-cli imports cookies.
|
||||
"""
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
from agent_reach.utils.process import utf8_subprocess_env
|
||||
|
||||
from .base import Channel
|
||||
|
||||
_CREDENTIAL_FILE = "~/.config/rdt-cli/credential.json"
|
||||
# Pinned to the 0.4.2 state — PyPI still only has 0.4.1 (upstream issue #10).
|
||||
_RDT_GIT_SOURCE = "git+https://github.com/public-clis/rdt-cli.git@5e4fb3720d5c174e976cd425ccc3b879d52cac66"
|
||||
|
||||
#: shell 对"找到但不可执行/找不到"使用的退出码(对齐 agent_reach.probe)
|
||||
_BROKEN_EXIT_CODES = (126, 127)
|
||||
|
||||
#: rdt 应从固定 git 源安装(PyPI 落后),断链处方与 probe 默认的 pipx/uv 不同
|
||||
_RDT_BROKEN_HINT = (
|
||||
"rdt 命令存在但无法执行——通常是系统 Python 升级后 venv 解释器丢失。\n"
|
||||
"PyPI 版本落后,推荐用固定 git 源强制重装:\n"
|
||||
f" pipx install --force '{_RDT_GIT_SOURCE}'"
|
||||
)
|
||||
|
||||
|
||||
class RedditChannel(Channel):
|
||||
name = "reddit"
|
||||
description = "Reddit 帖子和评论"
|
||||
backends = ["rdt-cli"]
|
||||
tier = 0
|
||||
backends = ["OpenCLI", "rdt-cli"]
|
||||
tier = 1 # no zero-config path exists — see module docstring
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
@@ -28,17 +45,64 @@ class RedditChannel(Channel):
|
||||
return "reddit.com" in d or "redd.it" in d
|
||||
|
||||
def check(self, config=None):
|
||||
"""Probe candidates in order; first fully-usable backend wins."""
|
||||
self.active_backend = None
|
||||
findings = []
|
||||
|
||||
for backend in self.ordered_backends(config):
|
||||
if backend == "OpenCLI":
|
||||
result = self._check_opencli()
|
||||
else:
|
||||
result = self._check_rdt()
|
||||
if result is None:
|
||||
continue
|
||||
findings.append((backend, *result))
|
||||
|
||||
for wanted in ("ok", "warn"):
|
||||
for backend, status, message in findings:
|
||||
if status == wanted:
|
||||
self.active_backend = backend
|
||||
return status, message
|
||||
|
||||
if findings:
|
||||
return "error", "\n".join(m for _, _, m in findings)
|
||||
|
||||
return "off", (
|
||||
"未安装任何 Reddit 后端。注意:Reddit 没有零配置路径"
|
||||
"(匿名 .json 已被封,官方 API 需人工审批),必须用登录态。推荐:\n"
|
||||
" 桌面:agent-reach install --channels opencli\n"
|
||||
" (复用 Chrome 登录态,登录过 reddit.com 即可用)\n"
|
||||
f" 服务器/存量:pipx install '{_RDT_GIT_SOURCE}'\n"
|
||||
" 然后 `rdt login` 或手动写入 Cookie(见 doctor 提示)\n"
|
||||
"中国大陆访问 Reddit 需要代理"
|
||||
)
|
||||
|
||||
def _check_opencli(self):
|
||||
"""OpenCLI candidate. None = not installed."""
|
||||
from agent_reach.backends import opencli_status
|
||||
|
||||
st = opencli_status()
|
||||
if not st.installed:
|
||||
return None
|
||||
if st.broken:
|
||||
return "error", st.hint
|
||||
if st.ready:
|
||||
return "ok", (
|
||||
"OpenCLI 可用(复用浏览器登录态)。用法:"
|
||||
"opencli reddit search/read/subreddit/hot -f yaml"
|
||||
)
|
||||
return "warn", st.hint
|
||||
|
||||
def _check_rdt(self):
|
||||
"""rdt-cli candidate. None = not installed."""
|
||||
rdt = shutil.which("rdt")
|
||||
if not rdt:
|
||||
return "off", (
|
||||
"需要安装 rdt-cli(推荐使用最新版 v0.4.2+):\n"
|
||||
" pip install 'rdt-cli>=0.4.2'\n"
|
||||
"或:\n"
|
||||
" uv tool install rdt-cli\n"
|
||||
"最新源码:https://github.com/public-clis/rdt-cli\n"
|
||||
"安装后运行 `rdt login` 登录(需先在浏览器登录 reddit.com)"
|
||||
)
|
||||
return None
|
||||
|
||||
# 不走 probe_command:实测 `rdt status --json` 成功时(rc=0)也会向 stderr
|
||||
# 打网络重试日志,probe 把 stdout+stderr 合并后 JSON 解析必炸。
|
||||
# 故保留手写 subprocess(stdout 单独捕获),但异常分类对齐 probe 语义:
|
||||
# exec 失败/126/127 → broken(venv 断链处方),TimeoutExpired → 超时。
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[rdt, "status", "--json"],
|
||||
@@ -46,31 +110,56 @@ class RedditChannel(Channel):
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
timeout=10,
|
||||
env=utf8_subprocess_env(),
|
||||
)
|
||||
data = json.loads(r.stdout or "{}")
|
||||
authenticated = data.get("data", {}).get("authenticated", False)
|
||||
username = data.get("data", {}).get("username") or ""
|
||||
except subprocess.TimeoutExpired:
|
||||
return "error", "rdt 响应超时(>10s),Reddit 状态未知。稍后重试或运行 `rdt status` 查看详情"
|
||||
except OSError:
|
||||
# 含 FileNotFoundError:which 命中但 exec 失败 = venv 断链(probe 的 broken)
|
||||
return "error", _RDT_BROKEN_HINT
|
||||
|
||||
if authenticated:
|
||||
suffix = f"(已登录:{username})" if username else ""
|
||||
return "ok", (f"rdt-cli 可用{suffix}(搜索帖子、阅读全文、查看评论)")
|
||||
if r.returncode in _BROKEN_EXIT_CODES:
|
||||
return "error", _RDT_BROKEN_HINT
|
||||
|
||||
return "warn", (
|
||||
"rdt-cli 已安装但未登录。Reddit 自 2024 年起要求认证,"
|
||||
"未登录时所有请求均返回 403。\n\n"
|
||||
"方法一(自动):运行 `rdt login`\n"
|
||||
" 先在浏览器登录 reddit.com,再运行此命令自动提取 Cookie。\n\n"
|
||||
"方法二(手动,适用于 Chrome/Edge 127+ 无法自动提取时):\n"
|
||||
" 1. Chrome 应用商店安装 Cookie-Editor 扩展:\n"
|
||||
" https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm\n"
|
||||
" 2. 在浏览器打开 reddit.com(确保已登录)\n"
|
||||
" 3. 点击 Cookie-Editor 图标,找到 `reddit_session`,复制其 Value\n"
|
||||
f" 4. 将以下内容写入 {_CREDENTIAL_FILE}:\n"
|
||||
' {"cookies": {"reddit_session": "<粘贴 Value>"}, '
|
||||
'"source": "manual", "username": "<你的用户名>", '
|
||||
'"modhash": null, "saved_at": 0, "last_verified_at": null}\n\n'
|
||||
"验证:`rdt status --json` 确认 authenticated: true"
|
||||
if r.returncode != 0:
|
||||
detail = (r.stderr or r.stdout or "").strip().splitlines()
|
||||
tail = detail[-1] if detail else "无输出"
|
||||
return "error", f"rdt 异常退出(exit {r.returncode}):{tail}。运行 `rdt status` 查看详情"
|
||||
|
||||
# 进程正常退出 → rdt 本身是活的(无论登录与否)
|
||||
try:
|
||||
data = json.loads(r.stdout or "")
|
||||
except json.JSONDecodeError:
|
||||
data = None
|
||||
if not isinstance(data, dict):
|
||||
return "warn", "rdt-cli 可用但状态输出无法解析,运行 `rdt status` 查看登录状态"
|
||||
|
||||
info = data.get("data")
|
||||
if not isinstance(info, dict):
|
||||
info = {}
|
||||
authenticated = info.get("authenticated", False)
|
||||
username = info.get("username") or ""
|
||||
|
||||
if authenticated:
|
||||
suffix = f"(已登录:{username})" if username else ""
|
||||
return "ok", (
|
||||
f"rdt-cli 可用{suffix}(搜索帖子、阅读全文、查看评论;"
|
||||
"上游 2026-03 起停更,桌面用户建议迁移到 OpenCLI)"
|
||||
)
|
||||
|
||||
except (json.JSONDecodeError, FileNotFoundError, subprocess.TimeoutExpired):
|
||||
return "warn", "rdt-cli 已安装但状态检查失败,运行 `rdt status` 查看详情"
|
||||
return "warn", (
|
||||
"rdt-cli 已安装但未登录。Reddit 自 2024 年起要求认证,"
|
||||
"未登录时所有请求均返回 403。\n\n"
|
||||
"方法一(自动):运行 `rdt login`\n"
|
||||
" 先在浏览器登录 reddit.com,再运行此命令自动提取 Cookie。\n\n"
|
||||
"方法二(手动,适用于 Chrome/Edge 127+ 无法自动提取时):\n"
|
||||
" 1. Chrome 应用商店安装 Cookie-Editor 扩展:\n"
|
||||
" https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm\n"
|
||||
" 2. 在浏览器打开 reddit.com(确保已登录)\n"
|
||||
" 3. 点击 Cookie-Editor 图标,找到 `reddit_session`,复制其 Value\n"
|
||||
f" 4. 将以下内容写入 {_CREDENTIAL_FILE}:\n"
|
||||
' {"cookies": {"reddit_session": "<粘贴 Value>"}, '
|
||||
'"source": "manual", "username": "<你的用户名>", '
|
||||
'"modhash": null, "saved_at": 0, "last_verified_at": null}\n\n'
|
||||
"验证:`rdt status --json` 确认 authenticated: true"
|
||||
)
|
||||
|
||||
@@ -15,7 +15,13 @@ class RSSChannel(Channel):
|
||||
|
||||
def check(self, config=None):
|
||||
try:
|
||||
import feedparser
|
||||
return "ok", "可读取 RSS/Atom 源"
|
||||
import feedparser # noqa: F401
|
||||
except ImportError:
|
||||
self.active_backend = None
|
||||
return "off", "feedparser 未安装。安装:pip install feedparser"
|
||||
except Exception as e:
|
||||
# 已安装但导入期崩溃(半残安装/版本冲突)→ 重装处方
|
||||
self.active_backend = None
|
||||
return "error", f"feedparser 导入失败:{e}\n修复:pip install --force-reinstall feedparser"
|
||||
self.active_backend = self.backends[0]
|
||||
return "ok", "可读取 RSS/Atom 源"
|
||||
|
||||
+112
-49
@@ -1,15 +1,14 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Twitter/X — check if twitter-cli or bird CLI is available."""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from .base import Channel
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
|
||||
class TwitterChannel(Channel):
|
||||
name = "twitter"
|
||||
description = "Twitter/X 推文"
|
||||
backends = ["twitter-cli", "bird CLI (legacy)"]
|
||||
backends = ["twitter-cli", "OpenCLI", "bird CLI (legacy)"]
|
||||
tier = 1
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
@@ -18,56 +17,121 @@ class TwitterChannel(Channel):
|
||||
return "x.com" in d or "twitter.com" in d
|
||||
|
||||
def check(self, config=None):
|
||||
# Prefer twitter-cli, fallback to bird/birdx
|
||||
twitter = shutil.which("twitter")
|
||||
bird = shutil.which("bird") or shutil.which("birdx")
|
||||
"""Probe candidates in order; first fully-usable backend wins.
|
||||
|
||||
if twitter:
|
||||
return self._check_twitter_cli(twitter)
|
||||
elif bird:
|
||||
return self._check_bird(bird)
|
||||
else:
|
||||
与其他多后端渠道同一套两段式:先收集全部候选状态,第一个 ok 获胜;
|
||||
没有 ok 才轮到第一个 warn——否则「装了但未登录」的 twitter-cli
|
||||
会把排在后面、完整可用的 OpenCLI 挡在门外。
|
||||
"""
|
||||
self.active_backend = None
|
||||
findings = []
|
||||
|
||||
for backend in self.ordered_backends(config):
|
||||
if backend == "twitter-cli":
|
||||
result = self._check_twitter_cli()
|
||||
elif backend == "OpenCLI":
|
||||
result = self._check_opencli()
|
||||
elif backend == "bird CLI (legacy)":
|
||||
result = self._check_bird()
|
||||
else:
|
||||
continue
|
||||
|
||||
if result is None:
|
||||
continue # 未安装——不参与候选
|
||||
findings.append((backend, *result))
|
||||
|
||||
for wanted in ("ok", "warn"):
|
||||
for backend, status, message in findings:
|
||||
if status == wanted:
|
||||
self.active_backend = backend
|
||||
return status, message
|
||||
|
||||
if findings: # 只剩 broken/timeout 候选
|
||||
return "error", "\n".join(m for _, _, m in findings)
|
||||
|
||||
return "warn", (
|
||||
"Twitter CLI 未安装。安装方式:\n"
|
||||
" pipx install twitter-cli\n"
|
||||
"或:\n"
|
||||
" uv tool install twitter-cli"
|
||||
)
|
||||
|
||||
def _check_twitter_cli(self):
|
||||
"""探测 twitter-cli。返回 None 表示未安装,否则返回 (status, message)。
|
||||
|
||||
`twitter status` 才是健康信号:已登录时输出 "ok: true",
|
||||
未登录时以非零退出码输出 "not_authenticated"——工具本身是活的,
|
||||
所以 probe 的 error 状态也要看 output 内容再分类。
|
||||
"""
|
||||
probe = probe_command(
|
||||
"twitter", ["status"], timeout=15, retries=1, package="twitter-cli"
|
||||
)
|
||||
if probe.status == "missing":
|
||||
return None
|
||||
if probe.status == "broken":
|
||||
return "error", "twitter-cli 命令存在但无法执行。\n" + probe.hint
|
||||
if probe.status == "timeout":
|
||||
return "error", "twitter-cli 健康检查超时(已重试 1 次)。\n" + probe.hint
|
||||
|
||||
output = probe.output
|
||||
if "ok: true" in output:
|
||||
return "ok", (
|
||||
"twitter-cli 完整可用(搜索、读推文、时间线、长文/Article、"
|
||||
"用户查询、Thread)"
|
||||
)
|
||||
if "not_authenticated" in output:
|
||||
return "warn", (
|
||||
"Twitter CLI 未安装。安装方式:\n"
|
||||
" pipx install twitter-cli\n"
|
||||
"或:\n"
|
||||
" uv tool install twitter-cli"
|
||||
"twitter-cli 已安装但未认证。设置方式:\n"
|
||||
" export TWITTER_AUTH_TOKEN=\"xxx\"\n"
|
||||
" export TWITTER_CT0=\"yyy\"\n"
|
||||
"或确保已在浏览器中登录 x.com"
|
||||
)
|
||||
return "warn", (
|
||||
"twitter-cli 已安装但认证检查失败。运行:\n"
|
||||
" twitter -v status 查看详细信息"
|
||||
)
|
||||
|
||||
def _check_twitter_cli(self, binary: str):
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[binary, "status"], capture_output=True,
|
||||
encoding="utf-8", errors="replace", timeout=10
|
||||
)
|
||||
output = (r.stdout or "") + (r.stderr or "")
|
||||
if r.returncode == 0 and "ok: true" in output:
|
||||
return "ok", (
|
||||
"twitter-cli 完整可用(搜索、读推文、时间线、长文/Article、"
|
||||
"用户查询、Thread)"
|
||||
)
|
||||
if "not_authenticated" in output:
|
||||
return "warn", (
|
||||
"twitter-cli 已安装但未认证。设置方式:\n"
|
||||
" export TWITTER_AUTH_TOKEN=\"xxx\"\n"
|
||||
" export TWITTER_CT0=\"yyy\"\n"
|
||||
"或确保已在浏览器中登录 x.com"
|
||||
)
|
||||
return "warn", (
|
||||
"twitter-cli 已安装但认证检查失败。运行:\n"
|
||||
" twitter -v status 查看详细信息"
|
||||
)
|
||||
except Exception:
|
||||
return "warn", "twitter-cli 已安装但连接失败"
|
||||
def _check_opencli(self):
|
||||
"""OpenCLI candidate. None = not installed."""
|
||||
from agent_reach.backends import opencli_status
|
||||
|
||||
def _check_bird(self, binary: str):
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[binary, "check"], capture_output=True,
|
||||
encoding="utf-8", errors="replace", timeout=10
|
||||
st = opencli_status()
|
||||
if not st.installed:
|
||||
return None
|
||||
if st.broken:
|
||||
return "error", st.hint
|
||||
if st.ready:
|
||||
return "ok", (
|
||||
"OpenCLI 可用(复用浏览器登录态)。用法:"
|
||||
"opencli twitter search/article/user-posts -f yaml"
|
||||
)
|
||||
output = (r.stdout or "") + (r.stderr or "")
|
||||
if r.returncode == 0:
|
||||
return "warn", st.hint
|
||||
|
||||
def _check_bird(self):
|
||||
"""探测 bird/birdx(legacy 回退)。返回 None 表示均未安装,否则返回 (status, message)。"""
|
||||
last_failure = None
|
||||
for cmd in ("bird", "birdx"):
|
||||
probe = probe_command(
|
||||
cmd, ["check"], timeout=15, retries=1, package="@steipete/bird"
|
||||
)
|
||||
if probe.status == "missing":
|
||||
continue
|
||||
if probe.status == "broken":
|
||||
last_failure = (
|
||||
"error",
|
||||
f"{cmd} 命令存在但无法执行(bird 是 npm 包,可用 "
|
||||
"npm install -g @steipete/bird 重装)。\n" + probe.hint,
|
||||
)
|
||||
continue # bird 坏了再试 birdx
|
||||
if probe.status == "timeout":
|
||||
last_failure = (
|
||||
"error",
|
||||
f"{cmd} 健康检查超时(已重试 1 次)。\n" + probe.hint,
|
||||
)
|
||||
continue
|
||||
|
||||
output = probe.output
|
||||
if probe.ok:
|
||||
return "ok", "bird CLI 可用(读取、搜索推文,含长文/X Article)"
|
||||
if "Missing credentials" in output or "missing" in output.lower():
|
||||
return "warn", (
|
||||
@@ -78,5 +142,4 @@ class TwitterChannel(Channel):
|
||||
return "warn", (
|
||||
"bird CLI 已安装但认证检查失败。"
|
||||
)
|
||||
except Exception:
|
||||
return "warn", "bird CLI 已安装但连接失败"
|
||||
return last_failure
|
||||
|
||||
@@ -41,8 +41,10 @@ class V2EXChannel(Channel):
|
||||
_get_json(
|
||||
"https://www.v2ex.com/api/topics/show.json?node_name=python&page=1"
|
||||
)
|
||||
self.active_backend = self.backends[0]
|
||||
return "ok", "公开 API 可用(热门主题、节点浏览、主题详情、用户信息)"
|
||||
except Exception as e:
|
||||
self.active_backend = None
|
||||
return "warn", f"V2EX API 连接失败(可能需要代理):{e}"
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
@@ -17,6 +17,8 @@ class WebChannel(Channel):
|
||||
return True # Fallback — handles any URL
|
||||
|
||||
def check(self, config=None):
|
||||
# 恒可用兜底渠道:无本地命令、不做网络探测(doctor 已有多个渠道触网),保持零开销
|
||||
self.active_backend = self.backends[0]
|
||||
return "ok", "通过 Jina Reader 读取任意网页(curl https://r.jina.ai/URL)"
|
||||
|
||||
def read(self, url: str) -> str:
|
||||
|
||||
@@ -1,63 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""WeChat Official Account articles — read and search.
|
||||
|
||||
Read: Exa crawling (primary) / Camoufox stealth browser (optional)
|
||||
Search: Exa web_search with includeDomains mp.weixin.qq.com
|
||||
"""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from .base import Channel
|
||||
|
||||
|
||||
def _exa_available() -> bool:
|
||||
mcporter = shutil.which("mcporter")
|
||||
if not mcporter:
|
||||
return False
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[mcporter, "config", "list"],
|
||||
capture_output=True, encoding="utf-8", errors="replace", timeout=5,
|
||||
)
|
||||
return "exa" in r.stdout.lower()
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
class WeChatChannel(Channel):
|
||||
name = "wechat"
|
||||
description = "微信公众号文章"
|
||||
backends = ["Exa via mcporter (搜索+阅读)", "Camoufox (可选阅读)"]
|
||||
tier = 0
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
d = urlparse(url).netloc.lower()
|
||||
return "mp.weixin.qq.com" in d or "weixin.qq.com" in d
|
||||
|
||||
def check(self, config=None):
|
||||
has_exa = _exa_available()
|
||||
has_camoufox = False
|
||||
try:
|
||||
import camoufox # noqa: F401
|
||||
has_camoufox = True
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
if has_exa and has_camoufox:
|
||||
return "ok", "完整可用(Exa 搜索 + Exa/Camoufox 阅读公众号文章)"
|
||||
elif has_exa:
|
||||
return "ok", (
|
||||
"通过 Exa 搜索和阅读微信公众号文章(免费,无需额外配置)。"
|
||||
"可选安装 Camoufox 获得更好的全文阅读效果。"
|
||||
)
|
||||
elif has_camoufox:
|
||||
return "warn", (
|
||||
"Camoufox 可阅读公众号文章,但搜索功能需要 Exa。"
|
||||
"运行 `agent-reach install --env=auto` 安装 Exa。"
|
||||
)
|
||||
else:
|
||||
return "off", (
|
||||
"需要 mcporter + Exa MCP 来搜索和阅读微信公众号文章。\n"
|
||||
"运行 `agent-reach install --env=auto` 安装。"
|
||||
)
|
||||
@@ -1,52 +0,0 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Weibo (微博) — check if mcporter + mcp-server-weibo is available."""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from .base import Channel
|
||||
|
||||
|
||||
class WeiboChannel(Channel):
|
||||
name = "weibo"
|
||||
description = "微博动态与热搜"
|
||||
backends = ["mcp-server-weibo"]
|
||||
tier = 1
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
d = urlparse(url).netloc.lower()
|
||||
return "weibo.com" in d or "weibo.cn" in d
|
||||
|
||||
def check(self, config=None):
|
||||
mcporter = shutil.which("mcporter")
|
||||
if not mcporter:
|
||||
return "off", (
|
||||
"需要 mcporter + mcp-server-weibo。安装步骤:\n"
|
||||
" 1. npm install -g mcporter\n"
|
||||
" 2. pip install git+https://github.com/Panniantong/mcp-server-weibo.git\n"
|
||||
" 3. mcporter config add weibo --command 'mcp-server-weibo'\n"
|
||||
" 详见 https://github.com/Panniantong/mcp-server-weibo"
|
||||
)
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[mcporter, "config", "list"], capture_output=True,
|
||||
encoding="utf-8", errors="replace", timeout=5
|
||||
)
|
||||
if "weibo" not in r.stdout:
|
||||
return "off", (
|
||||
"mcporter 已装但微博 MCP 未配置。运行:\n"
|
||||
" pip install git+https://github.com/Panniantong/mcp-server-weibo.git\n"
|
||||
" mcporter config add weibo --command 'mcp-server-weibo'"
|
||||
)
|
||||
except Exception:
|
||||
return "off", "mcporter 连接异常"
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[mcporter, "list", "weibo"], capture_output=True,
|
||||
encoding="utf-8", errors="replace", timeout=15
|
||||
)
|
||||
if r.returncode == 0 and "search_users" in r.stdout:
|
||||
return "ok", "完整可用(热搜、搜索、用户动态、评论)"
|
||||
return "warn", "MCP 已配置但工具加载失败,检查 mcp-server-weibo 版本"
|
||||
except Exception:
|
||||
return "warn", "MCP 连接异常,检查 mcp-server-weibo 是否可用"
|
||||
@@ -1,10 +1,41 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""XiaoHongShu — check if xhs-cli (xiaohongshu-cli) is available."""
|
||||
"""XiaoHongShu — multi-backend: OpenCLI / xiaohongshu-mcp / xhs-cli.
|
||||
|
||||
Backend order encodes the recommendation, and probing order makes the
|
||||
environment split automatic: OpenCLI needs a desktop Chrome so it simply
|
||||
never probes alive on a server, where xiaohongshu-mcp (self-contained
|
||||
headless browser) takes over. xhs-cli (upstream unmaintained since
|
||||
2026-03) keeps working for existing installs as the last candidate.
|
||||
"""
|
||||
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
from agent_reach.probe import probe_command
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from .base import Channel
|
||||
|
||||
_MCP_ENDPOINT = "http://localhost:18060/mcp"
|
||||
_MCP_INSTALL_URL = "https://github.com/xpzouying/xiaohongshu-mcp"
|
||||
|
||||
|
||||
def _mcp_service_reachable(timeout: int = 3) -> bool:
|
||||
"""True if the xiaohongshu-mcp HTTP service answers on localhost.
|
||||
|
||||
Any HTTP response counts (the MCP endpoint replies 405 to GET) —
|
||||
we only care that the service is up. Proxies are bypassed explicitly:
|
||||
localhost must never be routed through HTTP_PROXY.
|
||||
"""
|
||||
req = urllib.request.Request(_MCP_ENDPOINT, method="GET")
|
||||
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
try:
|
||||
opener.open(req, timeout=timeout)
|
||||
return True
|
||||
except urllib.error.HTTPError:
|
||||
return True # 405/404 etc. — service is alive
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def format_xhs_result(data):
|
||||
"""Clean XHS API response, keeping only useful fields.
|
||||
@@ -118,7 +149,7 @@ def _clean_comment(comment):
|
||||
class XiaoHongShuChannel(Channel):
|
||||
name = "xiaohongshu"
|
||||
description = "小红书笔记"
|
||||
backends = ["xhs-cli (xiaohongshu-cli)"]
|
||||
backends = ["OpenCLI", "xiaohongshu-mcp", "xhs-cli (xiaohongshu-cli)"]
|
||||
tier = 1
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
@@ -127,36 +158,101 @@ class XiaoHongShuChannel(Channel):
|
||||
return "xiaohongshu.com" in d or "xhslink.com" in d
|
||||
|
||||
def check(self, config=None):
|
||||
xhs = shutil.which("xhs")
|
||||
if not xhs:
|
||||
return "off", (
|
||||
"需要安装 xhs-cli:\n"
|
||||
" pipx install xiaohongshu-cli\n"
|
||||
"或:\n"
|
||||
" uv tool install xiaohongshu-cli\n"
|
||||
"安装后运行 `xhs login` 登录"
|
||||
)
|
||||
"""Probe candidates in order; first fully-usable backend wins.
|
||||
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[xhs, "status"], capture_output=True,
|
||||
encoding="utf-8", errors="replace", timeout=10,
|
||||
If none is fully usable, the first fixable candidate (warn) is
|
||||
reported, so the user gets one actionable prescription instead
|
||||
of three half-relevant ones.
|
||||
"""
|
||||
self.active_backend = None
|
||||
findings = [] # (backend, status, message)
|
||||
|
||||
for backend in self.ordered_backends(config):
|
||||
if backend == "OpenCLI":
|
||||
result = self._check_opencli()
|
||||
elif backend == "xiaohongshu-mcp":
|
||||
result = self._check_mcp()
|
||||
else:
|
||||
result = self._check_xhs_cli()
|
||||
if result is None:
|
||||
continue # not installed — not a candidate right now
|
||||
findings.append((backend, *result))
|
||||
|
||||
for wanted in ("ok", "warn"):
|
||||
for backend, status, message in findings:
|
||||
if status == wanted:
|
||||
self.active_backend = backend
|
||||
return status, message
|
||||
|
||||
if findings: # only broken candidates left
|
||||
return "error", "\n".join(m for _, _, m in findings)
|
||||
|
||||
return "off", (
|
||||
"未安装任何小红书后端。推荐:\n"
|
||||
" 桌面:agent-reach install --channels opencli\n"
|
||||
" (复用 Chrome 登录态,刷过小红书即零配置可用)\n"
|
||||
f" 服务器:xiaohongshu-mcp(自带无头浏览器+扫码登录):{_MCP_INSTALL_URL}"
|
||||
)
|
||||
|
||||
def _check_opencli(self):
|
||||
"""OpenCLI candidate. None = not installed."""
|
||||
from agent_reach.backends import opencli_status
|
||||
|
||||
st = opencli_status()
|
||||
if not st.installed:
|
||||
return None
|
||||
if st.broken:
|
||||
return "error", st.hint
|
||||
if st.ready:
|
||||
return "ok", (
|
||||
"OpenCLI 可用(复用浏览器登录态)。用法:"
|
||||
"opencli xiaohongshu search/note/comments/feed -f yaml"
|
||||
)
|
||||
output = (r.stdout or "") + (r.stderr or "")
|
||||
if r.returncode == 0 and "ok: true" in output:
|
||||
return "ok", (
|
||||
"完整可用(搜索、阅读、评论、发帖、热门、"
|
||||
"收藏、关注、用户查询)"
|
||||
)
|
||||
if "not_authenticated" in output or "expired" in output:
|
||||
return "warn", (
|
||||
"xhs-cli 已安装但未登录。运行:\n"
|
||||
" xhs login\n"
|
||||
"(自动从浏览器提取 Cookie,或扫码登录)"
|
||||
)
|
||||
return "warn", st.hint
|
||||
|
||||
def _check_mcp(self):
|
||||
"""xiaohongshu-mcp candidate. None = service not running."""
|
||||
if not _mcp_service_reachable():
|
||||
return None
|
||||
mcporter = probe_command(
|
||||
"mcporter", ["config", "list"], timeout=10, package="mcporter"
|
||||
)
|
||||
if mcporter.ok and "xiaohongshu" in mcporter.output:
|
||||
return "ok", (
|
||||
"xiaohongshu-mcp 服务运行中"
|
||||
"(mcporter call 'xiaohongshu.search_feeds(keyword: \"...\")')。"
|
||||
"若未登录,让 agent 调 get_login_qrcode 扫码"
|
||||
)
|
||||
return "warn", (
|
||||
"xiaohongshu-mcp 服务在跑但 mcporter 未接入。运行:\n"
|
||||
f" mcporter config add xiaohongshu {_MCP_ENDPOINT}"
|
||||
)
|
||||
|
||||
def _check_xhs_cli(self):
|
||||
"""Legacy xhs-cli candidate. None = not installed."""
|
||||
probe = probe_command(
|
||||
"xhs", ["status"], timeout=10, package="xiaohongshu-cli"
|
||||
)
|
||||
if probe.status == "missing":
|
||||
return None
|
||||
if probe.status == "broken":
|
||||
return "error", "xhs 命令存在但无法执行\n" + probe.hint
|
||||
if probe.status == "timeout":
|
||||
return "warn", "xhs-cli 已安装但状态检测超时\n" + probe.hint
|
||||
|
||||
# 进程是活的(执行成功或运行后非零退出)——按输出内容分类
|
||||
if probe.ok and "ok: true" in probe.output:
|
||||
return "ok", (
|
||||
"xhs-cli 可用(搜索、阅读、评论、热门;上游 2026-03 起停更,"
|
||||
"桌面用户建议迁移到 OpenCLI)"
|
||||
)
|
||||
if "not_authenticated" in probe.output or "expired" in probe.output:
|
||||
return "warn", (
|
||||
"xhs-cli 已安装但状态异常。运行:\n"
|
||||
" xhs -v status 查看详细信息"
|
||||
"xhs-cli 已安装但未登录。运行:\n"
|
||||
" xhs login\n"
|
||||
"(自动从浏览器提取 Cookie,或扫码登录)"
|
||||
)
|
||||
except Exception:
|
||||
return "warn", "xhs-cli 已安装但连接失败"
|
||||
return "warn", (
|
||||
"xhs-cli 已安装但状态异常。运行:\n"
|
||||
" xhs -v status 查看详细信息"
|
||||
)
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
"""Xiaoyuzhou Podcast (小宇宙播客) — transcribe podcasts via Groq Whisper API."""
|
||||
|
||||
import os
|
||||
import shutil
|
||||
from agent_reach.config import Config
|
||||
from agent_reach.probe import probe_command
|
||||
from .base import Channel
|
||||
|
||||
|
||||
@@ -19,13 +19,21 @@ class XiaoyuzhouChannel(Channel):
|
||||
return "xiaoyuzhoufm.com" in d
|
||||
|
||||
def check(self, config=None):
|
||||
# Check ffmpeg
|
||||
if not shutil.which("ffmpeg"):
|
||||
self.active_backend = None
|
||||
|
||||
# Check ffmpeg — really execute it: a stale pip-installed ffmpeg shim
|
||||
# passes shutil.which() but cannot run
|
||||
probe = probe_command("ffmpeg", ["-version"], timeout=10, package="ffmpeg")
|
||||
if probe.status == "missing":
|
||||
return "off", (
|
||||
"需要 ffmpeg(音频转码和切片)。安装:\n"
|
||||
" Ubuntu/Debian: apt install -y ffmpeg\n"
|
||||
" macOS: brew install ffmpeg"
|
||||
)
|
||||
if not probe.ok:
|
||||
return "error", (
|
||||
"ffmpeg 无法执行,重装:brew install ffmpeg(macOS)/ apt install ffmpeg(Linux)"
|
||||
)
|
||||
|
||||
# Check script exists
|
||||
script = os.path.expanduser("~/.agent-reach/tools/xiaoyuzhou/transcribe.sh")
|
||||
@@ -51,4 +59,5 @@ class XiaoyuzhouChannel(Channel):
|
||||
" 2. 运行: agent-reach configure groq-key gsk_xxxxx"
|
||||
)
|
||||
|
||||
self.active_backend = "groq-whisper"
|
||||
return "ok", "完整可用(播客下载 + Whisper 转录)"
|
||||
|
||||
@@ -162,12 +162,14 @@ class XueqiuChannel(Channel):
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
def check(self, config=None):
|
||||
self.active_backend = None
|
||||
try:
|
||||
data = _get_json(
|
||||
"https://stock.xueqiu.com/v5/stock/batch/quote.json?symbol=SH000001"
|
||||
)
|
||||
items = (data.get("data") or {}).get("items") or []
|
||||
if items:
|
||||
self.active_backend = self.backends[0]
|
||||
return "ok", "公开 API 可用(行情、搜索、热帖、热股)"
|
||||
return "warn", "API 响应异常(返回数据为空)"
|
||||
except Exception as e:
|
||||
|
||||
@@ -3,12 +3,23 @@
|
||||
|
||||
import shutil
|
||||
|
||||
from agent_reach.probe import probe_command
|
||||
from agent_reach.utils.paths import get_ytdlp_config_path, render_ytdlp_fix_command
|
||||
from agent_reach.utils.text import read_utf8_text
|
||||
|
||||
from .base import Channel
|
||||
|
||||
|
||||
def _has_js_runtime_config(config_path) -> bool:
|
||||
"""Return whether yt-dlp config explicitly enables a JS runtime."""
|
||||
try:
|
||||
if not config_path.exists():
|
||||
return False
|
||||
return "--js-runtimes" in read_utf8_text(config_path)
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
class YouTubeChannel(Channel):
|
||||
name = "youtube"
|
||||
description = "YouTube 视频和字幕"
|
||||
@@ -17,12 +28,25 @@ class YouTubeChannel(Channel):
|
||||
|
||||
def can_handle(self, url: str) -> bool:
|
||||
from urllib.parse import urlparse
|
||||
|
||||
d = urlparse(url).netloc.lower()
|
||||
return "youtube.com" in d or "youtu.be" in d
|
||||
|
||||
def check(self, config=None):
|
||||
if not shutil.which("yt-dlp"):
|
||||
# 真跑 yt-dlp --version 探活,区分未装 / venv 断链 / 跑不动
|
||||
probe = probe_command("yt-dlp", ["--version"], timeout=10, package="yt-dlp")
|
||||
if probe.status == "missing":
|
||||
self.active_backend = None
|
||||
return "off", "yt-dlp 未安装。安装:pip install yt-dlp"
|
||||
if probe.status == "broken":
|
||||
self.active_backend = None
|
||||
return "error", f"yt-dlp 已安装但无法执行\n{probe.hint}"
|
||||
if not probe.ok: # timeout / error:装了但跑不动
|
||||
self.active_backend = None
|
||||
detail = probe.hint or probe.output or probe.status
|
||||
return "error", f"yt-dlp 无法正常运行:{detail}"
|
||||
# yt-dlp 本体是活的;后面的 JS runtime/转写检查只影响 ok/warn,不影响后端归属
|
||||
self.active_backend = "yt-dlp"
|
||||
# Check JS runtime
|
||||
has_js = shutil.which("deno") or shutil.which("node")
|
||||
if not has_js:
|
||||
@@ -35,12 +59,33 @@ class YouTubeChannel(Channel):
|
||||
has_deno = shutil.which("deno")
|
||||
if not has_deno:
|
||||
ytdlp_config = get_ytdlp_config_path()
|
||||
has_js_config = False
|
||||
if ytdlp_config.exists():
|
||||
has_js_config = "--js-runtimes" in read_utf8_text(ytdlp_config)
|
||||
if not has_js_config:
|
||||
if not _has_js_runtime_config(ytdlp_config):
|
||||
return "warn", (
|
||||
"yt-dlp 已安装但未配置 JS runtime。运行:\n"
|
||||
f" {render_ytdlp_fix_command()}"
|
||||
f"yt-dlp 已安装但未配置 JS runtime。运行:\n {render_ytdlp_fix_command()}"
|
||||
)
|
||||
return "ok", "可提取视频信息和字幕"
|
||||
# Surface transcription readiness so `doctor` reports it.
|
||||
msg = "可提取视频信息和字幕"
|
||||
if config is not None:
|
||||
providers = []
|
||||
if config.is_configured("groq_whisper"):
|
||||
providers.append("groq")
|
||||
if config.is_configured("openai_whisper"):
|
||||
providers.append("openai")
|
||||
if providers:
|
||||
if not shutil.which("ffmpeg"):
|
||||
msg += "(音频转写需安装 ffmpeg)"
|
||||
else:
|
||||
msg += f",可转写音频({'→'.join(providers)})"
|
||||
return "ok", msg
|
||||
|
||||
def transcribe(self, url: str, *, provider: str = "auto", config=None) -> str:
|
||||
"""Download a YouTube video's audio and return its transcript.
|
||||
|
||||
Delegates to :func:`agent_reach.transcribe.transcribe`. Imported lazily
|
||||
so the channel module stays cheap to import for users who never
|
||||
transcribe.
|
||||
"""
|
||||
from agent_reach.transcribe import transcribe as _transcribe
|
||||
|
||||
return _transcribe(url, provider=provider, config=config)
|
||||
|
||||
|
||||
+237
-175
@@ -17,6 +17,9 @@ import time
|
||||
|
||||
from agent_reach import __version__
|
||||
|
||||
# Pinned to the 0.4.2 state — PyPI still only has 0.4.1 (upstream issue #10).
|
||||
_RDT_GIT_SOURCE = "git+https://github.com/public-clis/rdt-cli.git@5e4fb3720d5c174e976cd425ccc3b879d52cac66"
|
||||
|
||||
|
||||
def _ensure_utf8_console():
|
||||
"""Best-effort Windows console UTF-8 setup for CLI runtime only."""
|
||||
@@ -63,20 +66,21 @@ def main():
|
||||
p_install.add_argument("--env", choices=["local", "server", "auto"], default="auto",
|
||||
help="Environment: local, server, or auto-detect")
|
||||
p_install.add_argument("--proxy", default="",
|
||||
help="Residential proxy for Reddit/Bilibili (http://user:pass@ip:port)")
|
||||
help="Network proxy saved for agents to export as HTTP(S)_PROXY "
|
||||
"in restricted networks (http://user:pass@ip:port)")
|
||||
p_install.add_argument("--safe", action="store_true",
|
||||
help="Safe mode: skip automatic system changes, show what's needed instead")
|
||||
p_install.add_argument("--dry-run", action="store_true",
|
||||
help="Show what would be done without making any changes")
|
||||
p_install.add_argument("--channels", default="",
|
||||
help="Comma-separated optional channels to install "
|
||||
"(twitter,weibo,wechat,xiaoyuzhou,xueqiu,xiaohongshu,"
|
||||
"reddit,bilibili,douyin,linkedin,all)")
|
||||
"(twitter,xiaoyuzhou,xueqiu,xiaohongshu,"
|
||||
"reddit,bilibili,linkedin,all)")
|
||||
|
||||
# ── configure ──
|
||||
p_conf = sub.add_parser("configure", help="Set a config value or auto-extract from browser")
|
||||
p_conf.add_argument("key", nargs="?", default=None,
|
||||
choices=["proxy", "github-token", "groq-key",
|
||||
choices=["proxy", "github-token", "groq-key", "openai-key",
|
||||
"twitter-cookies", "youtube-cookies",
|
||||
"xhs-cookies"],
|
||||
help="What to configure (omit if using --from-browser)")
|
||||
@@ -86,7 +90,9 @@ def main():
|
||||
help="Auto-extract ALL platform cookies from browser (chrome/firefox/edge/brave/opera)")
|
||||
|
||||
# ── doctor ──
|
||||
sub.add_parser("doctor", help="Check platform availability")
|
||||
p_doctor = sub.add_parser("doctor", help="Check platform availability")
|
||||
p_doctor.add_argument("--json", action="store_true",
|
||||
help="Output machine-readable JSON instead of the text report")
|
||||
|
||||
# ── uninstall ──
|
||||
p_uninstall = sub.add_parser("uninstall", help="Remove all Agent Reach config, tokens, and skill files")
|
||||
@@ -108,6 +114,14 @@ def main():
|
||||
p_format.add_argument("platform", choices=["xhs"], help="Platform to format (xhs)")
|
||||
|
||||
# ── check-update ──
|
||||
# ── transcribe ──
|
||||
p_tr = sub.add_parser("transcribe", help="Transcribe a URL or local audio file (Whisper via Groq/OpenAI)")
|
||||
p_tr.add_argument("source", help="Audio/video URL or local file path")
|
||||
p_tr.add_argument("--provider", choices=["auto", "groq", "openai"], default="auto",
|
||||
help="Transcription provider (default: auto = groq → openai fallback)")
|
||||
p_tr.add_argument("-o", "--output", default=None,
|
||||
help="Write transcript to a file instead of stdout")
|
||||
|
||||
sub.add_parser("check-update", help="Check for new versions and changes")
|
||||
|
||||
# ── watch ──
|
||||
@@ -130,7 +144,7 @@ def main():
|
||||
sys.exit(0)
|
||||
|
||||
if args.command == "doctor":
|
||||
_cmd_doctor()
|
||||
_cmd_doctor(args)
|
||||
elif args.command == "check-update":
|
||||
_cmd_check_update()
|
||||
elif args.command == "watch":
|
||||
@@ -147,6 +161,8 @@ def main():
|
||||
_cmd_skill(args)
|
||||
elif args.command == "format":
|
||||
_cmd_format(args)
|
||||
elif args.command == "transcribe":
|
||||
_cmd_transcribe(args)
|
||||
|
||||
|
||||
# ── Command handlers ────────────────────────────────
|
||||
@@ -180,14 +196,13 @@ def _cmd_install(args):
|
||||
# ── Parse --channels ──
|
||||
CHANNEL_INSTALLERS = {
|
||||
"twitter": _install_twitter_deps,
|
||||
"weibo": _install_weibo_deps,
|
||||
"wechat": _install_wechat_deps,
|
||||
"xiaoyuzhou": _install_xiaoyuzhou_deps,
|
||||
"xiaohongshu": _install_xhs_deps,
|
||||
"reddit": _install_reddit_deps,
|
||||
"bilibili": _install_bili_deps,
|
||||
"opencli": _install_opencli_deps, # cross-channel backend, desktop only
|
||||
# xueqiu: cookie-only, no install step
|
||||
# douyin/linkedin: manual setup, no auto-install
|
||||
# linkedin: manual setup, no auto-install
|
||||
}
|
||||
COOKIE_CHANNELS = {"twitter", "xueqiu", "bilibili"}
|
||||
|
||||
@@ -195,7 +210,7 @@ def _cmd_install(args):
|
||||
if args.channels:
|
||||
raw = [c.strip().lower() for c in args.channels.split(",") if c.strip()]
|
||||
if "all" in raw:
|
||||
requested_channels = set(CHANNEL_INSTALLERS.keys()) | {"xueqiu", "douyin", "linkedin"}
|
||||
requested_channels = set(CHANNEL_INSTALLERS.keys()) | {"xueqiu", "linkedin"}
|
||||
else:
|
||||
requested_channels = set(raw)
|
||||
|
||||
@@ -212,10 +227,11 @@ def _cmd_install(args):
|
||||
# Apply explicit flags
|
||||
if args.proxy:
|
||||
if dry_run:
|
||||
print(f"[dry-run] Would configure proxy for Bilibili")
|
||||
print(f"[dry-run] Would save network proxy")
|
||||
else:
|
||||
config.set("bilibili_proxy", args.proxy)
|
||||
print(f"✅ Proxy configured for Bilibili")
|
||||
config.set("proxy", args.proxy)
|
||||
config.set("bilibili_proxy", args.proxy) # legacy key
|
||||
print(f"✅ 代理已保存(Agent 访问受限网络时使用)")
|
||||
|
||||
# ── Install core system dependencies (lightweight, always) ──
|
||||
print()
|
||||
@@ -239,6 +255,10 @@ def _cmd_install(args):
|
||||
if requested_channels and not dry_run and not safe_mode:
|
||||
print()
|
||||
print("Installing optional channels...")
|
||||
if env == "server" and "opencli" in requested_channels:
|
||||
# OpenCLI rides a real desktop Chrome session — useless headless
|
||||
requested_channels.discard("opencli")
|
||||
print(" -- OpenCLI 需要桌面环境 + Chrome,服务器环境跳过")
|
||||
for ch_name in sorted(requested_channels):
|
||||
installer = CHANNEL_INSTALLERS.get(ch_name)
|
||||
if installer:
|
||||
@@ -280,9 +300,9 @@ def _cmd_install(args):
|
||||
# Environment-specific advice
|
||||
if env == "server":
|
||||
print()
|
||||
print("Tip: Bilibili may block server IPs.")
|
||||
print(" Reddit: rdt-cli works without proxy (pipx install rdt-cli).")
|
||||
print(" For Bilibili full access: agent-reach configure proxy http://user:pass@ip:port")
|
||||
print("Tip: 部分平台对服务器 IP 有风控。")
|
||||
print(" Reddit 必须登录态(rdt-cli + Cookie,见 doctor 提示),中国大陆网络还需代理。")
|
||||
print(" 保存代理供 Agent 使用:agent-reach configure proxy http://user:pass@ip:port")
|
||||
print(" Cheap option: https://www.webshare.io ($1/month)")
|
||||
|
||||
# Test channels
|
||||
@@ -307,7 +327,7 @@ def _cmd_install(args):
|
||||
# First install — hint about optional channels
|
||||
print()
|
||||
print("More channels available! Use --channels to install:")
|
||||
print(" agent-reach install --channels=twitter,weibo,xiaohongshu,...")
|
||||
print(" agent-reach install --channels=twitter,xiaohongshu,reddit,...")
|
||||
print(" agent-reach install --channels=all (install everything)")
|
||||
|
||||
# Star reminder
|
||||
@@ -351,8 +371,11 @@ def _install_skill():
|
||||
def _copy_skill_dir(target: str) -> bool:
|
||||
"""Copy entire skill directory (locale-specific SKILL.md + references/)."""
|
||||
try:
|
||||
# Clear existing installation
|
||||
if os.path.exists(target):
|
||||
# Clear existing installation. A symlinked skill dir (dotfiles
|
||||
# setups) breaks shutil.rmtree — unlink the link itself instead.
|
||||
if os.path.islink(target):
|
||||
os.unlink(target)
|
||||
elif os.path.exists(target):
|
||||
shutil.rmtree(target)
|
||||
os.makedirs(target, exist_ok=True)
|
||||
|
||||
@@ -441,7 +464,10 @@ def _uninstall_skill():
|
||||
skill_path = os.path.expanduser(skill_path_template)
|
||||
if os.path.isdir(skill_path):
|
||||
try:
|
||||
shutil.rmtree(skill_path)
|
||||
if os.path.islink(skill_path):
|
||||
os.unlink(skill_path)
|
||||
else:
|
||||
shutil.rmtree(skill_path)
|
||||
print(f" Removed {platform_name} skill: {skill_path}")
|
||||
removed = True
|
||||
except Exception as e:
|
||||
@@ -603,7 +629,7 @@ def _install_system_deps():
|
||||
except Exception:
|
||||
print(" -- Could not configure yt-dlp JS runtime (YouTube may not work)")
|
||||
|
||||
# NOTE: twitter-cli, weibo, xiaoyuzhou, wechat, xhs-cli etc. are optional.
|
||||
# NOTE: twitter-cli, xiaoyuzhou, xhs-cli etc. are optional.
|
||||
# They are installed via --channels flag, not here.
|
||||
# See CHANNEL_INSTALLERS in _cmd_install().
|
||||
|
||||
@@ -675,30 +701,99 @@ def _install_twitter_deps():
|
||||
|
||||
|
||||
def _install_xhs_deps():
|
||||
"""Install xhs-cli (xiaohongshu-cli) for XiaoHongShu."""
|
||||
"""Set up XiaoHongShu — backend depends on environment.
|
||||
|
||||
Desktop: OpenCLI (reuses the browser session, zero config).
|
||||
Server: xiaohongshu-mcp guide (self-contained headless browser + QR
|
||||
login; we don't manage long-running services, so guide only).
|
||||
xhs-cli is no longer installed by default — upstream unmaintained
|
||||
since 2026-03; existing installs keep working as a fallback backend.
|
||||
"""
|
||||
import shutil
|
||||
|
||||
print("Setting up XiaoHongShu...")
|
||||
if _detect_environment() == "server":
|
||||
print(" 服务器环境推荐 xiaohongshu-mcp(自带无头浏览器,扫码登录):")
|
||||
print(" 1. 下载 binary:https://github.com/xpzouying/xiaohongshu-mcp/releases")
|
||||
print(" (建议放到 ~/.agent-reach/tools/ 下)")
|
||||
print(" 2. 启动服务(首次运行会下载约 150MB 浏览器,请等待完成)")
|
||||
print(" 3. 扫码登录后接入:mcporter config add xiaohongshu http://localhost:18060/mcp")
|
||||
print(" 4. 验证:agent-reach doctor")
|
||||
return
|
||||
|
||||
_install_opencli_deps()
|
||||
if shutil.which("xhs"):
|
||||
print(" ✅ 检测到存量 xhs-cli,将作为备选后端继续可用")
|
||||
|
||||
|
||||
def _install_opencli_deps():
|
||||
"""Install OpenCLI — cross-platform backend riding the user's Chrome session.
|
||||
|
||||
Desktop-only. The npm package installs automatically; the Chrome
|
||||
extension CANNOT be installed programmatically (Chrome security model),
|
||||
so we print a one-click guide instead.
|
||||
"""
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
print("Setting up XiaoHongShu (xhs-cli)...")
|
||||
if shutil.which("xhs"):
|
||||
print(" ✅ xhs-cli already installed")
|
||||
from agent_reach.backends import (
|
||||
OPENCLI_EXTENSION_URL,
|
||||
OPENCLI_PACKAGE,
|
||||
opencli_status,
|
||||
opencli_summary,
|
||||
)
|
||||
|
||||
print("Setting up OpenCLI (browser-session backend, desktop only)...")
|
||||
st = opencli_status()
|
||||
if st.installed and not st.broken:
|
||||
print(f" ✅ {opencli_summary(st)}")
|
||||
if not st.ready:
|
||||
print(f" {st.hint}")
|
||||
return
|
||||
for tool, cmd in [("pipx", ["pipx", "install", "xiaohongshu-cli"]),
|
||||
("uv", ["uv", "tool", "install", "xiaohongshu-cli"])]:
|
||||
if shutil.which(tool):
|
||||
try:
|
||||
subprocess.run(cmd, capture_output=True, encoding="utf-8",
|
||||
errors="replace", timeout=120)
|
||||
if shutil.which("xhs"):
|
||||
print(" ✅ xhs-cli installed (run `xhs login` to authenticate)")
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
print(" [!] xhs-cli install failed. Run: pipx install xiaohongshu-cli")
|
||||
|
||||
if not shutil.which("npm"):
|
||||
print(" [!] OpenCLI requires Node.js ≥ 20. Install Node first:")
|
||||
print(" https://nodejs.org (或 brew install node)")
|
||||
return
|
||||
|
||||
try:
|
||||
subprocess.run(
|
||||
["npm", "install", "-g", OPENCLI_PACKAGE],
|
||||
capture_output=True, encoding="utf-8", errors="replace", timeout=300,
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
st = opencli_status()
|
||||
if st.installed and not st.broken:
|
||||
print(" ✅ OpenCLI installed")
|
||||
print(" 最后一步(必须手动,Chrome 安全限制):安装浏览器扩展")
|
||||
print(f" 1. 打开 {OPENCLI_EXTENSION_URL}")
|
||||
print(" 2. 点「添加至 Chrome」")
|
||||
print(" 3. 运行 `opencli doctor` 验证连接")
|
||||
else:
|
||||
print(f" [!] OpenCLI install failed. Run: npm install -g {OPENCLI_PACKAGE}")
|
||||
|
||||
|
||||
def _install_reddit_deps():
|
||||
"""Install rdt-cli for Reddit search + reading."""
|
||||
"""Set up Reddit — desktop prefers OpenCLI, rdt-cli for servers/legacy.
|
||||
|
||||
No zero-config path exists (anonymous .json blocked, official API
|
||||
approval-gated since 2025-11) — every backend needs a logged-in session.
|
||||
"""
|
||||
if _detect_environment() != "server":
|
||||
_install_opencli_deps()
|
||||
print(" Reddit 走 OpenCLI(浏览器里登录过 reddit.com 即可用)")
|
||||
import shutil
|
||||
if shutil.which("rdt"):
|
||||
print(" ✅ 检测到存量 rdt-cli,将作为备选后端继续可用")
|
||||
return
|
||||
|
||||
_install_rdt_cli()
|
||||
|
||||
|
||||
def _install_rdt_cli():
|
||||
"""Install rdt-cli (pinned git source — PyPI lags upstream)."""
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
@@ -706,8 +801,10 @@ def _install_reddit_deps():
|
||||
if shutil.which("rdt"):
|
||||
print(" ✅ rdt-cli already installed")
|
||||
return
|
||||
for tool, cmd in [("pipx", ["pipx", "install", "rdt-cli"]),
|
||||
("uv", ["uv", "tool", "install", "rdt-cli"])]:
|
||||
for tool, cmd in [
|
||||
("pipx", ["pipx", "install", _RDT_GIT_SOURCE]),
|
||||
("uv", ["uv", "tool", "install", "--from", _RDT_GIT_SOURCE, "rdt-cli"]),
|
||||
]:
|
||||
if shutil.which(tool):
|
||||
try:
|
||||
subprocess.run(cmd, capture_output=True, encoding="utf-8",
|
||||
@@ -717,7 +814,7 @@ def _install_reddit_deps():
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
print(" [!] rdt-cli install failed. Run: pipx install rdt-cli")
|
||||
print(f" [!] rdt-cli install failed. Run: pipx install '{_RDT_GIT_SOURCE}'")
|
||||
|
||||
|
||||
def _install_bili_deps():
|
||||
@@ -743,123 +840,6 @@ def _install_bili_deps():
|
||||
print(" [!] bili-cli install failed. Run: pipx install bilibili-cli")
|
||||
|
||||
|
||||
def _install_weibo_deps():
|
||||
"""Install Weibo MCP server (Panniantong fork with visitor passport auth)."""
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
print("Setting up Weibo MCP server...")
|
||||
|
||||
# Check if already installed and working
|
||||
mcporter = shutil.which("mcporter")
|
||||
if mcporter:
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[mcporter, "config", "list"], capture_output=True,
|
||||
encoding="utf-8", errors="replace", timeout=5
|
||||
)
|
||||
if "weibo" in r.stdout:
|
||||
print(" ✅ Weibo MCP already configured")
|
||||
return
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Install from our fork (has visitor passport auth fix)
|
||||
try:
|
||||
subprocess.run(
|
||||
[sys.executable, "-m", "pip", "install", "-q",
|
||||
"git+https://github.com/Panniantong/mcp-server-weibo.git"],
|
||||
check=True, timeout=120
|
||||
)
|
||||
print(" ✅ mcp-server-weibo installed (Panniantong fork)")
|
||||
except Exception as e:
|
||||
print(f" [!] mcp-server-weibo install failed: {e}")
|
||||
return
|
||||
|
||||
# Register with mcporter
|
||||
if mcporter:
|
||||
try:
|
||||
subprocess.run(
|
||||
[mcporter, "config", "add", "weibo", "--command", "mcp-server-weibo"],
|
||||
check=True, capture_output=True, timeout=10
|
||||
)
|
||||
print(" ✅ Weibo MCP registered with mcporter")
|
||||
except Exception:
|
||||
print(" [!] mcporter config add failed. Run manually: mcporter config add weibo --command 'mcp-server-weibo'")
|
||||
else:
|
||||
print(" -- mcporter not found, skipping MCP registration. Install mcporter first, then run: mcporter config add weibo --command 'mcp-server-weibo'")
|
||||
|
||||
|
||||
def _install_wechat_deps():
|
||||
"""Install WeChat article reading and search dependencies."""
|
||||
import subprocess
|
||||
|
||||
print("Setting up WeChat article tools...")
|
||||
|
||||
# Check if already installed
|
||||
has_camoufox = False
|
||||
has_miku = False
|
||||
try:
|
||||
import camoufox # noqa: F401
|
||||
has_camoufox = True
|
||||
except ImportError:
|
||||
pass
|
||||
try:
|
||||
import miku_ai # noqa: F401
|
||||
has_miku = True
|
||||
except ImportError:
|
||||
pass
|
||||
|
||||
# Install Python packages
|
||||
if has_camoufox and has_miku:
|
||||
print(" ✅ WeChat Python packages already installed")
|
||||
else:
|
||||
pkgs = []
|
||||
if not has_camoufox:
|
||||
pkgs.extend(["camoufox[geoip]", "markdownify", "beautifulsoup4", "httpx"])
|
||||
if not has_miku:
|
||||
pkgs.append("miku_ai")
|
||||
try:
|
||||
cmd = [sys.executable, "-m", "pip", "install", "--break-system-packages", "-q"] + pkgs
|
||||
subprocess.run(cmd, capture_output=True, encoding="utf-8", errors="replace", timeout=120)
|
||||
# Verify
|
||||
ok = True
|
||||
try:
|
||||
import importlib
|
||||
if not has_camoufox:
|
||||
importlib.import_module("camoufox")
|
||||
if not has_miku:
|
||||
importlib.import_module("miku_ai")
|
||||
except ImportError:
|
||||
ok = False
|
||||
if ok:
|
||||
print(f" ✅ WeChat Python packages installed ({', '.join(pkgs)})")
|
||||
else:
|
||||
print(f" [!] Some WeChat packages failed to install. Try: pip install {' '.join(pkgs)}")
|
||||
except Exception:
|
||||
print(f" [!] WeChat packages install failed. Try: pip install {' '.join(pkgs)}")
|
||||
|
||||
# Clone wechat-article-for-ai tool
|
||||
tools_dir = os.path.expanduser("~/.agent-reach/tools")
|
||||
wechat_dir = os.path.join(tools_dir, "wechat-article-for-ai")
|
||||
if os.path.isfile(os.path.join(wechat_dir, "main.py")):
|
||||
print(" ✅ wechat-article-for-ai tool already installed")
|
||||
else:
|
||||
try:
|
||||
os.makedirs(tools_dir, exist_ok=True)
|
||||
subprocess.run(
|
||||
["git", "clone", "--depth", "1",
|
||||
"https://github.com/Panniantong/wechat-article-for-ai.git", wechat_dir],
|
||||
capture_output=True, encoding="utf-8", errors="replace", timeout=60,
|
||||
)
|
||||
if os.path.isfile(os.path.join(wechat_dir, "main.py")):
|
||||
print(" ✅ wechat-article-for-ai tool installed")
|
||||
else:
|
||||
print(" [!] wechat-article-for-ai clone failed. Try: git clone https://github.com/Panniantong/wechat-article-for-ai.git " + wechat_dir)
|
||||
except Exception:
|
||||
print(" [!] wechat-article-for-ai clone failed. Try: git clone https://github.com/Panniantong/wechat-article-for-ai.git " + wechat_dir)
|
||||
|
||||
|
||||
def _install_system_deps_safe():
|
||||
"""Safe mode: check what's installed, print instructions for what's missing."""
|
||||
import shutil
|
||||
@@ -1058,9 +1038,14 @@ def _cmd_configure(args):
|
||||
return
|
||||
|
||||
if args.key == "proxy":
|
||||
# Generic network proxy for restricted environments. Nothing reads
|
||||
# this key at runtime — agents read it back and export HTTP(S)_PROXY
|
||||
# before invoking upstream tools (see docs/install.md). The legacy
|
||||
# bilibili_proxy key is kept in sync for older configs.
|
||||
config.set("proxy", value)
|
||||
config.set("bilibili_proxy", value)
|
||||
print(f"✅ Proxy configured for Bilibili!")
|
||||
print(" Note: Reddit 已改为通过 rdt-cli 访问,无需代理。")
|
||||
print("✅ 代理已保存(供 Agent 在访问 Reddit/Twitter 等需要代理的网络时设置 HTTP_PROXY/HTTPS_PROXY)")
|
||||
print(" Note: B站走 bili-cli,国内网络无需代理。")
|
||||
|
||||
elif args.key == "twitter-cookies":
|
||||
# Accept two formats:
|
||||
@@ -1120,6 +1105,29 @@ def _cmd_configure(args):
|
||||
config.set("groq_api_key", value)
|
||||
print(f"✅ Groq key configured!")
|
||||
|
||||
elif args.key == "openai-key":
|
||||
config.set("openai_api_key", value)
|
||||
print(f"✅ OpenAI key configured!")
|
||||
|
||||
|
||||
def _cmd_transcribe(args):
|
||||
"""Transcribe a URL or local audio file via Whisper (Groq → OpenAI fallback)."""
|
||||
from pathlib import Path
|
||||
|
||||
from agent_reach.transcribe import TranscribeError, transcribe
|
||||
|
||||
try:
|
||||
text = transcribe(args.source, provider=args.provider)
|
||||
except TranscribeError as e:
|
||||
print(f"❌ {e}")
|
||||
sys.exit(1)
|
||||
|
||||
if args.output:
|
||||
Path(args.output).write_text(text + "\n", encoding="utf-8")
|
||||
print(f"✅ Transcript written to {args.output}")
|
||||
else:
|
||||
print(text)
|
||||
|
||||
|
||||
def _parse_twitter_cookie_input(value: str):
|
||||
"""Parse Twitter cookie input from either separate values or a cookie header."""
|
||||
@@ -1225,11 +1233,28 @@ def _configure_xhs_cookies(value):
|
||||
# Find the container
|
||||
docker = shutil.which("docker")
|
||||
if not docker:
|
||||
# No Docker - write to a local file for manual import
|
||||
# No Docker - write to a local file for manual import.
|
||||
# Create with 0o600 atomically so the file is never world-readable
|
||||
# between open() and a follow-up chmod() (same pattern Config.save()
|
||||
# uses in config.py).
|
||||
import stat
|
||||
cookie_path = os.path.expanduser("~/.agent-reach/xhs-cookies.json")
|
||||
with open(cookie_path, "w") as f:
|
||||
f.write(cookies_json)
|
||||
os.chmod(cookie_path, 0o600)
|
||||
try:
|
||||
fd = os.open(
|
||||
cookie_path,
|
||||
os.O_WRONLY | os.O_CREAT | os.O_TRUNC,
|
||||
stat.S_IRUSR | stat.S_IWUSR, # 0o600
|
||||
)
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
f.write(cookies_json)
|
||||
except OSError:
|
||||
# Windows / unsupported flags — fall back to plain open + chmod.
|
||||
with open(cookie_path, "w", encoding="utf-8") as f:
|
||||
f.write(cookies_json)
|
||||
try:
|
||||
os.chmod(cookie_path, 0o600)
|
||||
except OSError:
|
||||
pass
|
||||
print(f" Cookies saved to {cookie_path}")
|
||||
print(" Docker not found. Copy manually:")
|
||||
print(f" docker cp {cookie_path} xiaohongshu-mcp:/app/data/cookies.json")
|
||||
@@ -1369,7 +1394,10 @@ def _cmd_uninstall(args):
|
||||
print(f"[dry-run] Would remove {platform_name} skill: {skill_path}")
|
||||
else:
|
||||
try:
|
||||
shutil.rmtree(skill_path)
|
||||
if os.path.islink(skill_path):
|
||||
os.unlink(skill_path)
|
||||
else:
|
||||
shutil.rmtree(skill_path)
|
||||
print(f" Removed {platform_name} skill: {skill_path}")
|
||||
removed_any = True
|
||||
except Exception as e:
|
||||
@@ -1416,7 +1444,7 @@ def _cmd_uninstall(args):
|
||||
print(" npm uninstall -g undici")
|
||||
|
||||
|
||||
def _cmd_doctor():
|
||||
def _cmd_doctor(args=None):
|
||||
from agent_reach.config import Config
|
||||
from agent_reach.doctor import check_all, format_report
|
||||
try:
|
||||
@@ -1425,6 +1453,11 @@ def _cmd_doctor():
|
||||
rprint = print
|
||||
config = Config()
|
||||
results = check_all(config)
|
||||
|
||||
if args is not None and getattr(args, "json", False):
|
||||
print(json.dumps(results, ensure_ascii=False, indent=2))
|
||||
return
|
||||
|
||||
rprint(format_report(results))
|
||||
|
||||
# Auto-install skill if not already present (fixes #154)
|
||||
@@ -1494,8 +1527,9 @@ def _cmd_setup():
|
||||
print()
|
||||
|
||||
# Step 3: Reddit — rdt-cli
|
||||
print("【信息】Reddit — 通过 rdt-cli 搜索和阅读,无需配置")
|
||||
print(" 安装:pipx install rdt-cli")
|
||||
print("【信息】Reddit — 必须登录态(无零配置路径)。桌面推荐 OpenCLI;或 rdt-cli:")
|
||||
print(f" 安装:pipx install '{_RDT_GIT_SOURCE}'")
|
||||
print(" 然后运行:rdt login(需先在浏览器登录 reddit.com)")
|
||||
print()
|
||||
|
||||
# Step 4: Groq (Whisper)
|
||||
@@ -1611,6 +1645,35 @@ def _github_get_with_retry(url, timeout=10, retries=3, sleeper=time.sleep):
|
||||
return None, "unknown", retries
|
||||
|
||||
|
||||
#: Full update = package + upstream tools + skill. The one-liner walks an
|
||||
#: agent through all three (docs/update.md); bare pip only updates the package.
|
||||
_UPDATE_INSTRUCTIONS = (
|
||||
"更新方式(推荐,复制这句话给你的 AI Agent,会完整更新本体+上游工具+skill):\n"
|
||||
" 帮我更新 Agent Reach:https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md\n"
|
||||
"仅更新本体(不含上游工具和 skill):\n"
|
||||
" pip install --upgrade https://github.com/Panniantong/agent-reach/archive/main.zip"
|
||||
)
|
||||
|
||||
|
||||
def _is_newer_version(remote: str, local: str) -> bool:
|
||||
"""True if remote is strictly newer than local (semantic compare).
|
||||
|
||||
A plain != would tell users "update available" when their local build is
|
||||
AHEAD of the latest release (e.g. installed from main during a release
|
||||
window) — and walk them into a downgrade.
|
||||
"""
|
||||
def parse(v):
|
||||
try:
|
||||
return tuple(int(x) for x in v.strip().split("."))
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
r, l = parse(remote), parse(local)
|
||||
if r is None or l is None:
|
||||
return remote != local # unparseable — fall back to old behavior
|
||||
return r > l
|
||||
|
||||
|
||||
def _cmd_check_update():
|
||||
"""Check for newer versions on GitHub."""
|
||||
from agent_reach import __version__
|
||||
@@ -1630,7 +1693,7 @@ def _cmd_check_update():
|
||||
latest = data.get("tag_name", "").lstrip("v")
|
||||
body = data.get("body", "")
|
||||
|
||||
if latest and latest != __version__:
|
||||
if latest and _is_newer_version(latest, __version__):
|
||||
print(f"最新版本: v{latest} ← 有更新!")
|
||||
if body:
|
||||
print()
|
||||
@@ -1639,8 +1702,7 @@ def _cmd_check_update():
|
||||
for line in body.strip().split("\n")[:20]:
|
||||
print(f" {line}")
|
||||
print()
|
||||
print("更新命令:")
|
||||
print(" pip install --upgrade https://github.com/Panniantong/agent-reach/archive/main.zip")
|
||||
print(_UPDATE_INSTRUCTIONS)
|
||||
return "update_available"
|
||||
print(f"✅ 已是最新版本")
|
||||
return "up_to_date"
|
||||
@@ -1662,8 +1724,7 @@ def _cmd_check_update():
|
||||
date = commit.get("commit", {}).get("committer", {}).get("date", "")[:10]
|
||||
print(f"最新提交: {sha} ({date}) {msg}")
|
||||
print()
|
||||
print("更新命令:")
|
||||
print(" pip install --upgrade https://github.com/Panniantong/agent-reach/archive/main.zip")
|
||||
print(_UPDATE_INSTRUCTIONS)
|
||||
return "unknown"
|
||||
|
||||
commit_err = _classify_github_response_error(resp2)
|
||||
@@ -1711,7 +1772,7 @@ def _cmd_watch():
|
||||
if not err and resp and resp.status_code == 200:
|
||||
data = resp.json()
|
||||
latest = data.get("tag_name", "").lstrip("v")
|
||||
if latest and latest != __version__:
|
||||
if latest and _is_newer_version(latest, __version__):
|
||||
update_available = True
|
||||
new_version = latest
|
||||
release_body = data.get("body", "")
|
||||
@@ -1736,7 +1797,8 @@ def _cmd_watch():
|
||||
if release_body:
|
||||
for line in release_body.strip().split("\n")[:10]:
|
||||
print(f" {line}")
|
||||
print(f" 更新: pip install --upgrade https://github.com/Panniantong/agent-reach/archive/main.zip")
|
||||
print(" 更新(一句话发给 Agent 即可完整更新):")
|
||||
print(" 帮我更新 Agent Reach:https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -21,8 +21,9 @@ class Config:
|
||||
# Feature → required config keys
|
||||
FEATURE_REQUIREMENTS = {
|
||||
"exa_search": ["exa_api_key"],
|
||||
"twitter_xreach": ["twitter_auth_token", "twitter_ct0"], # legacy key name; used by bird CLI
|
||||
"twitter_xreach": ["twitter_auth_token", "twitter_ct0"], # legacy key name; used by twitter-cli
|
||||
"groq_whisper": ["groq_api_key"],
|
||||
"openai_whisper": ["openai_api_key"],
|
||||
"github_token": ["github_token"],
|
||||
}
|
||||
|
||||
|
||||
@@ -148,6 +148,28 @@ def extract_all(browser: str = "chrome") -> Dict[str, dict]:
|
||||
return results
|
||||
|
||||
|
||||
def _open_owner_only(path: str):
|
||||
"""Open *path* for writing, atomically creating it with mode 0o600.
|
||||
|
||||
Mirrors the pattern used by Config.save() in config.py: O_WRONLY|O_CREAT|
|
||||
O_TRUNC + an explicit mode argument so the file is never briefly
|
||||
world-readable between open() and a later os.chmod(). On Windows (or any
|
||||
OS that rejects the open flags) we fall back to a plain open().
|
||||
"""
|
||||
import os
|
||||
import stat
|
||||
|
||||
try:
|
||||
fd = os.open(
|
||||
path,
|
||||
os.O_WRONLY | os.O_CREAT | os.O_TRUNC,
|
||||
stat.S_IRUSR | stat.S_IWUSR, # 0o600
|
||||
)
|
||||
return os.fdopen(fd, "w", encoding="utf-8")
|
||||
except OSError:
|
||||
return open(path, "w", encoding="utf-8")
|
||||
|
||||
|
||||
def _sync_xfetch_session(auth_token: str, ct0: str) -> None:
|
||||
"""Sync Twitter credentials to ~/.config/xfetch/session.json (legacy xreach compat)."""
|
||||
import json
|
||||
@@ -166,9 +188,8 @@ def _sync_xfetch_session(auth_token: str, ct0: str) -> None:
|
||||
session_data = {}
|
||||
session_data["authToken"] = auth_token
|
||||
session_data["ct0"] = ct0
|
||||
with open(session_path, "w", encoding="utf-8") as sf:
|
||||
with _open_owner_only(session_path) as sf:
|
||||
json.dump(session_data, sf, indent=2)
|
||||
os.chmod(session_path, 0o600)
|
||||
except Exception:
|
||||
# Non-fatal: agent-reach config is the source of truth, xfetch sync is best-effort
|
||||
pass
|
||||
@@ -179,17 +200,19 @@ def _sync_bird_env(auth_token: str, ct0: str) -> None:
|
||||
|
||||
bird reads AUTH_TOKEN and CT0 from environment variables. This writes a
|
||||
shell-sourceable file so users can `source ~/.config/bird/credentials.env`.
|
||||
Values are passed through shlex.quote so a token containing a quote, $, or
|
||||
backtick cannot break out into shell syntax when the file is sourced.
|
||||
"""
|
||||
import os
|
||||
import shlex
|
||||
|
||||
try:
|
||||
bird_dir = os.path.join(os.path.expanduser("~"), ".config", "bird")
|
||||
os.makedirs(bird_dir, exist_ok=True)
|
||||
env_path = os.path.join(bird_dir, "credentials.env")
|
||||
with open(env_path, "w", encoding="utf-8") as f:
|
||||
f.write(f'AUTH_TOKEN="{auth_token}"\n')
|
||||
f.write(f'CT0="{ct0}"\n')
|
||||
os.chmod(env_path, 0o600)
|
||||
with _open_owner_only(env_path) as f:
|
||||
f.write(f"AUTH_TOKEN={shlex.quote(auth_token)}\n")
|
||||
f.write(f"CT0={shlex.quote(ct0)}\n")
|
||||
except Exception:
|
||||
# Non-fatal: agent-reach config is the source of truth, bird env sync is best-effort
|
||||
pass
|
||||
|
||||
+26
-7
@@ -10,20 +10,40 @@ from agent_reach.channels import get_all_channels
|
||||
|
||||
|
||||
def check_all(config: Config) -> Dict[str, dict]:
|
||||
"""Check all channels and return status dict."""
|
||||
"""Check all channels and return status dict.
|
||||
|
||||
A single misbehaving channel must never take the whole report down,
|
||||
so per-channel exceptions degrade to status="error".
|
||||
"""
|
||||
results = {}
|
||||
for ch in get_all_channels():
|
||||
status, message = ch.check(config)
|
||||
try:
|
||||
status, message = ch.check(config)
|
||||
active = getattr(ch, "active_backend", None)
|
||||
except Exception as e: # noqa: BLE001 — doctor must survive any channel
|
||||
# Channels are registry singletons: a stale active_backend from a
|
||||
# previous check must not leak into an errored result.
|
||||
status, message, active = "error", f"体检异常:{e}", None
|
||||
results[ch.name] = {
|
||||
"status": status,
|
||||
"name": ch.description,
|
||||
"message": message,
|
||||
"tier": ch.tier,
|
||||
"backends": ch.backends,
|
||||
"active_backend": active,
|
||||
}
|
||||
return results
|
||||
|
||||
|
||||
def _name_msg(r: dict, escape) -> str:
|
||||
"""Render one channel line; show the active backend when there is a choice."""
|
||||
text = f"[bold]{escape(r['name'])}[/bold] — {escape(r['message'])}"
|
||||
active = r.get("active_backend")
|
||||
if active and len(r.get("backends", [])) > 1:
|
||||
text += f" [dim](当前后端:{escape(active)})[/dim]"
|
||||
return text
|
||||
|
||||
|
||||
def format_report(results: Dict[str, dict]) -> str:
|
||||
"""Format results as a readable text report (with Rich markup)."""
|
||||
try:
|
||||
@@ -34,6 +54,7 @@ def format_report(results: Dict[str, dict]) -> str:
|
||||
lines = []
|
||||
lines.append("[bold cyan]Agent Reach 状态[/bold cyan]")
|
||||
lines.append("[cyan]" + "=" * 40 + "[/cyan]")
|
||||
lines.append("图例:[green]✅[/green] 可用 [yellow][!][/yellow] 已装但需配置/登录 [red][X][/red] 未安装")
|
||||
|
||||
ok_count = sum(1 for r in results.values() if r["status"] == "ok")
|
||||
total = len(results)
|
||||
@@ -43,7 +64,7 @@ def format_report(results: Dict[str, dict]) -> str:
|
||||
lines.append("[bold]✅ 装好即用:[/bold]")
|
||||
for key, r in results.items():
|
||||
if r["tier"] == 0:
|
||||
name_msg = f"[bold]{escape(r['name'])}[/bold] — {escape(r['message'])}"
|
||||
name_msg = _name_msg(r, escape)
|
||||
if r["status"] == "ok":
|
||||
lines.append(f" [green]✅[/green] {name_msg}")
|
||||
elif r["status"] == "warn":
|
||||
@@ -59,8 +80,7 @@ def format_report(results: Dict[str, dict]) -> str:
|
||||
lines.append("")
|
||||
lines.append("[bold]可选渠道(已安装):[/bold]")
|
||||
for key, r in tier1_active.items():
|
||||
name_msg = f"[bold]{escape(r['name'])}[/bold] — {escape(r['message'])}"
|
||||
lines.append(f" [green]✅[/green] {name_msg}")
|
||||
lines.append(f" [green]✅[/green] {_name_msg(r, escape)}")
|
||||
|
||||
# Tier 2 — optional complex setup
|
||||
tier2 = {k: r for k, r in results.items() if r["tier"] == 2}
|
||||
@@ -71,8 +91,7 @@ def format_report(results: Dict[str, dict]) -> str:
|
||||
lines.append("")
|
||||
lines.append("[bold]可选渠道(已安装):[/bold]")
|
||||
for key, r in tier2_active.items():
|
||||
name_msg = f"[bold]{escape(r['name'])}[/bold] — {escape(r['message'])}"
|
||||
lines.append(f" [green]✅[/green] {name_msg}")
|
||||
lines.append(f" [green]✅[/green] {_name_msg(r, escape)}")
|
||||
|
||||
lines.append("")
|
||||
status_color = "green" if ok_count == total else ("yellow" if ok_count > 0 else "red")
|
||||
|
||||
@@ -8,7 +8,7 @@ Agent Reach 通过 **rdt-cli** 实现 Reddit 的搜索和阅读功能:
|
||||
- **搜索**:`rdt search "关键词"`
|
||||
- **阅读完整帖子+评论**:`rdt read POST_ID`
|
||||
|
||||
免费,无需代理,无需 API Key,无需登录。
|
||||
免费,无需代理,无需 API Key。需要登录认证(`rdt login`,自动从浏览器提取 Cookie)。
|
||||
|
||||
## Agent 可自动完成的步骤
|
||||
|
||||
@@ -17,9 +17,9 @@ Agent Reach 通过 **rdt-cli** 实现 Reddit 的搜索和阅读功能:
|
||||
which rdt && echo "installed" || echo "not installed"
|
||||
```
|
||||
|
||||
2. 如果未安装,自动安装:
|
||||
2. 如果未安装,自动安装(PyPI 版本暂时落后,从 GitHub 安装最新版):
|
||||
```bash
|
||||
pipx install rdt-cli
|
||||
pipx install 'git+https://github.com/public-clis/rdt-cli.git'
|
||||
```
|
||||
|
||||
或一键安装:
|
||||
|
||||
@@ -1,40 +0,0 @@
|
||||
# 微信公众号配置指南
|
||||
|
||||
## 功能说明
|
||||
读取微信公众号文章。需要 Playwright 来处理微信的反爬机制。
|
||||
|
||||
## Agent 可自动完成的步骤
|
||||
|
||||
1. 检查 Playwright 是否安装:
|
||||
```bash
|
||||
python3 -c "import playwright; print('installed')" 2>&1
|
||||
```
|
||||
|
||||
2. 安装 Playwright + 浏览器:
|
||||
```bash
|
||||
pip install playwright
|
||||
playwright install chromium
|
||||
```
|
||||
|
||||
3. 安装完成后测试:
|
||||
```bash
|
||||
curl -s "https://r.jina.ai/https://mp.weixin.qq.com/s/一个测试链接" -H "Accept: text/markdown"
|
||||
```
|
||||
|
||||
## 需要用户手动做的步骤
|
||||
|
||||
请告诉用户:
|
||||
|
||||
> 微信公众号的配置很简单,只需要安装一个浏览器组件(约 150MB)。
|
||||
>
|
||||
> 我来帮你安装,你不需要做任何事情。安装过程大约 1-2 分钟。
|
||||
>
|
||||
> 安装好之后就可以直接读取微信公众号文章了,不需要登录。
|
||||
|
||||
## Agent 操作流程
|
||||
|
||||
1. 安装 Playwright:`pip install playwright`
|
||||
2. 安装 Chromium:`playwright install chromium`
|
||||
3. 测试:读一篇微信文章
|
||||
4. 反馈:"✅ 微信公众号已配置!发给我任何公众号文章链接,我都能读取。"
|
||||
5. 如果安装失败(空间不足等):"❌ 浏览器组件安装失败。可能是磁盘空间不足(需要约 150MB)。"
|
||||
@@ -0,0 +1,103 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Lightweight upstream command probing.
|
||||
|
||||
Distinguishes the three failure modes that look identical to shutil.which():
|
||||
- missing: command not on PATH
|
||||
- broken: command exists but cannot execute — most commonly a stale venv
|
||||
shebang after a system Python upgrade (pipx/uv tool installs break this
|
||||
way: which() finds the shim, but exec fails with FileNotFoundError
|
||||
pointing at the shim itself)
|
||||
- timeout/error: command runs but misbehaves
|
||||
|
||||
Channels use probe_command() inside check() so doctor reports real health,
|
||||
not just file existence.
|
||||
"""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional, Sequence
|
||||
|
||||
from agent_reach.utils.process import utf8_subprocess_env
|
||||
|
||||
#: Exit codes shells use for "found but not executable" / "not found".
|
||||
_BROKEN_EXIT_CODES = (126, 127)
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProbeResult:
|
||||
status: str # "ok" | "missing" | "broken" | "timeout" | "error"
|
||||
output: str = ""
|
||||
hint: str = ""
|
||||
|
||||
@property
|
||||
def ok(self) -> bool:
|
||||
return self.status == "ok"
|
||||
|
||||
|
||||
def reinstall_hint(package: str) -> str:
|
||||
"""Prescription for a broken (stale-venv) CLI install."""
|
||||
return (
|
||||
f"命令存在但无法执行——通常是系统 Python 升级后 venv 解释器丢失。重装即可修复:\n"
|
||||
f" uv tool install --force {package}\n"
|
||||
f"或:pipx reinstall {package}"
|
||||
)
|
||||
|
||||
|
||||
def probe_command(
|
||||
cmd: str,
|
||||
args: Sequence[str] = ("--version",),
|
||||
timeout: int = 10,
|
||||
retries: int = 0,
|
||||
package: Optional[str] = None,
|
||||
) -> ProbeResult:
|
||||
"""Actually execute `cmd *args` and classify the result.
|
||||
|
||||
Intended for SIDE-EFFECT-FREE health probes only (version/status
|
||||
commands): retries re-run the command verbatim with no backoff, so a
|
||||
non-idempotent command would repeat its effect.
|
||||
|
||||
package: pip/pipx package name used in the broken-install hint
|
||||
(defaults to cmd).
|
||||
"""
|
||||
path = shutil.which(cmd)
|
||||
if not path:
|
||||
return ProbeResult("missing")
|
||||
|
||||
last: Optional[ProbeResult] = None
|
||||
for _ in range(retries + 1):
|
||||
last = _run_once(path, args, timeout, package or cmd)
|
||||
if last.ok:
|
||||
return last
|
||||
# missing/broken won't heal between retries — only transient
|
||||
# failures (timeout/error) are worth a second attempt
|
||||
if last.status in ("missing", "broken"):
|
||||
return last
|
||||
return last
|
||||
|
||||
|
||||
def _run_once(path: str, args: Sequence[str], timeout: int, package: str) -> ProbeResult:
|
||||
try:
|
||||
r = subprocess.run(
|
||||
[path, *args],
|
||||
capture_output=True,
|
||||
encoding="utf-8",
|
||||
errors="replace",
|
||||
timeout=timeout,
|
||||
env=utf8_subprocess_env(),
|
||||
)
|
||||
except FileNotFoundError:
|
||||
# which() found it but exec failed: the shebang interpreter is gone
|
||||
return ProbeResult("broken", hint=reinstall_hint(package))
|
||||
except OSError:
|
||||
return ProbeResult("broken", hint=reinstall_hint(package))
|
||||
except subprocess.TimeoutExpired:
|
||||
return ProbeResult("timeout", hint=f"`{path}` 响应超时(>{timeout}s)")
|
||||
|
||||
if r.returncode in _BROKEN_EXIT_CODES:
|
||||
return ProbeResult("broken", hint=reinstall_hint(package))
|
||||
|
||||
output = (r.stdout or "") + (r.stderr or "")
|
||||
if r.returncode != 0:
|
||||
return ProbeResult("error", output=output.strip())
|
||||
return ProbeResult("ok", output=output.strip())
|
||||
@@ -1,11 +1,29 @@
|
||||
#!/bin/bash
|
||||
# 小宇宙播客转文字脚本
|
||||
# 用法: bash transcribe.sh <小宇宙链接> [输出文件路径]
|
||||
# 用法: bash transcribe.sh [--polish] <小宇宙链接> [输出文件路径]
|
||||
# 环境变量: GROQ_API_KEY (必须)
|
||||
#
|
||||
# --polish: 转录后调用 Groq Llama 3.3 70B 给文稿补中文标点+合理分段
|
||||
# (Whisper 对中文标点支持较弱,开启后阅读体验显著更好)
|
||||
|
||||
set -e
|
||||
|
||||
URL="${1:?用法: bash transcribe.sh <小宇宙链接> [输出文件路径]}"
|
||||
POLISH=0
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--polish) POLISH=1; shift ;;
|
||||
--) shift; break ;;
|
||||
-h|--help)
|
||||
echo "用法: bash transcribe.sh [--polish] <小宇宙链接> [输出文件路径]"
|
||||
exit 0 ;;
|
||||
--*)
|
||||
echo "未知选项: $1" >&2
|
||||
exit 1 ;;
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
|
||||
URL="${1:?用法: bash transcribe.sh [--polish] <小宇宙链接> [输出文件路径]}"
|
||||
OUTPUT="${2:-/tmp/podcast_transcript.txt}"
|
||||
TMPDIR="/tmp/xiaoyuzhou_$$"
|
||||
|
||||
@@ -35,8 +53,8 @@ echo "===================="
|
||||
# Step 1: 提取音频 URL 和标题
|
||||
echo "🔍 正在解析页面..."
|
||||
PAGE=$(curl -s "$URL")
|
||||
AUDIO_URL=$(echo "$PAGE" | grep -oP 'https://media\.xyzcdn\.net/[^"]*\.(m4a|mp3)' | head -1)
|
||||
TITLE=$(echo "$PAGE" | grep -oP '"title":"[^"]*"' | head -1 | sed 's/"title":"//;s/"//')
|
||||
AUDIO_URL=$(echo "$PAGE" | perl -ne 'while (/(https:\/\/media\.xyzcdn\.net\/[^"]*\.(?:m4a|mp3))/gi) { print "$1\n" }' | head -1)
|
||||
TITLE=$(echo "$PAGE" | perl -ne 'if (/"title":"([^"]*)"/) { print "$1\n"; last }' | head -1)
|
||||
|
||||
if [ -z "$AUDIO_URL" ]; then
|
||||
echo "❌ 无法从页面提取音频链接"
|
||||
@@ -99,6 +117,7 @@ for i in $(seq 0 $((NUM_CHUNKS - 1))); do
|
||||
-F file="@$TMPDIR/chunk_${i}.mp3" \
|
||||
-F model="whisper-large-v3" \
|
||||
-F language="zh" \
|
||||
-F prompt="以下是一段中文普通话播客录音,请输出包含完整中文标点(,。?!:;“”‘’)的转写文本。" \
|
||||
-F response_format="text")
|
||||
|
||||
HTTP_CODE=$(echo "$RESPONSE" | tail -1)
|
||||
@@ -111,7 +130,7 @@ for i in $(seq 0 $((NUM_CHUNKS - 1))); do
|
||||
# 如果是速率限制,等待后重试
|
||||
if [ "$HTTP_CODE" = "429" ]; then
|
||||
# 从错误信息中提取等待时间,默认 120 秒
|
||||
WAIT_SEC=$(echo "$BODY" | grep -oP 'in \K[0-9]+m' | sed 's/m//' | head -1)
|
||||
WAIT_SEC=$(echo "$BODY" | perl -ne 'if (/in (\d+)m/) { print "$1\n"; exit }')
|
||||
WAIT_SEC=${WAIT_SEC:-2}
|
||||
WAIT_SEC=$((WAIT_SEC * 60 + 30))
|
||||
echo " ⏳ 速率限制,等待 ${WAIT_SEC} 秒后重试..."
|
||||
@@ -122,6 +141,7 @@ for i in $(seq 0 $((NUM_CHUNKS - 1))); do
|
||||
-F file="@$TMPDIR/chunk_${i}.mp3" \
|
||||
-F model="whisper-large-v3" \
|
||||
-F language="zh" \
|
||||
-F prompt="以下是一段中文普通话播客录音,请输出包含完整中文标点(,。?!:;“”‘’)的转写文本。" \
|
||||
-F response_format="text")
|
||||
HTTP_CODE=$(echo "$RESPONSE" | tail -1)
|
||||
BODY=$(echo "$RESPONSE" | sed '$d')
|
||||
@@ -140,6 +160,81 @@ for i in $(seq 0 $((NUM_CHUNKS - 1))); do
|
||||
echo "✅ ($CHARS 字)"
|
||||
done
|
||||
|
||||
# Step 6.5 (可选): 用 Llama 3.3 70B 给文稿补标点+分段
|
||||
if [ "$POLISH" = "1" ]; then
|
||||
echo "✨ 正在润色(Llama 3.3 70B 加标点+分段)..."
|
||||
for i in $(seq 0 $((NUM_CHUNKS - 1))); do
|
||||
echo -n " 段 $((i+1))/$NUM_CHUNKS... "
|
||||
IN_FILE="$TMPDIR/transcript_${i}.txt" \
|
||||
OUT_FILE="$TMPDIR/polished_${i}.txt" \
|
||||
GROQ_API_KEY="$GROQ_API_KEY" \
|
||||
python3 <<'PY'
|
||||
import json, os, sys, urllib.request, urllib.error
|
||||
|
||||
KEY = os.environ["GROQ_API_KEY"]
|
||||
IN = os.environ["IN_FILE"]
|
||||
OUT = os.environ["OUT_FILE"]
|
||||
|
||||
MODEL = "llama-3.3-70b-versatile"
|
||||
MAX_DEPTH = 3
|
||||
PROMPT_TMPL = (
|
||||
"以下是一段中文普通话播客的语音转写片段,由于 Whisper 对中文标点支持较弱,"
|
||||
"整段几乎没有标点。请你**只做一件事**:在合适位置补充中文标点(,。!?:;),"
|
||||
"可以适度分段。\n\n"
|
||||
"**严格要求**:\n"
|
||||
"- 不得修改、删除、增加任何汉字或英文/数字\n"
|
||||
"- 不得改写、润色、总结\n"
|
||||
"- 不得添加任何解释、前言、后记\n"
|
||||
"- 直接输出加好标点+合理分段后的全文\n\n"
|
||||
"原文:\n{}"
|
||||
)
|
||||
|
||||
def call_groq(text):
|
||||
body = json.dumps({
|
||||
"model": MODEL,
|
||||
"temperature": 0.2,
|
||||
"max_completion_tokens": 8192,
|
||||
"messages": [{"role": "user", "content": PROMPT_TMPL.format(text)}],
|
||||
}).encode()
|
||||
req = urllib.request.Request(
|
||||
"https://api.groq.com/openai/v1/chat/completions",
|
||||
data=body,
|
||||
headers={
|
||||
"Authorization": f"Bearer {KEY}",
|
||||
"Content-Type": "application/json",
|
||||
"User-Agent": "agent-reach-xiaoyuzhou/1.0",
|
||||
},
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=180) as r:
|
||||
resp = json.load(r)
|
||||
return (
|
||||
resp["choices"][0]["message"]["content"].strip(),
|
||||
resp["choices"][0].get("finish_reason"),
|
||||
)
|
||||
|
||||
def polish(text, depth=0):
|
||||
try:
|
||||
out, fr = call_groq(text)
|
||||
except urllib.error.HTTPError as e:
|
||||
sys.stderr.write(f"polish HTTP {e.code}: {e.read().decode(errors='replace')[:200]}\n")
|
||||
return text # fallback to raw
|
||||
except Exception as e:
|
||||
sys.stderr.write(f"polish error: {e}\n")
|
||||
return text
|
||||
if fr != "length" or depth >= MAX_DEPTH:
|
||||
return out
|
||||
# 输出被截断:从中点切两半递归处理
|
||||
mid = len(text) // 2
|
||||
return polish(text[:mid], depth + 1) + polish(text[mid:], depth + 1)
|
||||
|
||||
content = open(IN, encoding="utf-8").read().strip()
|
||||
result = polish(content)
|
||||
open(OUT, "w", encoding="utf-8").write(result + "\n")
|
||||
print(f"✅ ({len(result)} 字)")
|
||||
PY
|
||||
done
|
||||
fi
|
||||
|
||||
# Step 7: 合并输出
|
||||
echo "📄 正在合并文字稿..."
|
||||
|
||||
@@ -149,12 +244,19 @@ echo "📄 正在合并文字稿..."
|
||||
echo "来源: $URL"
|
||||
echo "时长: ${DURATION_MIN}分${DURATION_SEC}秒"
|
||||
echo "转录时间: $(date '+%Y-%m-%d %H:%M')"
|
||||
if [ "$POLISH" = "1" ]; then
|
||||
echo "润色: Groq Llama 3.3 70B"
|
||||
fi
|
||||
echo ""
|
||||
echo "---"
|
||||
echo ""
|
||||
|
||||
|
||||
for i in $(seq 0 $((NUM_CHUNKS - 1))); do
|
||||
cat "$TMPDIR/transcript_${i}.txt"
|
||||
if [ "$POLISH" = "1" ] && [ -f "$TMPDIR/polished_${i}.txt" ]; then
|
||||
cat "$TMPDIR/polished_${i}.txt"
|
||||
else
|
||||
cat "$TMPDIR/transcript_${i}.txt"
|
||||
fi
|
||||
echo ""
|
||||
done
|
||||
} > "$OUTPUT"
|
||||
|
||||
+58
-32
@@ -1,28 +1,36 @@
|
||||
---
|
||||
name: agent-reach
|
||||
description: >
|
||||
Give your AI agent eyes to see the entire internet.
|
||||
17 platforms via CLI, MCP, curl, and Python scripts.
|
||||
Zero config for 8 channels.
|
||||
MUST USE when user wants to 调研/research/搜索/search/查/找/look up anything
|
||||
on the internet — e.g. 全网调研 X / 帮我调研一下 X / 查一下 X / 搜搜 X /
|
||||
看看大家怎么评价 X / X 上有什么讨论 / research this topic。
|
||||
|
||||
Also MUST USE when user mentions any platform or shares any URL/链接:
|
||||
小红书/xiaohongshu/xhs, Twitter/推特/X, B站/bilibili, Reddit, V2EX,
|
||||
LinkedIn/领英/招聘/求职/jobs, YouTube, GitHub code search, 小宇宙播客,
|
||||
雪球/股票行情, RSS feeds, or any web URL.
|
||||
|
||||
13 platforms, multi-backend routing (OpenCLI / per-platform CLIs / APIs).
|
||||
Zero config for 6 channels. Run `agent-reach doctor --json` to see which
|
||||
backend serves each platform right now.
|
||||
|
||||
NOT for: 写报告/数据分析/翻译等内容加工(本 skill 只负责从互联网获取内容);
|
||||
发帖/评论/点赞等写操作;已有专门 skill 的平台(先用专门 skill)。
|
||||
|
||||
【路由方式】SKILL.md 包含路由表和常用命令,复杂场景需按需阅读对应分类的 references/*.md。
|
||||
分类:search / social (小红书/抖音/微博/推特/B站/V2EX/Reddit) / career(LinkedIn) / dev(github) / web(网页/文章/公众号/RSS) / video(YouTube/B站/播客).
|
||||
|
||||
Use when user asks to search, read, or interact on any supported platform,
|
||||
shares a URL, or asks to search the web.
|
||||
分类:search / social (小红书/推特/B站/V2EX/Reddit) / career(LinkedIn) / dev(github) / web(网页/文章/RSS) / video(YouTube/B站/播客)。
|
||||
triggers:
|
||||
- search: 搜/查/找/search/搜索/查一下/帮我搜
|
||||
- research: 调研/全网调研/帮我调研/研究一下/research/深入了解
|
||||
- search: 搜/查/找/search/搜索/查一下/帮我搜/看看大家怎么说
|
||||
- social:
|
||||
- 小红书: xiaohongshu/xhs/小红书/红书
|
||||
- 抖音: douyin/抖音
|
||||
- Twitter: twitter/推特/x.com/推文
|
||||
- 微博: weibo/微博
|
||||
- B站: bilibili/b站/哔哩哔哩
|
||||
- V2EX: v2ex
|
||||
- Reddit: reddit
|
||||
- career: 招聘/职位/求职/linkedin/领英/找工作
|
||||
- dev: github/代码/仓库/gh/issue/pr/分支/commit
|
||||
- web: 网页/链接/文章/公众号/微信文章/rss/读一下/打开这个
|
||||
- web: 网页/链接/文章/rss/读一下/打开这个
|
||||
- video: youtube/视频/播客/字幕/小宇宙/转录/yt
|
||||
- finance: 雪球/股票/stock/xueqiu/行情/基金
|
||||
metadata:
|
||||
@@ -30,19 +38,32 @@ metadata:
|
||||
homepage: https://github.com/Panniantong/Agent-Reach
|
||||
---
|
||||
|
||||
# Agent Reach — 路由器
|
||||
# Agent Reach — 互联网能力路由器
|
||||
|
||||
17 平台工具集合。根据用户意图选择对应分类。
|
||||
13 平台、多后端。**本 skill 存在时必须用它访问这些平台,不要自己发明方案。**
|
||||
|
||||
## 常驻规则(全程适用)
|
||||
|
||||
1. **动手前先体检**:多后端平台(小红书/Reddit/B站/Twitter)先跑
|
||||
`agent-reach doctor --json`,按各平台 `active_backend` 字段选命令组。
|
||||
2. **声明你在用什么**:开始干活前说一句「使用 agent-reach 的 X 平台 / Y 后端」。
|
||||
3. **失败按 references 里的重试链处理**,不要瞎猜命令。
|
||||
4. **全网调研类任务**:组合多平台(Exa 搜索 + Twitter/Reddit 看讨论 + 小红书/B站看中文场景),并行收集再汇总。
|
||||
5. **替用户盯版本**:完成一次较大的调研/多平台任务后,顺手跑
|
||||
`agent-reach check-update`(很快,一个 API 调用)。有新版就在收尾汇报里附一句:
|
||||
「Agent Reach 有新版 vX.Y.Z,复制这句话给我即可更新:帮我更新 Agent Reach:
|
||||
https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md」。
|
||||
不要中断当前任务去更新,也不要重复提醒同一个版本。
|
||||
|
||||
## 路由表
|
||||
|
||||
| 用户意图 | 分类 | 详细文档 |
|
||||
|---------|------|---------|
|
||||
| 网页搜索/代码搜索 | search | [references/search.md](references/search.md) |
|
||||
| 小红书/抖音/微博/推特/B站/V2EX/Reddit | social | [references/social.md](references/social.md) |
|
||||
| 小红书/推特/B站/V2EX/Reddit | social | [references/social.md](references/social.md) |
|
||||
| 招聘/职位/LinkedIn | career | [references/career.md](references/career.md) |
|
||||
| GitHub/代码 | dev | [references/dev.md](references/dev.md) |
|
||||
| 网页/文章/公众号/RSS | web | [references/web.md](references/web.md) |
|
||||
| 网页/文章/RSS | web | [references/web.md](references/web.md) |
|
||||
| YouTube/B站/播客字幕 | video | [references/video.md](references/video.md) |
|
||||
|
||||
## 零配置快速命令
|
||||
@@ -57,30 +78,35 @@ curl -s "https://r.jina.ai/URL"
|
||||
# GitHub 搜索
|
||||
gh search repos "query" --sort stars --limit 10
|
||||
|
||||
# Twitter 搜索
|
||||
twitter search "query" --limit 10
|
||||
|
||||
# YouTube/B站字幕
|
||||
# YouTube 字幕(注意:B站不要用 yt-dlp,见 video.md)
|
||||
yt-dlp --write-sub --skip-download -o "/tmp/%(id)s" "URL"
|
||||
|
||||
# Reddit 搜索
|
||||
rdt search "query" --limit 10
|
||||
|
||||
# Reddit 读帖 + 评论
|
||||
rdt read POST_ID
|
||||
|
||||
# V2EX 热门
|
||||
curl -s "https://www.v2ex.com/api/topics/hot.json" -H "User-Agent: agent-reach/1.0"
|
||||
|
||||
# B站搜索(bili-cli,无需登录)
|
||||
bili search "query" --type video -n 5
|
||||
```
|
||||
|
||||
## 需登录态的平台(按 doctor 的 active_backend 选命令)
|
||||
|
||||
```bash
|
||||
# Twitter 搜索(twitter-cli 首选;失败重试链见 social.md)
|
||||
twitter search "query" -n 10
|
||||
|
||||
# Reddit(无零配置路径:OpenCLI 或 rdt-cli,必须登录态)
|
||||
opencli reddit search "query" -f yaml # 桌面
|
||||
rdt search "query" --limit 10 # 存量/服务器
|
||||
|
||||
# 小红书(桌面首选 OpenCLI)
|
||||
opencli xiaohongshu search "query" -f yaml
|
||||
```
|
||||
|
||||
## 环境检查
|
||||
|
||||
```bash
|
||||
# 检查可用 channel
|
||||
agent-reach doctor
|
||||
|
||||
# 查看所有 MCP 服务
|
||||
mcporter_list_servers()
|
||||
# 检查可用 channel 与每个平台当前激活的后端
|
||||
agent-reach doctor --json
|
||||
```
|
||||
|
||||
## 工作区规则
|
||||
@@ -92,10 +118,10 @@ mcporter_list_servers()
|
||||
根据用户需求,阅读对应的详细文档:
|
||||
|
||||
- [搜索工具](references/search.md) — Exa AI 搜索
|
||||
- [社交媒体](references/social.md) — 小红书, 抖音, Twitter, B站, V2EX, Reddit
|
||||
- [社交媒体](references/social.md) — 小红书, Twitter, B站, V2EX, Reddit(多后端命令组)
|
||||
- [职场招聘](references/career.md) — LinkedIn
|
||||
- [开发工具](references/dev.md) — GitHub CLI
|
||||
- [网页阅读](references/web.md) — Jina Reader, 微信公众号, RSS
|
||||
- [网页阅读](references/web.md) — Jina Reader, RSS
|
||||
- [视频播客](references/video.md) — YouTube, B站, 小宇宙
|
||||
|
||||
## 配置渠道
|
||||
|
||||
+84
-271
@@ -1,312 +1,125 @@
|
||||
---
|
||||
name: agent-reach
|
||||
description: >
|
||||
Give your AI agent eyes to see the entire internet.
|
||||
Search and read 17 platforms: Twitter/X, Reddit, YouTube, GitHub, Bilibili,
|
||||
XiaoHongShu, Douyin, Weibo, WeChat Articles, Xiaoyuzhou Podcast, LinkedIn,
|
||||
V2EX, Xueqiu, RSS, Exa web search, and any web page.
|
||||
Zero config for 8 channels. Use when the user asks to search, read, or interact
|
||||
on any supported platform, shares a URL, or asks to search the web.
|
||||
Triggers: "search twitter", "search xiaohongshu", "watch this video",
|
||||
"search the web", "look this up", "research", "youtube transcript",
|
||||
"search reddit", "read this link", "bilibili", "douyin video",
|
||||
"wechat article", "wechat official account", "weibo", "V2EX",
|
||||
"xiaoyuzhou", "podcast", "xueqiu", "stock quote",
|
||||
"install agent reach".
|
||||
MUST USE when user wants to research/search/look up/find anything on the
|
||||
internet — e.g. "research this topic", "do a deep dive on X", "search the
|
||||
web for X", "see what people say about X", "look this up".
|
||||
|
||||
Also MUST USE when user mentions any platform or shares any URL/link:
|
||||
Twitter/X, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu,
|
||||
Xiaoyuzhou Podcast, LinkedIn/jobs/recruiting, V2EX, Xueqiu (stocks), RSS.
|
||||
|
||||
13 platforms, multi-backend routing (OpenCLI / per-platform CLIs / APIs).
|
||||
Zero config for 6 channels. Run `agent-reach doctor --json` to see which
|
||||
backend serves each platform right now.
|
||||
|
||||
NOT for: writing reports/analysis/translation (this skill only FETCHES
|
||||
internet content); posting/commenting/liking (write operations); platforms
|
||||
that already have a dedicated skill installed (prefer that skill).
|
||||
metadata:
|
||||
openclaw:
|
||||
homepage: https://github.com/Panniantong/Agent-Reach
|
||||
---
|
||||
|
||||
# Agent Reach — Usage Guide
|
||||
# Agent Reach — internet capability router
|
||||
|
||||
Upstream tools for 17 platforms. Call them directly.
|
||||
13 platforms, multiple backends each. **When this skill exists, use it for
|
||||
these platforms — do not invent your own approach.**
|
||||
|
||||
Run `agent-reach doctor` to check which channels are available.
|
||||
## Standing rules (apply for the whole session)
|
||||
|
||||
## ⚠️ Workspace Rules
|
||||
1. **Health-check before acting**: for multi-backend platforms (XiaoHongShu /
|
||||
Reddit / Bilibili / Twitter), run `agent-reach doctor --json` first and
|
||||
pick the command group matching each platform's `active_backend`.
|
||||
2. **Announce what you use**: say "using agent-reach, platform X via backend Y"
|
||||
before starting.
|
||||
3. **On failure, follow the retry chains in references/** — never guess
|
||||
commands.
|
||||
4. **For broad research tasks**: combine platforms (Exa for web search +
|
||||
Twitter/Reddit for discussions + XiaoHongShu/Bilibili for Chinese
|
||||
perspectives), collect in parallel, then synthesize.
|
||||
5. **Watch versions for the user**: after finishing a substantial
|
||||
multi-platform task, run `agent-reach check-update` (fast, one API call).
|
||||
If a new version exists, append one line to your wrap-up: "Agent Reach
|
||||
vX.Y.Z is available — paste this to me to update: 帮我更新 Agent Reach:
|
||||
https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md".
|
||||
Never interrupt the current task to update; never nag about the same version twice.
|
||||
|
||||
**Never create files in the agent workspace.** Use `/tmp/` for temporary output and `~/.agent-reach/` for persistent data.
|
||||
## Routing table
|
||||
|
||||
## Web — Any URL
|
||||
|
||||
```bash
|
||||
curl -s "https://r.jina.ai/URL"
|
||||
```
|
||||
|
||||
## Web Search (Exa)
|
||||
| User intent | Category | Details |
|
||||
|---------|------|---------|
|
||||
| Web / code search | search | [references/search.md](references/search.md) |
|
||||
| XiaoHongShu / Twitter / Bilibili / V2EX / Reddit | social | [references/social.md](references/social.md) |
|
||||
| Jobs / LinkedIn | career | [references/career.md](references/career.md) |
|
||||
| GitHub / code | dev | [references/dev.md](references/dev.md) |
|
||||
| Web pages / articles / RSS | web | [references/web.md](references/web.md) |
|
||||
| YouTube / Bilibili / podcast transcripts | video | [references/video.md](references/video.md) |
|
||||
|
||||
## Zero-config quick commands
|
||||
|
||||
```bash
|
||||
# Exa web search
|
||||
mcporter call 'exa.web_search_exa(query: "query", numResults: 5)'
|
||||
mcporter call 'exa.get_code_context_exa(query: "code question", tokensNum: 3000)'
|
||||
```
|
||||
|
||||
## Twitter/X (bird)
|
||||
# Read any web page
|
||||
curl -s "https://r.jina.ai/URL"
|
||||
|
||||
```bash
|
||||
bird search "query" -n 10 # search
|
||||
bird read URL_OR_ID # read tweet (supports /status/ and /article/ URLs)
|
||||
bird user-tweets @username -n 20 # user timeline
|
||||
bird thread URL_OR_ID # full thread
|
||||
```
|
||||
|
||||
## YouTube (yt-dlp)
|
||||
|
||||
```bash
|
||||
yt-dlp --dump-json "URL" # video metadata
|
||||
yt-dlp --write-sub --write-auto-sub --sub-lang "zh-Hans,zh,en" --skip-download -o "/tmp/%(id)s" "URL"
|
||||
# download subtitles, then read the .vtt file
|
||||
yt-dlp --dump-json "ytsearch5:query" # search
|
||||
```
|
||||
|
||||
## Bilibili (yt-dlp)
|
||||
|
||||
```bash
|
||||
yt-dlp --dump-json "https://www.bilibili.com/video/BVxxx"
|
||||
yt-dlp --write-sub --write-auto-sub --sub-lang "zh-Hans,zh,en" --convert-subs vtt --skip-download -o "/tmp/%(id)s" "URL"
|
||||
```
|
||||
|
||||
> Server IPs may get 412. Use `--cookies-from-browser chrome` or configure a proxy.
|
||||
|
||||
## Reddit
|
||||
|
||||
```bash
|
||||
curl -s "https://www.reddit.com/r/SUBREDDIT/hot.json?limit=10" -H "User-Agent: agent-reach/1.0"
|
||||
curl -s "https://www.reddit.com/search.json?q=QUERY&limit=10" -H "User-Agent: agent-reach/1.0"
|
||||
```
|
||||
|
||||
> Server IPs may get 403. Search via Exa instead, or configure a proxy.
|
||||
|
||||
## GitHub (gh CLI)
|
||||
|
||||
```bash
|
||||
# GitHub search
|
||||
gh search repos "query" --sort stars --limit 10
|
||||
gh repo view owner/repo
|
||||
gh search code "query" --language python
|
||||
gh issue list -R owner/repo --state open
|
||||
gh issue view 123 -R owner/repo
|
||||
```
|
||||
|
||||
## XiaoHongShu (mcporter)
|
||||
# YouTube subtitles (NOTE: never use yt-dlp for Bilibili — see video.md)
|
||||
yt-dlp --write-sub --skip-download -o "/tmp/%(id)s" "URL"
|
||||
|
||||
```bash
|
||||
mcporter call 'xiaohongshu.search_feeds(keyword: "query")'
|
||||
mcporter call 'xiaohongshu.get_feed_detail(feed_id: "xxx", xsec_token: "yyy")'
|
||||
mcporter call 'xiaohongshu.get_feed_detail(feed_id: "xxx", xsec_token: "yyy", load_all_comments: true)'
|
||||
mcporter call 'xiaohongshu.publish_content(title: "Title", content: "Body text", images: ["/path/img.jpg"], tags: ["tag"])'
|
||||
```
|
||||
|
||||
> Requires login. Use Cookie-Editor to import cookies.
|
||||
|
||||
> **Tip: Clean bloated output.** The XHS API returns large JSON with many unused fields.
|
||||
> Pipe through the formatter to save context:
|
||||
> ```bash
|
||||
> mcporter call 'xiaohongshu.search_feeds(keyword: "query")' | agent-reach format xhs
|
||||
> ```
|
||||
> This keeps only: title, content, author, engagement counts, image URLs, and tags.
|
||||
|
||||
## Douyin (mcporter)
|
||||
|
||||
```bash
|
||||
mcporter call 'douyin.parse_douyin_video_info(share_link: "https://v.douyin.com/xxx/")'
|
||||
mcporter call 'douyin.get_douyin_download_link(share_link: "https://v.douyin.com/xxx/")'
|
||||
```
|
||||
|
||||
> No login needed.
|
||||
|
||||
## WeChat Articles
|
||||
|
||||
**Search** (`miku_ai`):
|
||||
```bash
|
||||
# miku_ai is installed inside the agent-reach Python environment.
|
||||
# Use the same interpreter that runs agent-reach (handles pipx / venv installs):
|
||||
AGENT_REACH_PYTHON=$(python3 -c "import agent_reach, sys; print(sys.executable)" 2>/dev/null || echo python3)
|
||||
$AGENT_REACH_PYTHON -c "
|
||||
import asyncio
|
||||
from miku_ai import get_wexin_article
|
||||
async def s():
|
||||
for a in await get_wexin_article('query', 5):
|
||||
print(f'{a[\"title\"]} | {a[\"url\"]}')
|
||||
asyncio.run(s())
|
||||
"
|
||||
```
|
||||
|
||||
**Read** (Camoufox — bypasses WeChat anti-bot):
|
||||
```bash
|
||||
cd ~/.agent-reach/tools/wechat-article-for-ai && python3 main.py "https://mp.weixin.qq.com/s/ARTICLE_ID"
|
||||
```
|
||||
|
||||
> WeChat articles cannot be read with Jina Reader or curl. Use Camoufox.
|
||||
|
||||
## Weibo (mcporter)
|
||||
|
||||
```bash
|
||||
# Trending topics
|
||||
mcporter call 'weibo.get_trendings(limit: 20)'
|
||||
|
||||
# Search users
|
||||
mcporter call 'weibo.search_users(keyword: "Lei Jun", limit: 10)'
|
||||
|
||||
# Get a user profile
|
||||
mcporter call 'weibo.get_profile(uid: "1195230310")'
|
||||
|
||||
# Get a user's feed
|
||||
mcporter call 'weibo.get_feeds(uid: "1195230310", limit: 20)'
|
||||
|
||||
# Get a user's hot posts
|
||||
mcporter call 'weibo.get_hot_feeds(uid: "1195230310", limit: 10)'
|
||||
|
||||
# Search post content
|
||||
mcporter call 'weibo.search_content(keyword: "artificial intelligence", limit: 20)'
|
||||
|
||||
# Search topics
|
||||
mcporter call 'weibo.search_topics(keyword: "AI", limit: 10)'
|
||||
|
||||
# Get post comments
|
||||
mcporter call 'weibo.get_comments(mid: "5099916367123456", limit: 50)'
|
||||
|
||||
# Get fans
|
||||
mcporter call 'weibo.get_fans(uid: "1195230310", limit: 20)'
|
||||
|
||||
# Get followings
|
||||
mcporter call 'weibo.get_followers(uid: "1195230310", limit: 20)'
|
||||
```
|
||||
|
||||
> Zero config. No login needed. Uses the mobile API with auto-generated visitor cookies.
|
||||
|
||||
## Xiaoyuzhou Podcast (groq-whisper + ffmpeg)
|
||||
|
||||
```bash
|
||||
# Transcribe a single podcast episode (outputs text to /tmp/)
|
||||
~/.agent-reach/tools/xiaoyuzhou/transcribe.sh "https://www.xiaoyuzhoufm.com/episode/EPISODE_ID"
|
||||
```
|
||||
|
||||
> Requires `ffmpeg` and a Groq API key (free).
|
||||
> Configure the key with `agent-reach configure groq-key YOUR_KEY`.
|
||||
> On first run, install the tools with `agent-reach install --env=auto`.
|
||||
> Run `agent-reach doctor` to check status.
|
||||
> Output Markdown files are saved to `/tmp/` by default.
|
||||
|
||||
## LinkedIn (mcporter)
|
||||
|
||||
```bash
|
||||
mcporter call 'linkedin.get_person_profile(linkedin_url: "https://linkedin.com/in/username")'
|
||||
mcporter call 'linkedin.search_people(keyword: "AI engineer", limit: 10)'
|
||||
```
|
||||
|
||||
Fallback: `curl -s "https://r.jina.ai/https://linkedin.com/in/username"`
|
||||
|
||||
## V2EX (public API)
|
||||
|
||||
```bash
|
||||
# Hot topics
|
||||
# V2EX hot topics
|
||||
curl -s "https://www.v2ex.com/api/topics/hot.json" -H "User-Agent: agent-reach/1.0"
|
||||
|
||||
# Topics in a node (node_name examples: python, tech, jobs, qna)
|
||||
curl -s "https://www.v2ex.com/api/topics/show.json?node_name=python&page=1" -H "User-Agent: agent-reach/1.0"
|
||||
|
||||
# Topic details (extract topic_id from URLs like https://www.v2ex.com/t/1234567)
|
||||
curl -s "https://www.v2ex.com/api/topics/show.json?id=TOPIC_ID" -H "User-Agent: agent-reach/1.0"
|
||||
|
||||
# Topic replies
|
||||
curl -s "https://www.v2ex.com/api/replies/show.json?topic_id=TOPIC_ID&page=1" -H "User-Agent: agent-reach/1.0"
|
||||
|
||||
# User profile
|
||||
curl -s "https://www.v2ex.com/api/members/show.json?username=USERNAME" -H "User-Agent: agent-reach/1.0"
|
||||
# Bilibili search (bili-cli, no login needed)
|
||||
bili search "query" --type video -n 5
|
||||
```
|
||||
|
||||
Python example (`V2EXChannel`):
|
||||
## Login-backed platforms (pick by doctor's active_backend)
|
||||
|
||||
```python
|
||||
from agent_reach.channels.v2ex import V2EXChannel
|
||||
```bash
|
||||
# Twitter search (twitter-cli preferred; retry chain in social.md)
|
||||
twitter search "query" -n 10
|
||||
|
||||
ch = V2EXChannel()
|
||||
# Reddit (NO zero-config path — OpenCLI or rdt-cli, login required)
|
||||
opencli reddit search "query" -f yaml # desktop
|
||||
rdt search "query" --limit 10 # legacy/server
|
||||
|
||||
# Get hot topics (default 20 items)
|
||||
# Returned fields: id, title, url, replies, node_name, node_title, content(first 200 chars), created
|
||||
topics = ch.get_hot_topics(limit=10)
|
||||
for t in topics:
|
||||
print(f"[{t['node_title']}] {t['title']} ({t['replies']} replies) {t['url']}")
|
||||
print(f" id={t['id']} created={t['created']}")
|
||||
|
||||
# Get latest topics for a specific node
|
||||
# Returned fields: id, title, url, replies, node_name, node_title, content(first 200 chars), created
|
||||
node_topics = ch.get_node_topics("python", limit=5)
|
||||
for t in node_topics:
|
||||
print(t["id"], t["title"], t["url"])
|
||||
|
||||
# Get one topic plus replies
|
||||
# Returned fields: id, title, url, content, replies_count, node_name, node_title,
|
||||
# author, created, replies (list of {author, content, created})
|
||||
topic = ch.get_topic(1234567)
|
||||
print(topic["title"], "—", topic["author"])
|
||||
for r in topic["replies"]:
|
||||
print(f" {r['author']}: {r['content'][:80]}")
|
||||
|
||||
# Get user info
|
||||
# Returned fields: id, username, url, website, twitter, psn, github, btc, location, bio, avatar, created
|
||||
user = ch.get_user("Livid")
|
||||
print(user["username"], user["bio"], user["github"])
|
||||
|
||||
# Search (not supported by the public V2EX API; returns guidance instead)
|
||||
result = ch.search("asyncio")
|
||||
print(result[0]["error"]) # Use built-in site search or the Exa channel instead
|
||||
# XiaoHongShu (desktop prefers OpenCLI)
|
||||
opencli xiaohongshu search "query" -f yaml
|
||||
```
|
||||
|
||||
> No auth required. Results are public JSON. V2EX node names are listed at https://www.v2ex.com/planes
|
||||
## Environment check
|
||||
|
||||
## Xueqiu (public API)
|
||||
|
||||
```python
|
||||
from agent_reach.channels.xueqiu import XueqiuChannel
|
||||
|
||||
ch = XueqiuChannel()
|
||||
|
||||
# Get stock quotes (symbol examples: SH600519 mainland China, SZ000858 Shenzhen, AAPL US, 00700 HK)
|
||||
# Returned fields: symbol, name, current, percent, chg, high, low, open, last_close,
|
||||
# volume, amount, market_capital, turnover_rate, pe_ttm, timestamp
|
||||
quote = ch.get_stock_quote("AAPL")
|
||||
print(f"{quote['name']} ({quote['symbol']}): {quote['current']} ({quote['percent']}%)")
|
||||
|
||||
# Search stocks
|
||||
# Returned fields: symbol, name, exchange
|
||||
stocks = ch.search_stock("Apple", limit=5)
|
||||
for s in stocks:
|
||||
print(f"{s['name']} ({s['symbol']}) - {s['exchange']}")
|
||||
|
||||
# Hot posts
|
||||
# Returned fields: id, title, text(first 200 chars), author, likes, url
|
||||
posts = ch.get_hot_posts(limit=10)
|
||||
for p in posts:
|
||||
print(f"{p['author']}: {p['text'][:50]}... ({p['likes']} likes)")
|
||||
|
||||
# Hot stocks (stock_type=10 popularity ranking, stock_type=12 watchlist ranking)
|
||||
# Returned fields: symbol, name, current, percent, rank
|
||||
hot = ch.get_hot_stocks(limit=10, stock_type=10)
|
||||
for s in hot:
|
||||
print(f"#{s['rank']} {s['name']} ({s['symbol']}): {s['current']} ({s['percent']}%)")
|
||||
```bash
|
||||
# Channel availability + which backend serves each platform
|
||||
agent-reach doctor --json
|
||||
```
|
||||
|
||||
> No login required. Agent Reach auto-fetches session cookies, and all public APIs can be used directly.
|
||||
## Workspace rules
|
||||
|
||||
## RSS (feedparser)
|
||||
**Never create files in the agent workspace.** Use `/tmp/` for temporary
|
||||
output and `~/.agent-reach/` for persistent data.
|
||||
|
||||
```python
|
||||
python3 -c "
|
||||
import feedparser
|
||||
for e in feedparser.parse('FEED_URL').entries[:5]:
|
||||
print(f'{e.title} — {e.link}')
|
||||
"
|
||||
```
|
||||
## Detailed references
|
||||
|
||||
## Troubleshooting
|
||||
Read the matching file when you need specifics (commands above cover the
|
||||
common cases; references hold per-backend command groups, caveats, retry
|
||||
chains — note: reference docs are written in Chinese, commands are universal):
|
||||
|
||||
- **Channel not working?** Run `agent-reach doctor` — it shows status and fix instructions.
|
||||
- **Twitter fetch failed?** Ensure `undici` is installed: `npm install -g undici`. Configure a proxy if needed: `agent-reach configure proxy URL`.
|
||||
- [Search](references/search.md) — Exa AI search
|
||||
- [Social](references/social.md) — XiaoHongShu, Twitter, Bilibili, V2EX, Reddit (multi-backend groups)
|
||||
- [Career](references/career.md) — LinkedIn
|
||||
- [Dev](references/dev.md) — GitHub CLI
|
||||
- [Web](references/web.md) — Jina Reader, RSS
|
||||
- [Video](references/video.md) — YouTube, Bilibili, Xiaoyuzhou
|
||||
|
||||
## Setting Up a Channel ("help me configure XXX")
|
||||
## Configure a channel
|
||||
|
||||
If a channel needs setup (cookies, Docker, etc.), fetch the install guide:
|
||||
If a channel needs setup, fetch the install guide:
|
||||
https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
|
||||
The user only provides cookies. Everything else is your job.
|
||||
The user only provides cookies / one extension click; the agent does the rest.
|
||||
|
||||
@@ -1,82 +1,67 @@
|
||||
# 社交媒体 & 社区
|
||||
|
||||
小红书、抖音、Twitter/X、微博、B站、V2EX、Reddit。
|
||||
小红书、Twitter/X、B站、V2EX、Reddit。
|
||||
|
||||
## 小红书 / XiaoHongShu (xhs-cli)
|
||||
## 小红书 / XiaoHongShu(多后端)
|
||||
|
||||
### 稳定可用的命令
|
||||
小红书有三个后端,**先跑 `agent-reach doctor --json` 看 xiaohongshu 的 `active_backend` 是哪个**,再用对应命令组。
|
||||
|
||||
### 后端 A:OpenCLI(桌面首选,复用浏览器登录态)
|
||||
|
||||
```bash
|
||||
# 搜索笔记(推荐入口)
|
||||
xhs search "query"
|
||||
# 搜索笔记
|
||||
opencli xiaohongshu search "query" -f yaml
|
||||
|
||||
# 阅读笔记详情(必须用搜索结果中的 URL 或 ID,不能裸 note_id)
|
||||
xhs read NOTE_ID_OR_URL
|
||||
# 读笔记正文+互动数据(用搜索结果里的完整 URL,含 xsec_token)
|
||||
opencli xiaohongshu note "NOTE_URL" -f yaml
|
||||
|
||||
# 查看评论
|
||||
xhs comments NOTE_ID_OR_URL
|
||||
# 评论(支持楼中楼)
|
||||
opencli xiaohongshu comments NOTE_ID -f yaml
|
||||
|
||||
# 浏览热门
|
||||
xhs hot
|
||||
# 首页推荐 feed
|
||||
opencli xiaohongshu feed -f yaml
|
||||
|
||||
# 推荐 feed
|
||||
xhs feed
|
||||
# 用户主页公开笔记
|
||||
opencli xiaohongshu user USER_ID -f yaml
|
||||
```
|
||||
|
||||
### 已知不稳定的命令(v0.6.4)
|
||||
> 要求 Chrome 打开且装了 OpenCLI 扩展。报 AUTH_REQUIRED 说明浏览器里没登录小红书,让用户在 Chrome 里登录一次即可。
|
||||
|
||||
### 后端 B:xiaohongshu-mcp(服务器场景)
|
||||
|
||||
```bash
|
||||
# 以下命令当前可能返回 API error,谨慎使用:
|
||||
xhs user USER_ID # 可能返回 {code: -1}
|
||||
xhs user-posts USER_ID # 可能返回 {code: -1}
|
||||
xhs favorites # 可能返回 API error
|
||||
# 未登录时:先查状态,再取二维码给用户扫
|
||||
mcporter call 'xiaohongshu.check_login_status()' --timeout 120000
|
||||
mcporter call 'xiaohongshu.get_login_qrcode()' --timeout 120000
|
||||
|
||||
# 搜索
|
||||
mcporter call 'xiaohongshu.search_feeds(keyword: "query")' --timeout 120000
|
||||
|
||||
# 笔记详情+评论(feed_id 和 xsec_token 从搜索结果取)
|
||||
mcporter call 'xiaohongshu.get_feed_detail(feed_id: "...", xsec_token: "...")' --timeout 120000
|
||||
```
|
||||
|
||||
### 重要注意事项
|
||||
> 首次调用会自动下载约 150MB 无头浏览器,务必带 `--timeout 120000`。未登录时 search 会挂死,先 check_login_status。
|
||||
|
||||
> **安装**: `pipx install xiaohongshu-cli`,然后 `xhs login`(自动从浏览器提取 Cookie)。
|
||||
### 后端 C:xhs-cli(存量备选,上游 2026-03 起停更)
|
||||
|
||||
```bash
|
||||
xhs search "query" # 搜索
|
||||
xhs read NOTE_ID_OR_URL # 读笔记(必须用搜索结果中的 URL/ID,不能裸 note_id)
|
||||
xhs comments NOTE_ID_OR_URL # 评论
|
||||
xhs hot # 热门
|
||||
xhs feed # 推荐
|
||||
```
|
||||
|
||||
> 已知不稳定:`xhs user` / `xhs user-posts` / `xhs favorites` 可能返回 API error(上游停更无人修)。新装用户建议直接走后端 A/B。
|
||||
|
||||
### 通用注意事项
|
||||
|
||||
> **xsec_token 限制**: 小红书强制 xsec_token 机制,**不能直接用裸 note_id 去读**。正确流程:先搜索/feed 拿结果,再用结果中的完整 URL/ID 去读。三个后端都一样。
|
||||
>
|
||||
> **xsec_token 限制**: 小红书强制 xsec_token 机制,**不能直接用裸 note_id 去读**。正确流程是:先 `xhs search` 或 `xhs feed` 获取结果,再用结果中的 URL/ID 去 `xhs read`。直接构造 note_id 会被拦截。
|
||||
> **频率控制**: 高频请求(批量搜索、深翻评论)会触发验证码,平台限制无法绕过。每次操作间隔 2-3 秒。
|
||||
>
|
||||
> **频率控制**: 高频请求(批量搜索、深翻评论)会触发验证码,这是平台限制无法绕过。建议每次操作间隔 2-3 秒。
|
||||
>
|
||||
> **POST 操作风险**: 发帖(post)、评论(comment)、点赞(like) 等写操作在 v0.6.x 可能因签名问题返回 406。如需使用,建议降级到 v0.3.5 (`pipx install xiaohongshu-cli==0.3.5`)。
|
||||
|
||||
## 抖音 / Douyin
|
||||
|
||||
### 安装与配置
|
||||
|
||||
`douyin-mcp-server` 是 **stdio 模式**的 MCP server,需先安装再注册到 mcporter:
|
||||
|
||||
```bash
|
||||
# 1. 安装
|
||||
pipx install douyin-mcp-server
|
||||
|
||||
# 2. 查找安装路径
|
||||
pipx runpip douyin-mcp-server show -f 2>/dev/null | grep "Location" \
|
||||
|| find ~/.local -name "douyin-mcp-server" 2>/dev/null | head -1
|
||||
|
||||
# 3. 注册到 mcporter(使用 stdio 模式,将路径替换为上一步的输出)
|
||||
mcporter config add douyin --command "/path/to/douyin-mcp-server" --scope home
|
||||
```
|
||||
|
||||
> **注意**:`agent-reach install --channels douyin` 暂不支持抖音渠道(抖音在"可选渠道待解锁"列表)。
|
||||
> HTTP 模式(`mcporter config add douyin http://localhost:18070/mcp`)**无法正常工作**,请使用上方 stdio 方式。
|
||||
|
||||
### 用法
|
||||
|
||||
```bash
|
||||
# 解析视频信息
|
||||
mcporter call 'douyin.parse_douyin_video_info(share_link: "https://v.douyin.com/xxx/")'
|
||||
|
||||
# 获取无水印下载链接
|
||||
mcporter call 'douyin.get_douyin_download_link(share_link: "https://v.douyin.com/xxx/")'
|
||||
|
||||
# 提取视频文案
|
||||
mcporter call 'douyin.extract_douyin_text(share_link: "https://v.douyin.com/xxx/")'
|
||||
```
|
||||
|
||||
> **无需登录**
|
||||
> **写操作(发帖/评论/点赞)**: 建议只读。xhs-cli v0.6.x 写操作可能因签名问题返回 406。
|
||||
|
||||
## Twitter/X (twitter-cli)
|
||||
|
||||
@@ -104,12 +89,18 @@ twitter user @username
|
||||
```bash
|
||||
# 搜索推文(Twitter 频繁改 GraphQL 端点,可能 404)
|
||||
twitter search "query" -n 10
|
||||
# 如果 search 返回 404,升级 twitter-cli:pipx upgrade twitter-cli
|
||||
|
||||
# likes(2024 年后只能看自己的,平台限制)
|
||||
twitter likes
|
||||
```
|
||||
|
||||
### search 失败时的重试链(按序执行,成功即停)
|
||||
|
||||
1. 直接重试一次(偶发失败常见):`twitter search "query" -n 10`
|
||||
2. 升级后再试:`pipx upgrade twitter-cli && twitter search "query" -n 10`
|
||||
3. 换 OpenCLI 备选(桌面,复用浏览器登录态):`opencli twitter search "query" -f yaml`
|
||||
4. 都不行就改用 `twitter feed` / `twitter user-posts @somebody` 等稳定命令绕路
|
||||
|
||||
### 重要注意事项
|
||||
|
||||
> **安装**: `pipx install twitter-cli`(确保 v0.8.5+)
|
||||
@@ -118,30 +109,25 @@ twitter likes
|
||||
>
|
||||
> **IP 风控**: 不要在 VPS/数据中心 IP 上频繁调用,尤其是 followers/following,有封号风险。使用住宅代理或本地环境。
|
||||
>
|
||||
> **search 可能失效**: Twitter 频繁修改 GraphQL API,search 命令可能随时返回 404。如遇到,先 `pipx upgrade twitter-cli`。如果最新版仍不行,说明上游还没跟上 Twitter 的改动,用 `twitter feed` 替代。
|
||||
> **OpenCLI 备选**: 桌面装了 OpenCLI 的话,`opencli twitter search/article/user-posts -f yaml` 全套可用(浏览器登录态,无需 cookie 环境变量)。
|
||||
>
|
||||
> **输出格式**: 建议用 `--yaml` 或 `--json` 获得结构化输出,对 AI agent 更友好。
|
||||
|
||||
## 微博 / Weibo
|
||||
|
||||
```bash
|
||||
# 使用 Jina Reader 读取
|
||||
curl -s "https://r.jina.ai/https://weibo.com/USER_ID/POST_ID"
|
||||
```
|
||||
|
||||
> 微博主要通过网页抓取,推荐使用通用网页读取方式。
|
||||
|
||||
## B站 / Bilibili
|
||||
|
||||
```bash
|
||||
# 获取视频元数据
|
||||
yt-dlp --dump-json "https://www.bilibili.com/video/BVxxx"
|
||||
> ⚠️ **不要用 yt-dlp 读 B站**(风控已全面 412 拦截,实测无解)。用 bili-cli / OpenCLI。
|
||||
|
||||
# 下载字幕
|
||||
yt-dlp --write-sub --write-auto-sub --sub-lang "zh-Hans,zh,en" --convert-subs vtt --skip-download -o "/tmp/%(id)s" "URL"
|
||||
```bash
|
||||
# 搜索 / 热门 / 视频详情(bili-cli,只读无需登录)
|
||||
bili search "query" --type video -n 5
|
||||
bili hot -n 10
|
||||
bili video BVxxx
|
||||
|
||||
# 字幕(OpenCLI,需桌面 Chrome)
|
||||
opencli bilibili subtitle BVxxx
|
||||
```
|
||||
|
||||
> **注意**: 服务器 IP 可能遇到 412 错误。使用 `--cookies-from-browser chrome` 或配置代理。
|
||||
> 详细命令(音频转写、API 直连兜底)见 [references/video.md](video.md)。
|
||||
|
||||
## V2EX (公开 API)
|
||||
|
||||
@@ -204,25 +190,43 @@ user = ch.get_user("Livid")
|
||||
|
||||
> **节点列表**: https://www.v2ex.com/planes
|
||||
|
||||
## Reddit (rdt-cli)
|
||||
## Reddit(多后端,必须登录态)
|
||||
|
||||
**Reddit 没有零配置路径**:匿名 `.json` 端点已被封(403),官方 API 自 2025-11 起人工审批基本不批。两个后端都靠登录态,先跑 `agent-reach doctor --json` 看 reddit 的 `active_backend`。中国大陆访问需代理。
|
||||
|
||||
### 后端 A:OpenCLI(桌面首选,复用浏览器登录态)
|
||||
|
||||
```bash
|
||||
# 搜索帖子
|
||||
rdt search "query" --limit 10
|
||||
opencli reddit search "query" -f yaml
|
||||
|
||||
# 读帖子全文 + 评论
|
||||
rdt read POST_ID
|
||||
opencli reddit read POST_ID -f yaml
|
||||
|
||||
# 浏览 subreddit
|
||||
rdt sub python --limit 20
|
||||
# 浏览 subreddit / 热门 / Popular
|
||||
opencli reddit subreddit LocalLLaMA -f yaml
|
||||
opencli reddit hot -f yaml
|
||||
opencli reddit popular -f yaml
|
||||
|
||||
# 浏览热门
|
||||
rdt popular --limit 10
|
||||
|
||||
# 浏览 /r/all
|
||||
rdt all --limit 10
|
||||
# subreddit 元信息(订阅数、简介)
|
||||
opencli reddit subreddit-info LocalLLaMA -f yaml
|
||||
```
|
||||
|
||||
> **安装**: `pipx install rdt-cli`(确保 v0.4.2+)。无需登录即可搜索和阅读。
|
||||
> 需要登录的功能:`rdt feed --subs-only`(订阅列表)、`rdt saved`(收藏)。
|
||||
> 要求 Chrome 打开且浏览器里登录过 reddit.com。
|
||||
|
||||
### 后端 B:rdt-cli(存量/服务器备选,上游 2026-03 起停更)
|
||||
|
||||
```bash
|
||||
rdt search "query" --limit 10 # 搜索帖子
|
||||
rdt read POST_ID # 读帖子全文 + 评论
|
||||
rdt sub python --limit 20 # 浏览 subreddit
|
||||
rdt popular --limit 10 # 浏览热门
|
||||
rdt all --limit 10 # 浏览 /r/all
|
||||
```
|
||||
|
||||
> **安装**: `pipx install 'git+https://github.com/public-clis/rdt-cli.git'`(PyPI 版本落后,需从 GitHub 装 v0.4.2+)。先 `rdt login` 才能搜索和阅读(服务器无浏览器时手动写 Cookie,见 doctor 提示)。
|
||||
> 建议使用 `--yaml` 输出,对 AI agent 更友好。
|
||||
|
||||
### 高级选项:官方 API + PRAW(仅限已有凭证的用户)
|
||||
|
||||
2025-11 前注册过 Reddit script app(持有 client_id/client_secret)的用户可以用 PRAW 走官方 API(100 QPM 免费)。新申请需人工审批且个人项目基本不批,**不要推荐新用户走这条路**。
|
||||
|
||||
@@ -39,45 +39,71 @@ yt-dlp --dump-json "ytsearch5:query"
|
||||
> **字幕注意**: 手动上传的字幕提取可靠;自动生成字幕可能存在行间重复,需后处理。
|
||||
> **评论注意**: `--write-comments` 基于网页抓取(非 YouTube Data API),部分评论可能丢失。
|
||||
|
||||
## B站 / Bilibili (yt-dlp + bili-cli)
|
||||
|
||||
### 视频元数据 (yt-dlp)
|
||||
### 无字幕兜底:Whisper 音频转写
|
||||
|
||||
```bash
|
||||
yt-dlp --dump-json "https://www.bilibili.com/video/BVxxx"
|
||||
# 视频没有字幕时的兜底:下载音频并用 Whisper 转写(Groq 免费 key 即可)
|
||||
agent-reach transcribe "https://www.youtube.com/watch?v=VIDEO_ID"
|
||||
agent-reach transcribe ./local_audio.mp3 -o /tmp/transcript.txt
|
||||
```
|
||||
|
||||
### 字幕 (yt-dlp)
|
||||
> 需要先配置 key:`agent-reach configure groq-key gsk_xxx`(免费,console.groq.com)
|
||||
> 或 `agent-reach configure openai-key sk-xxx`。默认 auto 模式:groq 失败自动降级 openai。
|
||||
|
||||
## B站 / Bilibili(bili-cli 为主,OpenCLI 补字幕)
|
||||
|
||||
> ⚠️ **不要用 yt-dlp 读 B站**:B站风控已全面 412 拦截 yt-dlp(实测最新版、直连/代理/带 Cookie 全部无效)。yt-dlp 只用于 YouTube。
|
||||
|
||||
### 视频详情/搜索/热门/排行 (bili-cli,只读无需登录)
|
||||
|
||||
```bash
|
||||
yt-dlp --write-sub --write-auto-sub --sub-lang "zh-Hans,zh,en" --convert-subs vtt --skip-download -o "/tmp/%(id)s" "URL"
|
||||
```
|
||||
# 视频详情(标题/UP主/时长/播放互动数据/字幕可用性)
|
||||
bili video BVxxx
|
||||
|
||||
### 搜索/热门/排行 (bili-cli)
|
||||
|
||||
```bash
|
||||
# 搜索视频
|
||||
bili search "query" --type video -n 5
|
||||
|
||||
# 热门视频
|
||||
# 热门视频 / 排行榜
|
||||
bili hot -n 10
|
||||
|
||||
# 排行榜
|
||||
bili rank -n 10
|
||||
|
||||
# 下载音频并切分为 ASR-ready WAV(无字幕时配合 agent-reach transcribe 转写)
|
||||
bili audio BVxxx
|
||||
```
|
||||
|
||||
> **412 风控**: 海外 IP 必须提供 Cookie(`--cookies-from-browser chrome` 或 `--cookies /path/to/cookies.txt`),国内 IP 一般不受影响。
|
||||
> **安装 bili-cli**: `pipx install bilibili-cli`,然后 `bili login` 扫码登录。
|
||||
### 字幕 (OpenCLI,需要桌面 Chrome)
|
||||
|
||||
```bash
|
||||
# 字幕逐句带时间轴
|
||||
opencli bilibili subtitle BVxxx
|
||||
|
||||
# OpenCLI 也能搜索/读视频元数据(备选)
|
||||
opencli bilibili search "query" -f yaml
|
||||
opencli bilibili video BVxxx -f yaml
|
||||
```
|
||||
|
||||
### 零配置兜底:搜索 API 直连
|
||||
|
||||
```bash
|
||||
UA="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36"
|
||||
curl -s -c /tmp/bili_ck.txt -o /dev/null -A "$UA" "https://www.bilibili.com/"
|
||||
curl -s -b /tmp/bili_ck.txt -A "$UA" -e "https://www.bilibili.com/" \
|
||||
"https://api.bilibili.com/x/web-interface/search/all/v2?keyword=QUERY&page=1"
|
||||
```
|
||||
|
||||
> **安装 bili-cli**: `pipx install bilibili-cli`(上游 2026-03 起停更但实测健康;只读场景无需登录,`bili login` 扫码可解锁动态/收藏等个人功能)。
|
||||
|
||||
## 小宇宙播客 / Xiaoyuzhou Podcast
|
||||
|
||||
### 转录单集播客
|
||||
### 转录单集播客(可选 --polish 增强标点)
|
||||
|
||||
```bash
|
||||
# 输出 Markdown 文件到 /tmp/
|
||||
~/.agent-reach/tools/xiaoyuzhou/transcribe.sh "https://www.xiaoyuzhoufm.com/episode/EPISODE_ID"
|
||||
# 输出 Markdown 文件到 /tmp/。--polish 让 Llama 3.3 70B 给文稿补中文标点+合理分段
|
||||
~/.agent-reach/tools/xiaoyuzhou/transcribe.sh --polish "https://www.xiaoyuzhoufm.com/episode/EPISODE_ID"
|
||||
```
|
||||
|
||||
> 转写 prompt 已要求 Whisper 输出中文标点;若标点效果仍不理想,可加 `--polish` 用 Groq 上免费的 Llama 3.3 70B 补标点+合理分段(9 分钟播客约多 ~7 秒)。每次转写多一轮 LLM 调用,按需使用。
|
||||
|
||||
### 前置要求
|
||||
|
||||
1. **ffmpeg**: `brew install ffmpeg`
|
||||
@@ -93,23 +119,12 @@ agent-reach doctor
|
||||
|
||||
> 输出 Markdown 文件默认保存到 `/tmp/`。
|
||||
|
||||
## 抖音视频解析
|
||||
|
||||
```bash
|
||||
# 解析视频信息
|
||||
mcporter call 'douyin.parse_douyin_video_info(share_link: "https://v.douyin.com/xxx/")'
|
||||
|
||||
# 获取无水印下载链接
|
||||
mcporter call 'douyin.get_douyin_download_link(share_link: "https://v.douyin.com/xxx/")'
|
||||
```
|
||||
|
||||
> 详见 [social.md](social.md#抖音--douyin)
|
||||
|
||||
## 选择指南
|
||||
|
||||
| 场景 | 推荐工具 |
|
||||
|-----|---------|
|
||||
| YouTube 字幕 | yt-dlp |
|
||||
| B站字幕 | yt-dlp |
|
||||
| B站视频详情/搜索 | bili-cli |
|
||||
| B站字幕 | opencli bilibili subtitle |
|
||||
| 播客转录 | 小宇宙 transcribe.sh |
|
||||
| 抖音视频解析 | douyin MCP |
|
||||
| 无字幕音视频 | agent-reach transcribe(B站音频先 `bili audio`) |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# 网页阅读
|
||||
|
||||
通用网页、微信公众号、RSS。
|
||||
通用网页、RSS。
|
||||
|
||||
## 通用网页 (Jina Reader)
|
||||
|
||||
@@ -29,30 +29,6 @@ mcporter call 'web-reader.webReader(url: "https://example.com", return_format: "
|
||||
|
||||
**适用场景**: 需要更精确控制输出格式时使用。
|
||||
|
||||
## 微信公众号 / WeChat Articles
|
||||
|
||||
### 搜索公众号文章(通过 Exa)
|
||||
|
||||
```bash
|
||||
# 搜索微信公众号文章
|
||||
mcporter call 'exa.web_search_exa(query: "搜索关键词", numResults: 5, includeDomains: ["mp.weixin.qq.com"])'
|
||||
```
|
||||
|
||||
### 阅读公众号文章全文(通过 Exa)
|
||||
|
||||
```bash
|
||||
# 抓取文章全文
|
||||
mcporter call 'exa.crawling_exa(urls: ["https://mp.weixin.qq.com/s/ARTICLE_ID"], maxCharacters: 10000)'
|
||||
```
|
||||
|
||||
### 可选:Camoufox 阅读(反爬更强)
|
||||
|
||||
```bash
|
||||
cd ~/.agent-reach/tools/wechat-article-for-ai && python3 main.py "https://mp.weixin.qq.com/s/ARTICLE_ID"
|
||||
```
|
||||
|
||||
> **注意**: Jina Reader 无法读取微信文章(被 CAPTCHA 拦截),推荐用 Exa。
|
||||
|
||||
## RSS (feedparser)
|
||||
|
||||
```python
|
||||
@@ -71,6 +47,4 @@ for e in feedparser.parse('FEED_URL').entries[:5]:
|
||||
|-----|---------|
|
||||
| 通用网页 | Jina Reader (`curl r.jina.ai`) |
|
||||
| 需要图片/格式控制 | web-reader MCP |
|
||||
| 微信公众号 | Exa (搜索+阅读) / Camoufox (可选阅读) |
|
||||
| RSS 订阅 | feedparser |
|
||||
| 微博/知乎等 | Jina Reader |
|
||||
|
||||
@@ -0,0 +1,261 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Whisper audio transcription with Groq → OpenAI fallback.
|
||||
|
||||
Downloads audio (yt-dlp), compresses + chunks (ffmpeg), and posts to a
|
||||
Whisper-compatible API. Defaults to Groq's free `whisper-large-v3` and falls
|
||||
back to OpenAI's `whisper-1` on HTTP error.
|
||||
|
||||
Public entry point:
|
||||
transcribe(source, *, provider="auto", out_dir=None, config=None) -> str
|
||||
|
||||
Designed to be importable from channels (e.g. YouTubeChannel.transcribe).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
|
||||
import requests
|
||||
|
||||
from agent_reach.config import Config
|
||||
|
||||
# Whisper API limit is 25MB; leave headroom for multipart overhead.
|
||||
SIZE_LIMIT_BYTES = 24 * 1024 * 1024
|
||||
CHUNK_SECONDS = 600 # 10 min — small enough that boundary cuts rarely lose meaning
|
||||
|
||||
PROVIDERS = {
|
||||
"groq": {
|
||||
"endpoint": "https://api.groq.com/openai/v1/audio/transcriptions",
|
||||
"model": "whisper-large-v3",
|
||||
"key_field": "groq_api_key",
|
||||
},
|
||||
"openai": {
|
||||
"endpoint": "https://api.openai.com/v1/audio/transcriptions",
|
||||
"model": "whisper-1",
|
||||
"key_field": "openai_api_key",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class TranscribeError(RuntimeError):
|
||||
"""Raised when transcription cannot complete."""
|
||||
|
||||
|
||||
class MissingDependency(TranscribeError):
|
||||
"""Raised when a required external binary is missing."""
|
||||
|
||||
|
||||
class NoProviderConfigured(TranscribeError):
|
||||
"""Raised when no provider has an API key configured."""
|
||||
|
||||
|
||||
def _require(binary: str) -> None:
|
||||
if not shutil.which(binary):
|
||||
raise MissingDependency(f"{binary} not found in PATH")
|
||||
|
||||
|
||||
def _run(cmd: List[str], timeout: int = 600) -> None:
|
||||
"""Run a subprocess, raising TranscribeError on nonzero exit or timeout.
|
||||
|
||||
cmd carries user-supplied URLs/paths into yt-dlp/ffmpeg — a stalled
|
||||
network read or a hung probe must not block the CLI forever.
|
||||
"""
|
||||
try:
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
|
||||
except subprocess.TimeoutExpired:
|
||||
raise TranscribeError(f"{cmd[0]} timed out after {timeout}s")
|
||||
if proc.returncode != 0:
|
||||
raise TranscribeError(
|
||||
f"{cmd[0]} failed (exit {proc.returncode}): {proc.stderr.strip()[:300]}"
|
||||
)
|
||||
|
||||
|
||||
def download_audio(url: str, out_dir: Path) -> Path:
|
||||
"""Download audio with yt-dlp into out_dir; return the resulting file path."""
|
||||
_require("yt-dlp")
|
||||
template = out_dir / "source.%(ext)s"
|
||||
_run(
|
||||
[
|
||||
"yt-dlp",
|
||||
"-x",
|
||||
"--audio-format",
|
||||
"m4a",
|
||||
"--audio-quality",
|
||||
"0",
|
||||
"-o",
|
||||
str(template),
|
||||
url,
|
||||
],
|
||||
timeout=1800, # long podcasts over slow networks — generous but bounded
|
||||
)
|
||||
files = sorted(out_dir.glob("source.*"))
|
||||
if not files:
|
||||
raise TranscribeError("yt-dlp produced no output file")
|
||||
return files[0]
|
||||
|
||||
|
||||
def compress_audio(src: Path, out_dir: Path) -> Path:
|
||||
"""Re-encode to mono / 16kHz / 32kbps m4a — keeps most content under 25MB."""
|
||||
_require("ffmpeg")
|
||||
dst = out_dir / "compressed.m4a"
|
||||
_run(
|
||||
[
|
||||
"ffmpeg",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-y",
|
||||
"-i",
|
||||
str(src),
|
||||
"-vn",
|
||||
"-ac",
|
||||
"1",
|
||||
"-ar",
|
||||
"16000",
|
||||
"-b:a",
|
||||
"32k",
|
||||
str(dst),
|
||||
]
|
||||
)
|
||||
return dst
|
||||
|
||||
|
||||
def chunk_audio(src: Path, out_dir: Path, segment_seconds: int = CHUNK_SECONDS) -> List[Path]:
|
||||
"""Split src into segments. Re-encodes each segment so cuts align to keyframes."""
|
||||
_require("ffmpeg")
|
||||
pattern = out_dir / "chunk_%03d.m4a"
|
||||
_run(
|
||||
[
|
||||
"ffmpeg",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-y",
|
||||
"-i",
|
||||
str(src),
|
||||
"-f",
|
||||
"segment",
|
||||
"-segment_time",
|
||||
str(segment_seconds),
|
||||
"-ac",
|
||||
"1",
|
||||
"-ar",
|
||||
"16000",
|
||||
"-b:a",
|
||||
"32k",
|
||||
str(pattern),
|
||||
]
|
||||
)
|
||||
chunks = sorted(out_dir.glob("chunk_*.m4a"))
|
||||
if not chunks:
|
||||
raise TranscribeError("ffmpeg produced no chunks")
|
||||
return chunks
|
||||
|
||||
|
||||
def _provider_key(provider: str, config: Config) -> Optional[str]:
|
||||
field = PROVIDERS[provider]["key_field"]
|
||||
val = config.get(field)
|
||||
return val or None
|
||||
|
||||
|
||||
def transcribe_chunk(
|
||||
chunk: Path,
|
||||
provider: str,
|
||||
*,
|
||||
config: Optional[Config] = None,
|
||||
timeout: int = 120,
|
||||
) -> str:
|
||||
"""Transcribe one chunk via the named provider. Raises TranscribeError on failure."""
|
||||
if provider not in PROVIDERS:
|
||||
raise TranscribeError(f"unknown provider: {provider}")
|
||||
cfg = config or Config()
|
||||
key = _provider_key(provider, cfg)
|
||||
if not key:
|
||||
raise NoProviderConfigured(
|
||||
f"{provider}: missing {PROVIDERS[provider]['key_field']} "
|
||||
f"(configure with `agent-reach configure {provider}-key ...`)"
|
||||
)
|
||||
|
||||
info = PROVIDERS[provider]
|
||||
with chunk.open("rb") as fh:
|
||||
try:
|
||||
resp = requests.post(
|
||||
info["endpoint"],
|
||||
headers={"Authorization": f"Bearer {key}"},
|
||||
files={"file": (chunk.name, fh, "audio/m4a")},
|
||||
data={"model": info["model"], "response_format": "text"},
|
||||
timeout=timeout,
|
||||
)
|
||||
except requests.RequestException as e:
|
||||
raise TranscribeError(f"{provider}: network error: {e}") from e
|
||||
|
||||
if not resp.ok:
|
||||
raise TranscribeError(f"{provider}: HTTP {resp.status_code}: {resp.text[:300]}")
|
||||
return resp.text
|
||||
|
||||
|
||||
def _provider_order(provider: str) -> List[str]:
|
||||
if provider == "auto":
|
||||
return ["groq", "openai"]
|
||||
if provider in PROVIDERS:
|
||||
return [provider]
|
||||
raise TranscribeError(f"unknown provider: {provider} (use groq|openai|auto)")
|
||||
|
||||
|
||||
def transcribe(
|
||||
source: str,
|
||||
*,
|
||||
provider: str = "auto",
|
||||
out_dir: Optional[Path] = None,
|
||||
config: Optional[Config] = None,
|
||||
) -> str:
|
||||
"""Transcribe a URL or local file path. Returns the joined transcript text.
|
||||
|
||||
`provider` is one of `auto` (groq → openai), `groq`, or `openai`.
|
||||
`out_dir` defaults to a fresh temp directory; intermediate files stay there.
|
||||
"""
|
||||
cfg = config or Config()
|
||||
order = _provider_order(provider)
|
||||
|
||||
# Validate at least one provider is configured before doing expensive work.
|
||||
if not any(_provider_key(p, cfg) for p in order):
|
||||
names = ", ".join(PROVIDERS[p]["key_field"] for p in order)
|
||||
raise NoProviderConfigured(f"no provider key configured (need one of: {names})")
|
||||
|
||||
work_dir = Path(out_dir) if out_dir else Path(tempfile.mkdtemp(prefix="transcribe-"))
|
||||
work_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
src_path = Path(source)
|
||||
if src_path.is_file():
|
||||
audio = src_path
|
||||
else:
|
||||
audio = download_audio(source, work_dir)
|
||||
|
||||
compressed = compress_audio(audio, work_dir)
|
||||
if compressed.stat().st_size <= SIZE_LIMIT_BYTES:
|
||||
chunks = [compressed]
|
||||
else:
|
||||
chunks = chunk_audio(compressed, work_dir)
|
||||
|
||||
pieces: List[str] = []
|
||||
for chunk in chunks:
|
||||
text = _transcribe_with_fallback(chunk, order, cfg)
|
||||
pieces.append(text.strip())
|
||||
return "\n".join(p for p in pieces if p)
|
||||
|
||||
|
||||
def _transcribe_with_fallback(chunk: Path, order: List[str], config: Config) -> str:
|
||||
"""Try each provider in order; return first success or raise the last error."""
|
||||
last_err: Optional[Exception] = None
|
||||
for p in order:
|
||||
if not _provider_key(p, config):
|
||||
# Skip silently — caller already validated at least one is configured.
|
||||
continue
|
||||
try:
|
||||
return transcribe_chunk(chunk, p, config=config)
|
||||
except TranscribeError as e:
|
||||
last_err = e
|
||||
continue
|
||||
raise TranscribeError(f"all providers failed for {chunk.name}: {last_err}")
|
||||
@@ -0,0 +1,26 @@
|
||||
"""Subprocess helpers for consistent cross-platform text handling."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from collections.abc import Mapping
|
||||
|
||||
UTF8_ENV = {
|
||||
"PYTHONUTF8": "1",
|
||||
"PYTHONIOENCODING": "utf-8",
|
||||
}
|
||||
|
||||
|
||||
def utf8_subprocess_env(base: Mapping[str, str] | None = None) -> dict[str, str]:
|
||||
"""Return an environment that forces Python child processes into UTF-8 mode."""
|
||||
env = dict(base or os.environ)
|
||||
env.update(UTF8_ENV)
|
||||
return env
|
||||
|
||||
|
||||
def mcporter_utf8_env_args() -> list[str]:
|
||||
"""Return mcporter --env arguments for UTF-8 Python stdio servers."""
|
||||
args = []
|
||||
for key, value in UTF8_ENV.items():
|
||||
args.extend(["--env", f"{key}={value}"])
|
||||
return args
|
||||
+63
-90
@@ -4,6 +4,10 @@
|
||||
<strong>Give your AI Agent one-click access to the entire internet</strong>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
The most reliable access path for each platform — chosen, installed, and health-checked for you. Backends come and go; you won't notice.
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="../LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a>
|
||||
<a href="https://www.python.org/"><img src="https://img.shields.io/badge/Python-3.10+-green.svg?style=for-the-badge&logo=python&logoColor=white" alt="Python 3.10+"></a>
|
||||
@@ -51,7 +55,7 @@ Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/ma
|
||||
|---|---|
|
||||
| 💰 **Completely free** | All tools are open source, all APIs are free. The only possible cost is a server proxy ($1/month) — local computers don't need one |
|
||||
| 🔒 **Privacy safe** | Cookies stay local. Never uploaded. Fully open source — audit anytime |
|
||||
| 🔄 **Kept up to date** | Upstream tools (yt-dlp, twitter-cli, rdt-cli, Jina Reader, etc.) are tracked and updated regularly |
|
||||
| 🔄 **Kept up to date** | Every platform routes through a primary + fallback backend list. When an access path dies, we switch to the next — you won't notice (June 2026: Bilibili 412-blocked yt-dlp → switched to bili-cli, zero action on your side) |
|
||||
| 🤖 **Works with any Agent** | Claude Code, OpenClaw, Cursor, Windsurf… any Agent that can run commands |
|
||||
| 🩺 **Built-in diagnostics** | `agent-reach doctor` — one command shows what works, what doesn't, and how to fix it |
|
||||
|
||||
@@ -63,20 +67,17 @@ Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/ma
|
||||
|----------|-------------|:-----:|-------|
|
||||
| 🌐 **Web** | Read | Zero config | Any URL → clean Markdown ([Jina Reader](https://github.com/jina-ai/reader) ⭐9.8K) |
|
||||
| 🐦 **Twitter/X** | Read · Search | Cookie | Cookie unlocks search, timeline, tweet reading, articles ([twitter-cli](https://github.com/public-clis/twitter-cli)) |
|
||||
| 📕 **XiaoHongShu** | Read · Search · **Post · Comment · Like** | Cookie | `pipx install xiaohongshu-cli` + `xhs login` ([xhs-cli](https://github.com/jackwener/xiaohongshu-cli)) |
|
||||
| 🎵 **Douyin** | Video parsing · Watermark-free download | mcporter | Via [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server), no login needed |
|
||||
| 📕 **XiaoHongShu** | Read · Search · Comments | OpenCLI / MCP | Desktop: [OpenCLI](https://github.com/jackwener/opencli) (reuses browser session); Server: [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) (QR login); legacy xhs-cli still works |
|
||||
| 💼 **LinkedIn** | Jina Reader (public pages) | Full profiles, companies, job search | Tell your Agent "help me set up LinkedIn" |
|
||||
| 💬 **WeChat Articles** | Search + Read | Zero config | Search + read WeChat Official Account articles via Exa (zero config) + optional [Camoufox](https://github.com/daijro/camoufox) |
|
||||
| 📰 **Weibo** | Trending · Search · Feeds · Comments | Zero config | Hot search, content/user/topic search, feeds, comments ([mcp-server-weibo](https://github.com/Panniantong/mcp-server-weibo)) |
|
||||
| 💻 **V2EX** | Hot topics · Node topics · Topic detail + replies · User profile | Zero config | Public JSON API, no auth required. Great for tech community content |
|
||||
| 📈 **Xueqiu (雪球)** | Stock quotes · Search · Hot posts · Hot stocks | Browser cookie | Tell your Agent "help me set up Xueqiu" |
|
||||
| 🎙️ **Xiaoyuzhou Podcast** | Transcription | Free API key | Podcast audio → full text transcript via Groq Whisper (free) |
|
||||
| 🔍 **Web Search** | Search | Auto-configured | Auto-configured during install, free, no API key ([Exa](https://exa.ai) via [mcporter](https://github.com/nicepkg/mcporter)) |
|
||||
| 📦 **GitHub** | Read · Search | Zero config | [gh CLI](https://cli.github.com) powered. Public repos work immediately. `gh auth login` unlocks Fork, Issue, PR |
|
||||
| 📺 **YouTube** | Read · **Search** | Zero config | Subtitles + search across 1800+ video sites ([yt-dlp](https://github.com/yt-dlp/yt-dlp) ⭐148K) |
|
||||
| 📺 **Bilibili** | Read · **Search** | Zero config / Proxy | Video info + subtitles + search. Local works directly, servers need a proxy ([yt-dlp](https://github.com/yt-dlp/yt-dlp)) |
|
||||
| 📺 **Bilibili** | Read · **Search** | Zero config | Search + video detail via [bili-cli](https://github.com/public-clis/bilibili-cli) (no login needed); subtitles via [OpenCLI](https://github.com/jackwener/opencli). yt-dlp is 412-blocked by Bilibili and no longer used here |
|
||||
| 📡 **RSS** | Read | Zero config | Any RSS/Atom feed ([feedparser](https://github.com/kurtmckee/feedparser) ⭐2.3K) |
|
||||
| 📖 **Reddit** | Search · Read | Cookie | Requires auth since 2024 — `rdt login` after install ([rdt-cli](https://github.com/public-clis/rdt-cli)) |
|
||||
| 📖 **Reddit** | Search · Read | OpenCLI / Cookie | No zero-config path (anonymous endpoints blocked). Desktop: [OpenCLI](https://github.com/jackwener/opencli) via browser session; or [rdt-cli](https://github.com/public-clis/rdt-cli) + cookie |
|
||||
|
||||
> **Setup levels:** Zero config = install and go · Auto-configured = handled during install · mcporter = needs MCP service · Cookie = export from browser · Proxy = $1/month
|
||||
|
||||
@@ -84,6 +85,15 @@ Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/ma
|
||||
|
||||
## Quick Start
|
||||
|
||||
> ⚠️ **OpenClaw users: enable `exec` permission first**
|
||||
>
|
||||
> Agent Reach relies on the Agent running shell commands (`pip install`, `mcporter`, `twitter`, etc.). If your OpenClaw uses the default `messaging` tool profile, the Agent won't be able to run them. **Enable `exec` before installing:**
|
||||
>
|
||||
> ```bash
|
||||
> openclaw config set tools.profile "coding"
|
||||
> ```
|
||||
> Or set `"tools": { "profile": "coding" }` in `~/.openclaw/openclaw.json`. After changing it, restart the Gateway (`openclaw gateway restart`) and start a new conversation. Other platforms (Claude Code, Cursor, Windsurf, etc.) are not affected.
|
||||
|
||||
Copy this to your AI Agent (Claude Code, OpenClaw, Cursor, etc.):
|
||||
|
||||
```
|
||||
@@ -97,6 +107,12 @@ The Agent auto-installs, detects your environment, and tells you what's ready.
|
||||
> Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md
|
||||
> ```
|
||||
|
||||
> 🛡️ **Worried about security?** Use safe mode — it won't auto-install system packages, it only tells you what you need:
|
||||
> ```
|
||||
> Install Agent Reach (safe mode): https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/install.md
|
||||
> Use the --safe flag during install
|
||||
> ```
|
||||
|
||||
<details>
|
||||
<summary>Manual install</summary>
|
||||
|
||||
@@ -148,11 +164,11 @@ Don't use it? Don't configure it. Every step is optional.
|
||||
|
||||
Tell your Agent "help me configure Twitter cookies" — it'll guide you through exporting from your browser. Local computers can auto-import.
|
||||
|
||||
### 🌐 Proxy — $1/month, servers only
|
||||
### 🌐 Proxy — $1/month, restricted networks only
|
||||
|
||||
Bilibili blocks server IPs. Get a proxy ([Webshare](https://webshare.io) recommended, $1/month) and send the address to your Agent.
|
||||
Most users need no proxy. If your network blocks Reddit/Twitter (e.g. mainland China) get one ([Webshare](https://webshare.io) recommended, $1/month) and send the address to your Agent — it saves it and exports HTTP(S)_PROXY when calling those tools.
|
||||
|
||||
> Reddit now works free via rdt-cli without any proxy. Local computers don't need a proxy for Bilibili either.
|
||||
> Reddit needs a logged-in session either way — OpenCLI rides your browser session, or rdt-cli after `rdt login`. Bilibili works via bili-cli without a proxy.
|
||||
|
||||
---
|
||||
|
||||
@@ -168,7 +184,7 @@ $ agent-reach doctor
|
||||
✅ GitHub repos and code — public repos readable and searchable
|
||||
✅ Twitter/X tweets — readable. Cookie unlocks search and posting
|
||||
✅ YouTube video subtitles — yt-dlp
|
||||
⚠️ Bilibili video info — server IPs may be blocked, configure proxy
|
||||
✅ Bilibili search & video detail — bili-cli (subtitles via OpenCLI)
|
||||
✅ RSS/Atom feeds — feedparser
|
||||
✅ Web pages (any URL) — Jina Reader API
|
||||
|
||||
@@ -176,8 +192,8 @@ $ agent-reach doctor
|
||||
⬜ Web semantic search — sign up at exa.ai for free key
|
||||
|
||||
🔧 Configurable:
|
||||
✅ Reddit posts and comments — search and read via rdt-cli (free, no proxy)
|
||||
⬜ XiaoHongShu notes — needs cookie. Export from browser
|
||||
⬜ Reddit posts and comments — needs login: rdt-cli after `rdt login`, or OpenCLI browser session
|
||||
⬜ XiaoHongShu notes — desktop: OpenCLI (browser session); server: xiaohongshu-mcp (QR)
|
||||
|
||||
Status: 6/9 channels available
|
||||
```
|
||||
@@ -186,56 +202,50 @@ Status: 6/9 channels available
|
||||
|
||||
## Design Philosophy
|
||||
|
||||
**Agent Reach is a scaffolding tool, not a framework.**
|
||||
**Agent Reach is a capability layer, not yet another tool.**
|
||||
|
||||
Every time you spin up a new Agent, you spend time finding tools, installing deps, and debugging configs — what reads Twitter? How do you bypass Reddit blocks? How do you extract YouTube subtitles? Every time, you re-do the same work.
|
||||
It sits one level above any specific implementation — it handles **selection, installation, health checks, and routing**, not the reading itself. Reading is done by your Agent calling upstream tools directly; there is no wrapper layer.
|
||||
|
||||
Agent Reach does one simple thing: **it makes those tool selection and configuration decisions for you.**
|
||||
Every time you spin up a new Agent, you spend time finding tools, installing deps, and debugging configs — what reads Twitter? How do you log into Reddit? What replaces a discontinued XiaoHongShu CLI? Every time, you re-do the same work. Agent Reach does one simple thing: **the most reliable access path for each platform, chosen, installed, and health-checked for you. Access paths come and go (in March 2026 a batch of single-platform CLIs went unmaintained — we re-routed), so you don't have to care.**
|
||||
|
||||
After installation, your Agent calls the upstream tools directly (twitter-cli, rdt-cli, xhs-cli, yt-dlp, mcporter, gh CLI, etc.) — no wrapper layer in between.
|
||||
### 🔌 Every platform = an ordered backend list (primary + fallbacks)
|
||||
|
||||
### 🔌 Every Channel is Pluggable
|
||||
|
||||
Each platform maps to an upstream tool. **Don't like one? Swap it out.**
|
||||
Switching access paths means reordering the list, not rewriting code. `agent-reach doctor` tells you **which backend each platform is currently using**.
|
||||
|
||||
```
|
||||
channels/
|
||||
├── web.py → Jina Reader ← swap to Firecrawl, Crawl4AI…
|
||||
├── twitter.py → twitter-cli ← swap to official API…
|
||||
├── youtube.py → yt-dlp ← swap to YouTube API, Whisper…
|
||||
├── github.py → gh CLI ← swap to REST API, PyGithub…
|
||||
├── bilibili.py → yt-dlp ← swap to bilibili-api…
|
||||
├── reddit.py → rdt-cli ← search + read, cookie auth required
|
||||
├── xiaohongshu.py → mcporter MCP ← swap to other XHS tools…
|
||||
├── douyin.py → mcporter MCP ← swap to other Douyin tools…
|
||||
├── linkedin.py → linkedin-mcp ← swap to LinkedIn API…
|
||||
├── rss.py → feedparser ← swap to atoma…
|
||||
├── exa_search.py → mcporter MCP ← swap to Tavily, SerpAPI…
|
||||
├── web.py → Jina Reader
|
||||
├── twitter.py → twitter-cli ▸ OpenCLI ▸ bird
|
||||
├── youtube.py → yt-dlp
|
||||
├── github.py → gh CLI
|
||||
├── bilibili.py → bili-cli ▸ OpenCLI ▸ search API (yt-dlp retired, 412-blocked)
|
||||
├── reddit.py → OpenCLI ▸ rdt-cli (no zero-config path, login required)
|
||||
├── xiaohongshu.py → OpenCLI ▸ xiaohongshu-mcp ▸ xhs-cli
|
||||
├── linkedin.py → linkedin-mcp ▸ Jina Reader
|
||||
├── rss.py → feedparser
|
||||
├── exa_search.py → Exa via mcporter
|
||||
└── __init__.py → Channel registry (for doctor checks)
|
||||
```
|
||||
|
||||
Each channel file only checks whether its upstream tool is installed and working (`check()` method for `agent-reach doctor`). The actual reading and searching is done by calling the upstream tools directly.
|
||||
Each channel file **actually probes** its candidate backends in order (not just checking that a command exists) — the first fully working one becomes the active backend, and broken ones come with a fix prescription. The actual reading and searching is done by the Agent calling the upstream tools directly.
|
||||
|
||||
### Current Tool Choices
|
||||
|
||||
| Scenario | Tool | Why |
|
||||
|----------|------|-----|
|
||||
| Read web pages | [Jina Reader](https://github.com/jina-ai/reader) | 9.8K stars, free, no API key needed |
|
||||
| Read tweets | [twitter-cli](https://github.com/public-clis/twitter-cli) | 2.1K stars, cookie auth, search/read/timeline/articles |
|
||||
| Reddit | [rdt-cli](https://github.com/public-clis/rdt-cli) | 304 stars, cookie auth, search + full posts + comments |
|
||||
| Video subtitles + search | [yt-dlp](https://github.com/yt-dlp/yt-dlp) | 154K stars, YouTube + Bilibili + 1800 sites |
|
||||
| Bilibili enhanced | [bili-cli](https://github.com/public-clis/bilibili-cli) | 590 stars, hot/rank/search/feed |
|
||||
| Search the web | [Exa](https://exa.ai) via [mcporter](https://github.com/nicobailon/mcporter) | AI semantic search, MCP integration, no API key |
|
||||
| GitHub | [gh CLI](https://cli.github.com) | Official tool, full API after auth |
|
||||
| Read RSS | [feedparser](https://github.com/kurtmckee/feedparser) | Python ecosystem standard, 2.3K stars |
|
||||
| XiaoHongShu | [xhs-cli](https://github.com/jackwener/xiaohongshu-cli) | 1.5K stars, pipx install, search/read/comment/post |
|
||||
| Douyin | [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) | MCP server, no login needed, video parsing + watermark-free download |
|
||||
| LinkedIn | [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) | 1.2K stars, MCP server, browser automation |
|
||||
| WeChat Articles | [Exa](https://exa.ai) (search + read) + [Camoufox](https://github.com/daijro/camoufox) (optional) | Zero-config search + full article reading |
|
||||
| Weibo | `mcporter` | `mcporter call 'weibo.get_trendings(limit: 10)'` |
|
||||
| Xiaoyuzhou Podcast | `transcribe.sh` | `bash ~/.agent-reach/tools/xiaoyuzhou/transcribe.sh <URL>` |
|
||||
| Scenario | Primary | Fallback | Why |
|
||||
|----------|---------|----------|-----|
|
||||
| Read web pages | [Jina Reader](https://github.com/jina-ai/reader) | — | Free, no API key needed |
|
||||
| Read tweets | [twitter-cli](https://github.com/public-clis/twitter-cli) | [OpenCLI](https://github.com/jackwener/opencli) | Reliable search in real-world tests; OpenCLI falls back on your browser session |
|
||||
| Reddit | [OpenCLI](https://github.com/jackwener/opencli) (desktop) | [rdt-cli](https://github.com/public-clis/rdt-cli) | Anonymous endpoints blocked, official API gated — logged-in sessions are the only route left |
|
||||
| YouTube subtitles + search | [yt-dlp](https://github.com/yt-dlp/yt-dlp) | — | 154K stars, still the best for YouTube (no longer used for Bilibili) |
|
||||
| Bilibili | [bili-cli](https://github.com/public-clis/bilibili-cli) | OpenCLI ▸ search API | yt-dlp is 412-blocked by Bilibili (verified June 2026); bili-cli searches and reads without login |
|
||||
| Search the web | [Exa](https://exa.ai) via [mcporter](https://github.com/nicobailon/mcporter) | — | AI semantic search, MCP integration, no API key |
|
||||
| GitHub | [gh CLI](https://cli.github.com) | — | Official tool, full API after auth |
|
||||
| Read RSS | [feedparser](https://github.com/kurtmckee/feedparser) | — | Python ecosystem standard |
|
||||
| XiaoHongShu | [OpenCLI](https://github.com/jackwener/opencli) (desktop) | [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) (server) ▸ xhs-cli | The xhs-cli author moved to OpenCLI (24K stars); browser sessions mean zero friction |
|
||||
| LinkedIn | [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server) | Jina Reader | MCP server, browser automation |
|
||||
| Xiaoyuzhou Podcast | `transcribe.sh` | — | `bash ~/.agent-reach/tools/xiaoyuzhou/transcribe.sh <URL>` |
|
||||
|
||||
> 📌 These are the *current* choices. Don't like one? Swap out the file. That's the whole point of scaffolding.
|
||||
> 📌 These are the *current* choices, re-verified regularly on real machines. When a path dies we switch to the next — `agent-reach doctor` always tells you which one is active.
|
||||
|
||||
---
|
||||
|
||||
@@ -268,7 +278,7 @@ Agent Reach uses [twitter-cli](https://github.com/public-clis/twitter-cli) with
|
||||
<details>
|
||||
<summary><strong>Reddit returns 403 from server / datacenter IP blocked?</strong></summary>
|
||||
|
||||
Agent Reach uses [rdt-cli](https://github.com/public-clis/rdt-cli) for Reddit. Since 2024, Reddit requires authentication for all API requests. Install with `pipx install rdt-cli`, then run `rdt login` (auto-extracts cookies from your browser). Your agent can then search with `rdt search "query"` and read full posts + comments with `rdt read POST_ID`.
|
||||
Reddit requires a logged-in session for everything (anonymous endpoints are blocked, and official API registration has been approval-gated since 2025-11). On desktop, the preferred path is OpenCLI riding your browser's reddit.com session. Otherwise install rdt-cli from the pinned git source (`pipx install 'git+https://github.com/public-clis/rdt-cli.git'` — PyPI lags), then `rdt login`. Your agent can then search with `rdt search "query"` and read full posts + comments with `rdt read POST_ID`.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
@@ -280,7 +290,7 @@ Yes! Agent Reach is an installer + configuration tool. Any AI coding agent that
|
||||
<details>
|
||||
<summary><strong>Is Agent Reach free? Any API costs?</strong></summary>
|
||||
|
||||
100% free and open source. All backends (twitter-cli, rdt-cli, xhs-cli, yt-dlp, Jina Reader, Exa) are free tools that don't require paid API keys. The only optional cost is a residential proxy (~$1/month) if you need Bilibili access from a server. Reddit works free via rdt-cli without any proxy.
|
||||
100% free and open source. All backends (twitter-cli, rdt-cli, xhs-cli, yt-dlp, Jina Reader, Exa) are free tools that don't require paid API keys. The only optional cost is a residential proxy (~$1/month) for some server scenarios. Reddit costs nothing but needs a logged-in session (rdt-cli after `rdt login`, or OpenCLI reusing your browser session).
|
||||
</details>
|
||||
|
||||
<details>
|
||||
@@ -292,47 +302,14 @@ Agent Reach uses twitter-cli which accesses Twitter via cookie auth — same as
|
||||
<details>
|
||||
<summary><strong>How to read XiaoHongShu / 小红书 content programmatically?</strong></summary>
|
||||
|
||||
Install `pipx install xiaohongshu-cli`, then `xhs login` (auto-extracts cookies from browser). Your agent can then use `xhs search "query"` to search notes, `xhs read NOTE_ID` to read details, `xhs comments NOTE_ID` to view comments. No Docker needed.
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>How to parse Douyin / 抖音 videos with AI agent?</strong></summary>
|
||||
|
||||
Install douyin-mcp-server, then your agent can use `mcporter call 'douyin.parse_douyin_video_info(share_link: "share_url")'` to parse video info and get watermark-free download links. No login required — just share the Douyin link. See https://github.com/yzfly/douyin-mcp-server
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>How to extract scripts from both Douyin and XiaoHongShu with one MCP?</strong></summary>
|
||||
|
||||
If you want one MCP server that can handle:
|
||||
|
||||
- Douyin videos
|
||||
- XiaoHongShu video notes
|
||||
- XiaoHongShu image notes
|
||||
|
||||
and directly write `script.md` + `info.json`, you can point the existing `douyin` mcporter alias at:
|
||||
|
||||
- https://github.com/JNHFlow21/social-post-extractor-mcp
|
||||
|
||||
It keeps backward compatibility with:
|
||||
|
||||
- `parse_douyin_video_info`
|
||||
- `get_douyin_download_link`
|
||||
- `extract_douyin_text`
|
||||
|
||||
and adds unified tools:
|
||||
|
||||
- `parse_social_post_info`
|
||||
- `extract_social_post_script`
|
||||
|
||||
This is useful when your agent workflow is “paste a link, get a script file”.
|
||||
On desktop, prefer **OpenCLI** (`agent-reach install --channels opencli`) — it reuses your browser's logged-in session, so if you've browsed XiaoHongShu you're set; one Chrome Web Store click installs the extension. Then `opencli xiaohongshu search "query"` / `opencli xiaohongshu note URL`. On servers use [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) (bundled headless browser, QR login). Existing xhs-cli installs keep working as a fallback backend (upstream unmaintained since 2026-03, not recommended for new setups).
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
## Credits
|
||||
|
||||
[twitter-cli](https://github.com/public-clis/twitter-cli) · [rdt-cli](https://github.com/public-clis/rdt-cli) · [xhs-cli](https://github.com/jackwener/xiaohongshu-cli) · [bili-cli](https://github.com/public-clis/bilibili-cli) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [Jina Reader](https://github.com/jina-ai/reader) · [Exa](https://exa.ai) · [mcporter](https://github.com/nicobailon/mcporter) · [feedparser](https://github.com/kurtmckee/feedparser) · [douyin-mcp-server](https://github.com/yzfly/douyin-mcp-server) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server)
|
||||
[twitter-cli](https://github.com/public-clis/twitter-cli) · [rdt-cli](https://github.com/public-clis/rdt-cli) · [xhs-cli](https://github.com/jackwener/xiaohongshu-cli) · [bili-cli](https://github.com/public-clis/bilibili-cli) · [yt-dlp](https://github.com/yt-dlp/yt-dlp) · [Jina Reader](https://github.com/jina-ai/reader) · [Exa](https://exa.ai) · [mcporter](https://github.com/nicobailon/mcporter) · [feedparser](https://github.com/kurtmckee/feedparser) · [linkedin-scraper-mcp](https://github.com/stickerdaniel/linkedin-mcp-server)
|
||||
|
||||
## Contact
|
||||
|
||||
@@ -353,10 +330,6 @@ For collaboration or questions, add me on WeChat — I'll invite you to the comm
|
||||
|
||||
## Friends
|
||||
|
||||
[FluxNode](https://fluxnode.org) — Low-cost AI API gateway, 90% off official pricing, pay-as-you-go or subscription. Works with OpenClaw, Claude Code, and any Agent.
|
||||
|
||||
[OpenClaw for Enterprise](https://github.com/littleben/openclaw-for-enterprise) — Enterprise-grade multi-user OpenClaw deployment, use AI directly in Feishu/Lark, container isolation, one-command management.
|
||||
|
||||
[OpenClaw on Tencent Cloud](https://www.tencentcloud.com/act/pro/intl-openclaw?referral_code=G76Y819A&lang=en&pg=) — One-click OpenClaw on Tencent Cloud: chat to connect Agent Reach & unlock internet power.
|
||||
|
||||
## Star History
|
||||
|
||||
@@ -348,10 +348,6 @@ douyin-mcp-server를 설치한 다음, 에이전트가 `mcporter call 'douyin.pa
|
||||
|
||||
## 관련 프로젝트
|
||||
|
||||
[FluxNode](https://fluxnode.org) — 저비용 AI API 게이트웨이, 공식 가격의 90% 할인, 종량제 또는 구독. OpenClaw, Claude Code 및 모든 에이전트와 호환.
|
||||
|
||||
[OpenClaw for Enterprise](https://github.com/littleben/openclaw-for-enterprise) — 엔터프라이즈급 다중 사용자 OpenClaw 배포, Feishu/Lark에서 AI 직접 사용, 컨테이너 격리, 원 명령어 관리.
|
||||
|
||||
[OpenClaw on Tencent Cloud](https://www.tencentcloud.com/act/pro/intl-openclaw?referral_code=G76Y819A&lang=en&pg=) — Tencent Cloud에서 원클릭 OpenClaw: 채팅으로 Agent Reach를 연결하고 인터넷 기능을 활성화하세요.
|
||||
|
||||
## Star History
|
||||
|
||||
+62
-132
@@ -20,7 +20,7 @@ Copy this to your AI Agent:
|
||||
|
||||
### Goal
|
||||
|
||||
Install Agent Reach and all upstream tools so your user's agent has full internet access. After installation, you'll use the upstream tools directly (twitter-cli, rdt-cli, xhs-cli, yt-dlp, mcporter, gh CLI, etc.) — Agent Reach is just the installer and health checker.
|
||||
Install Agent Reach and all upstream tools so your user's agent has full internet access. After installation, you'll use the upstream tools directly (OpenCLI, twitter-cli, bili-cli, rdt-cli, yt-dlp, mcporter, gh CLI, etc.) — Agent Reach is the selector, installer, health checker and router, never a wrapper.
|
||||
|
||||
### ⚠️ Boundaries
|
||||
|
||||
@@ -40,7 +40,7 @@ All Agent Reach files go in dedicated directories — **never in the agent works
|
||||
| Purpose | Directory | Example |
|
||||
|---------|-----------|---------|
|
||||
| Config & tokens | `~/.agent-reach/` | `~/.agent-reach/config.json` |
|
||||
| Upstream tool repos | `~/.agent-reach/tools/` | `~/.agent-reach/tools/douyin-mcp-server/` |
|
||||
| Upstream tool repos | `~/.agent-reach/tools/` | `~/.agent-reach/tools/xiaoyuzhou/` |
|
||||
| Temporary files | `/tmp/` | `/tmp/yt-dlp-output/` |
|
||||
| Skills | `~/.openclaw/skills/agent-reach/` | SKILL.md |
|
||||
|
||||
@@ -61,6 +61,19 @@ pip install https://github.com/Panniantong/agent-reach/archive/main.zip
|
||||
agent-reach install --env=auto
|
||||
```
|
||||
|
||||
> 💡 **Windows / Microsoft Store Python alias?**
|
||||
> 如果 `python3 --version` 打开 Microsoft Store,或 `where python3` 指向
|
||||
> `...\AppData\Local\Microsoft\WindowsApps\python3.exe`,说明 `python3` 是 Windows
|
||||
> 的 Store alias,不是可用的 Python 安装。请改用 Python Launcher `py -3`,或实际安装目录里的 `python.exe`。
|
||||
>
|
||||
> PowerShell 示例:
|
||||
> ```powershell
|
||||
> py -3 -m venv $env:USERPROFILE\.agent-reach-venv
|
||||
> $env:USERPROFILE\.agent-reach-venv\Scripts\Activate.ps1
|
||||
> python -m pip install https://github.com/Panniantong/agent-reach/archive/main.zip
|
||||
> agent-reach install --env=auto
|
||||
> ```
|
||||
|
||||
This installs core infrastructure (gh CLI, Node.js, mcporter, Exa search, yt-dlp config) and activates these zero-config channels:
|
||||
|
||||
- Web (Jina Reader), YouTube, GitHub, RSS, Exa Search, V2EX, Bilibili (basic)
|
||||
@@ -83,27 +96,25 @@ After installing the basics, **ask the user** which additional channels they nee
|
||||
>
|
||||
> 还有这些可选渠道,你需要哪些?
|
||||
>
|
||||
> - 🌟 **OpenCLI**(桌面推荐)— 一次安装,小红书/Reddit/B站字幕/Twitter 备选全解锁(复用浏览器登录态,零配置;只需在 Chrome 商店点一次"添加扩展")
|
||||
> - 🐦 **Twitter/X** — 搜推文、看时间线(需要登录 Cookie)
|
||||
> - 📰 **微博** — 热搜、搜索、用户动态
|
||||
> - 💬 **微信公众号** — 搜索和阅读公众号文章
|
||||
> - 📈 **雪球** — 股票行情、热门帖子(需要登录 Cookie)
|
||||
> - 🎙️ **小宇宙播客** — 音频转文字(需要免费 Groq Key)
|
||||
> - 📕 **小红书** — 阅读、搜索、发帖(需要登录)
|
||||
> - 📖 **Reddit** — 搜索和阅读帖子
|
||||
> - 📺 **B站完整版** — 热门、排行、搜索
|
||||
> - 🎵 **抖音** — 视频解析
|
||||
> - 📕 **小红书** — 搜索、阅读、评论(桌面走 OpenCLI;服务器用 xiaohongshu-mcp 扫码)
|
||||
> - 📖 **Reddit** — 搜索和阅读帖子(必须登录态:桌面 OpenCLI 或 rdt-cli + Cookie)
|
||||
> - 📺 **B站完整版** — 热门、排行、搜索、视频详情(bili-cli,无需登录)
|
||||
> - 💼 **LinkedIn** — Profile、职位搜索
|
||||
>
|
||||
> 告诉我你要哪些,比如"帮我装微博和 Twitter"。或者说"全部装"。
|
||||
> 告诉我你要哪些,比如"帮我装小红书和 Twitter"。或者说"全部装"。
|
||||
|
||||
Based on the user's choice, run:
|
||||
|
||||
```bash
|
||||
agent-reach install --env=auto --channels=twitter,weibo # Example: user chose Twitter + Weibo
|
||||
agent-reach install --env=auto --channels=opencli,xiaohongshu # Example: desktop user chose XHS (OpenCLI-backed)
|
||||
agent-reach install --env=auto --channels=all # User wants everything
|
||||
```
|
||||
|
||||
Supported channel names: `twitter`, `weibo`, `wechat`, `xiaoyuzhou`, `xueqiu`, `xiaohongshu`, `reddit`, `bilibili`, `douyin`, `linkedin`, `all`
|
||||
Supported channel names: `opencli`, `twitter`, `xiaoyuzhou`, `xueqiu`, `xiaohongshu`, `reddit`, `bilibili`, `linkedin`, `all`
|
||||
|
||||
### Step 3: Fix what's broken
|
||||
|
||||
@@ -149,57 +160,45 @@ agent-reach configure twitter-cookies "PASTED_STRING"
|
||||
>
|
||||
> 如果用户报告 "fetch failed",参考 [troubleshooting.md](troubleshooting.md)
|
||||
|
||||
**Reddit & Bilibili full access (server users):**
|
||||
> "Reddit and Bilibili block server IPs. To unlock full access, I need a residential proxy. You can get one at https://webshare.io ($1/month). Send me the proxy address."
|
||||
**Reddit (login is mandatory — no zero-config path):**
|
||||
> Reddit 的匿名接口已被封、官方 API 需人工审批。桌面用户首选 OpenCLI(浏览器里登录过 reddit.com 即可用);服务器/存量用户用 rdt-cli:
|
||||
|
||||
```bash
|
||||
agent-reach configure proxy http://user:pass@ip:port
|
||||
# PyPI 落后,从 GitHub 装(与代码内 _RDT_GIT_SOURCE 同一钉定版本)
|
||||
pipx install 'git+https://github.com/public-clis/rdt-cli.git@5e4fb3720d5c174e976cd425ccc3b879d52cac66'
|
||||
rdt login # 自动提取浏览器 Cookie;服务器无浏览器时按 doctor 提示手动写 Cookie
|
||||
```
|
||||
|
||||
**XiaoHongShu / 小红书 (xhs-cli):**
|
||||
> "小红书通过 xhs-cli 访问,pipx 一行安装,不需要 Docker。"
|
||||
|
||||
```bash
|
||||
pipx install xiaohongshu-cli
|
||||
xhs login
|
||||
```
|
||||
|
||||
> `xhs login` 会自动从浏览器提取 Cookie。如果自动提取失败,可以手动导入:
|
||||
>
|
||||
> **手动导入 Cookie(Cookie-Editor 方式):**
|
||||
> 1. 用户在自己的浏览器登录小红书 (xiaohongshu.com)
|
||||
> 2. 用 [Cookie-Editor](https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm) 插件导出 Cookie(JSON 或 Header String 格式均可)
|
||||
> 3. 把 Cookie 字符串发给 Agent
|
||||
> 4. Agent 运行命令完成登录:
|
||||
>
|
||||
> 中国大陆访问 Reddit 需要代理;服务器 IP 被风控时可配住宅代理(如 https://webshare.io,约 $1/月):
|
||||
> ```bash
|
||||
> # JSON 格式(Cookie-Editor → Export → JSON)
|
||||
> agent-reach configure xhs-cookies '[{"name":"web_session","value":"xxx","domain":".xiaohongshu.com",...}]'
|
||||
> agent-reach configure proxy http://user:pass@ip:port
|
||||
> ```
|
||||
|
||||
**XiaoHongShu / 小红书(多后端,按环境选):**
|
||||
|
||||
> **桌面电脑(推荐 OpenCLI):**
|
||||
> "小红书走 OpenCLI——复用你浏览器里的登录态,平时刷过小红书就直接能用,零配置。"
|
||||
|
||||
```bash
|
||||
agent-reach install --channels opencli
|
||||
```
|
||||
|
||||
> 装完后引导用户做唯一一步手动操作(Chrome 安全限制,无法代劳):
|
||||
> 1. 打开 https://chromewebstore.google.com/detail/opencli/ildkmabpimmkaediidaifkhjpohdnifk
|
||||
> 2. 点「添加至 Chrome」
|
||||
> 3. 运行 `opencli doctor` 验证(显示 Extension: connected 即成功)
|
||||
>
|
||||
> # 或 Header String 格式(Cookie-Editor → Export → Header String)
|
||||
> **服务器 / 无桌面环境(xiaohongshu-mcp):**
|
||||
> 1. 从 https://github.com/xpzouying/xiaohongshu-mcp/releases 下载对应平台 binary 到 `~/.agent-reach/tools/`
|
||||
> 2. 启动服务(首次运行会自动下载约 150MB 无头浏览器,耐心等完成)
|
||||
> 3. 让用户扫码登录(agent 调 `get_login_qrcode` 工具取二维码)
|
||||
> 4. 接入:`mcporter config add xiaohongshu http://localhost:18060/mcp`
|
||||
> 5. 调用时务必带 `--timeout 120000`
|
||||
>
|
||||
> **存量用户(xhs-cli):** 已装好的 xhs-cli 继续作为备选后端工作(上游 2026-03 起停更,不推荐新装)。`xhs login` 自动提取浏览器 Cookie;失败时用 Cookie-Editor 导出后:
|
||||
> ```bash
|
||||
> agent-reach configure xhs-cookies "key1=val1; key2=val2; ..."
|
||||
> ```
|
||||
>
|
||||
> **注意:** 推荐使用 Cookie-Editor 导出方式,不要依赖 QR 扫码登录。
|
||||
>
|
||||
> **备选方案:Docker MCP**
|
||||
> 如果你已经在使用 [xiaohongshu-mcp](https://github.com/xpzouying/xiaohongshu-mcp) Docker 方案,它也能正常工作:
|
||||
> ```bash
|
||||
> docker run -d --name xiaohongshu-mcp -p 18060:18060 xpzouying/xiaohongshu-mcp
|
||||
> mcporter config add xiaohongshu http://localhost:18060/mcp
|
||||
> ```
|
||||
|
||||
**微博 / Weibo (mcp-server-weibo):**
|
||||
> "微博已默认安装,装好即用。可搜索微博内容、查看热搜、获取用户动态和评论。"
|
||||
|
||||
如果自动安装失败,手动安装:
|
||||
|
||||
```bash
|
||||
pip install git+https://github.com/Panniantong/mcp-server-weibo.git
|
||||
mcporter config add weibo --command 'mcp-server-weibo'
|
||||
```
|
||||
|
||||
> 无需登录、无需 Cookie、无需代理。海外服务器也可以直接访问。
|
||||
|
||||
**雪球 / Xueqiu (股票行情 + 热门帖子):**
|
||||
> "雪球需要登录后的 Cookie。请先在 Chrome 里登录 xueqiu.com,然后运行:"
|
||||
@@ -239,75 +238,6 @@ agent-reach configure groq-key gsk_xxxxx
|
||||
> - 转录质量高(Whisper large-v3),但不区分说话人
|
||||
> - 2 小时以上的播客建议分批处理
|
||||
|
||||
**抖音 / Douyin (douyin-mcp-server):**
|
||||
> "抖音视频解析需要一个 MCP 服务。安装 douyin-mcp-server 后即可解析视频、获取无水印下载链接。"
|
||||
|
||||
```bash
|
||||
# 1. 安装
|
||||
pip install douyin-mcp-server
|
||||
|
||||
# 2. 启动 HTTP 服务(端口 18070)
|
||||
# 方式一:用 uv(推荐)
|
||||
mkdir -p ~/.agent-reach/tools && cd ~/.agent-reach/tools
|
||||
git clone https://github.com/yzfly/douyin-mcp-server.git && cd douyin-mcp-server
|
||||
uv sync && uv run python run_http.py
|
||||
|
||||
# 方式二:直接用 Python 启动
|
||||
python -c "
|
||||
from douyin_mcp_server.server import mcp
|
||||
mcp.settings.host = '127.0.0.1'
|
||||
mcp.settings.port = 18070
|
||||
mcp.run(transport='streamable-http')
|
||||
"
|
||||
|
||||
# 3. 注册到 mcporter
|
||||
mcporter config add douyin http://localhost:18070/mcp
|
||||
```
|
||||
|
||||
> 无需认证即可解析视频信息和获取下载链接。
|
||||
> 如需 AI 语音识别提取文案功能,需要配置硅基流动 API Key(`export API_KEY="sk-xxx"`)。
|
||||
>
|
||||
> 详见 https://github.com/yzfly/douyin-mcp-server
|
||||
|
||||
**可选实现:Douyin + XiaoHongShu unified extractor**
|
||||
> "如果你想把抖音和小红书统一成一个 MCP,并直接输出 `script.md` 和 `info.json`,可以改用 social-post-extractor-mcp。"
|
||||
|
||||
适用场景:
|
||||
|
||||
- 抖音视频转文字稿
|
||||
- 小红书视频笔记转文字稿
|
||||
- 小红书图文笔记正文 + 图片文字提取
|
||||
|
||||
兼容性:
|
||||
|
||||
- 仍然可以注册成 `douyin` 这个 mcporter server 名称
|
||||
- 兼容旧工具名 `parse_douyin_video_info` / `get_douyin_download_link` / `extract_douyin_text`
|
||||
- 同时新增 `parse_social_post_info` / `extract_social_post_script`
|
||||
|
||||
示例配置:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/JNHFlow21/social-post-extractor-mcp.git
|
||||
cd social-post-extractor-mcp
|
||||
uv sync
|
||||
|
||||
mcporter config add douyin \
|
||||
--command /bin/zsh \
|
||||
--arg -lc \
|
||||
--arg "cd '$PWD' && exec '.venv/bin/python' -m social_post_extractor_mcp" \
|
||||
--env ASR_PROVIDER=bailian \
|
||||
--env ASR_MODEL=paraformer-v2 \
|
||||
--env VISION_PROVIDER=bailian \
|
||||
--env VISION_MODEL=qwen3-vl-flash \
|
||||
--env CLEAN_PROVIDER=bailian \
|
||||
--env CLEAN_MODEL=qwen-flash \
|
||||
--env BAILIAN_API_KEY=YOUR_BAILIAN_API_KEY
|
||||
```
|
||||
|
||||
> 这个实现更适合“把链接直接交给 Agent,然后拿到脚本文件”的工作流。
|
||||
>
|
||||
> 详见 https://github.com/JNHFlow21/social-post-extractor-mcp
|
||||
|
||||
**LinkedIn (可选 — linkedin-scraper-mcp):**
|
||||
> "LinkedIn 基本内容可通过 Jina Reader 读取。完整功能(Profile 详情、职位搜索)需要 linkedin-scraper-mcp。"
|
||||
|
||||
@@ -363,7 +293,7 @@ If the user agrees, create a **cron job** (daily, `sessionTarget: "isolated"`, `
|
||||
运行 agent-reach watch 命令。
|
||||
如果输出包含"全部正常",不需要通知用户,静默结束。
|
||||
如果输出包含问题(❌ ⚠️)或新版本(🆕),把完整报告发给用户,并建议修复方案。
|
||||
如果有新版本可用,问用户是否要升级(升级命令:pip install --upgrade https://github.com/Panniantong/agent-reach/archive/main.zip)。
|
||||
如果有新版本可用,问用户是否要升级(把这句话发给用户的 Agent 即可完整更新:帮我更新 Agent Reach:https://raw.githubusercontent.com/Panniantong/agent-reach/main/docs/update.md)。
|
||||
```
|
||||
|
||||
If the user wants a different agent to handle it, let them choose.
|
||||
@@ -375,7 +305,7 @@ If the user wants a different agent to handle it, let them choose.
|
||||
| Command | What it does |
|
||||
|---------|-------------|
|
||||
| `agent-reach install --env=auto` | Install core channels (lightweight, zero-config) |
|
||||
| `agent-reach install --env=auto --channels=twitter,weibo` | Install core + optional channels |
|
||||
| `agent-reach install --env=auto --channels=twitter,xiaohongshu` | Install core + optional channels |
|
||||
| `agent-reach install --env=auto --channels=all` | Install everything |
|
||||
| `agent-reach install --env=auto --safe` | Safe setup (no auto system changes) |
|
||||
| `agent-reach install --env=auto --dry-run` | Preview what would be done |
|
||||
@@ -383,23 +313,23 @@ If the user wants a different agent to handle it, let them choose.
|
||||
| `agent-reach watch` | Quick health + update check (for scheduled tasks) |
|
||||
| `agent-reach check-update` | Check for new versions |
|
||||
| `agent-reach configure twitter-cookies "..."` | Unlock Twitter search + posting |
|
||||
| `agent-reach configure proxy URL` | Unlock Reddit + Bilibili on servers |
|
||||
| `agent-reach configure proxy URL` | 保存代理地址(Agent 访问 Reddit/Twitter 等受限网络时读取它设置 HTTP_PROXY/HTTPS_PROXY,不是自动解锁开关) |
|
||||
| `agent-reach configure groq-key gsk_xxx` | Unlock Xiaoyuzhou podcast transcription |
|
||||
|
||||
After installation, use upstream tools directly. See SKILL.md for the full command reference:
|
||||
|
||||
| Platform | Upstream Tool | Example |
|
||||
|----------|--------------|---------|
|
||||
| Twitter/X | `twitter` | `twitter search "query" -n 10` |
|
||||
| Twitter/X | `twitter`(备选 `opencli`) | `twitter search "query" -n 10` |
|
||||
| YouTube | `yt-dlp` | `yt-dlp --dump-json URL` |
|
||||
| Bilibili | `yt-dlp` + `bili` | `bili hot` / `bili search "query" --type video` |
|
||||
| Reddit | `rdt` | `rdt search "query"` / `rdt read POST_ID` |
|
||||
| Bilibili | `bili`(字幕走 `opencli`) | `bili search "query" --type video` / `opencli bilibili subtitle BVxxx` |
|
||||
| Reddit | `opencli`(备选 `rdt`) | `opencli reddit search "query" -f yaml` / `rdt read POST_ID` |
|
||||
| GitHub | `gh` | `gh search repos "query"` |
|
||||
| Web | `curl` + Jina | `curl -s "https://r.jina.ai/URL"` |
|
||||
| Exa Search | `mcporter` | `mcporter call 'exa.web_search_exa(...)'` |
|
||||
| 小红书 | `mcporter` | `mcporter call 'xiaohongshu.search_feeds(...)'` |
|
||||
| 微博 | `mcporter` | `mcporter call 'weibo.get_trendings(limit: 10)'` |
|
||||
| 小红书 | `opencli`(服务器 `mcporter`) | `opencli xiaohongshu search "query" -f yaml` |
|
||||
| 小宇宙播客 | `transcribe.sh` | `bash ~/.agent-reach/tools/xiaoyuzhou/transcribe.sh <URL>` |
|
||||
| 抖音 | `mcporter` | `mcporter call 'douyin.parse_douyin_video_info(...)'` |
|
||||
| LinkedIn | `mcporter` | `mcporter call 'linkedin.get_person_profile(...)'` |
|
||||
| RSS | `feedparser` | `python3 -c "import feedparser; ..."` |
|
||||
|
||||
> 多后端平台以 `agent-reach doctor --json` 的 `active_backend` 为准。
|
||||
|
||||
+51
-68
@@ -24,7 +24,7 @@ Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/ma
|
||||
|
||||
### Goal
|
||||
|
||||
Update Agent Reach to the latest version, install any new upstream tools, migrate from deprecated tools, and verify everything works. The user should not need to do anything manually.
|
||||
Update Agent Reach to the latest version, refresh upstream tools, migrate from retired backends, and verify everything works. The user should not need to do anything manually (except things only a human can do, like clicking a browser-extension install button).
|
||||
|
||||
### Step 1: Check current version
|
||||
|
||||
@@ -32,7 +32,7 @@ Update Agent Reach to the latest version, install any new upstream tools, migrat
|
||||
agent-reach check-update
|
||||
```
|
||||
|
||||
If it says "已是最新版本", skip to Step 5 (verify). Otherwise continue.
|
||||
If it says "已是最新版本", skip to Step 4 (verify). Otherwise continue.
|
||||
|
||||
### Step 2: Update Agent Reach package
|
||||
|
||||
@@ -40,40 +40,47 @@ If it says "已是最新版本", skip to Step 5 (verify). Otherwise continue.
|
||||
pip install --upgrade https://github.com/Panniantong/agent-reach/archive/main.zip
|
||||
```
|
||||
|
||||
### Step 3: Install / migrate upstream tools
|
||||
> If pip complains about `externally-managed-environment` (PEP 668), the user
|
||||
> originally installed via pipx or a venv — use the matching command:
|
||||
> `pipx install --force https://github.com/Panniantong/agent-reach/archive/main.zip`
|
||||
> or activate `~/.agent-reach-venv` first.
|
||||
|
||||
Run these commands to ensure all upstream CLI tools are installed. Skip any that are already installed.
|
||||
### Step 3: Refresh upstream tools
|
||||
|
||||
Run these to keep installed tools current. **Only upgrade what is already
|
||||
installed — do not install new tools the user never asked for** (the one
|
||||
exception: OpenCLI on desktop, see below).
|
||||
|
||||
```bash
|
||||
# === Required tools (core functionality) ===
|
||||
# Python-based CLIs the user already has (upgrade keeps signatures fresh)
|
||||
which twitter >/dev/null 2>&1 && { pipx upgrade twitter-cli 2>/dev/null || uv tool upgrade twitter-cli 2>/dev/null; }
|
||||
which bili >/dev/null 2>&1 && { pipx upgrade bilibili-cli 2>/dev/null || uv tool upgrade bilibili-cli 2>/dev/null; }
|
||||
which xhs >/dev/null 2>&1 && { pipx upgrade xiaohongshu-cli 2>/dev/null || uv tool upgrade xiaohongshu-cli 2>/dev/null; }
|
||||
which yt-dlp >/dev/null 2>&1 && { pipx upgrade yt-dlp 2>/dev/null || uv tool upgrade yt-dlp 2>/dev/null || pip install -U yt-dlp 2>/dev/null; }
|
||||
|
||||
# Twitter/X — twitter-cli (replaces deprecated bird CLI)
|
||||
which twitter >/dev/null 2>&1 || pipx install twitter-cli 2>/dev/null || uv tool install twitter-cli 2>/dev/null
|
||||
# rdt-cli is pinned to a git source (PyPI lags upstream) — same pin as the code's _RDT_GIT_SOURCE
|
||||
which rdt >/dev/null 2>&1 && pipx install --force 'git+https://github.com/public-clis/rdt-cli.git@5e4fb3720d5c174e976cd425ccc3b879d52cac66' 2>/dev/null
|
||||
|
||||
# Reddit — rdt-cli (replaces Exa-based approach)
|
||||
which rdt >/dev/null 2>&1 || pipx install rdt-cli 2>/dev/null || uv tool install rdt-cli 2>/dev/null
|
||||
|
||||
# XiaoHongShu — xhs-cli (replaces Docker MCP)
|
||||
which xhs >/dev/null 2>&1 || pipx install xiaohongshu-cli 2>/dev/null || uv tool install xiaohongshu-cli 2>/dev/null
|
||||
|
||||
# === Optional enhancements ===
|
||||
|
||||
# Bilibili — bili-cli (adds hot/rank/search on top of yt-dlp)
|
||||
which bili >/dev/null 2>&1 || pipx install bilibili-cli 2>/dev/null || uv tool install bilibili-cli 2>/dev/null
|
||||
|
||||
# mcporter + Exa (for WeChat, web search)
|
||||
which mcporter >/dev/null 2>&1 || npm install -g mcporter 2>/dev/null
|
||||
|
||||
# yt-dlp (YouTube + Bilibili video)
|
||||
which yt-dlp >/dev/null 2>&1 || pip install yt-dlp 2>/dev/null
|
||||
|
||||
# gh CLI (GitHub)
|
||||
which gh >/dev/null 2>&1 || echo "GitHub CLI not installed — see https://cli.github.com"
|
||||
# npm-based
|
||||
which mcporter >/dev/null 2>&1 && npm update -g mcporter 2>/dev/null
|
||||
which opencli >/dev/null 2>&1 && npm update -g @jackwener/opencli 2>/dev/null
|
||||
```
|
||||
|
||||
**Desktop users without OpenCLI**: since v1.5.0 OpenCLI is the preferred
|
||||
backend for 小红书/Reddit (and adds B站 subtitles) by riding the user's
|
||||
browser session. Offer it once:
|
||||
|
||||
> "这次更新引入了 OpenCLI 后端(复用你的浏览器登录态,小红书/Reddit 零配置)。要装吗?装完只需你在 Chrome 商店点一次『添加扩展』。"
|
||||
|
||||
If yes: `agent-reach install --channels opencli` and guide them through the
|
||||
extension click. If no, everything keeps working on existing backends.
|
||||
|
||||
### Step 4: Coexistence (DO NOT uninstall old tools)
|
||||
|
||||
**IMPORTANT: Never uninstall any existing tools the user already has installed.** Old tools (bird, browser_cookie3, miku_ai, etc.) may still be working on the user's machine even if their upstream repos are archived or deleted. Agent Reach supports both old and new tools as fallback. Only install new tools alongside existing ones.
|
||||
**Never uninstall tools the user already has.** Retired backends (e.g. yt-dlp
|
||||
no longer serves Bilibili; xhs-cli is no longer installed by default) keep
|
||||
working as fallbacks where they still function. Agent Reach routes around
|
||||
them automatically — removal is the user's call, not yours.
|
||||
|
||||
### Step 5: Verify
|
||||
|
||||
@@ -82,51 +89,27 @@ agent-reach version
|
||||
agent-reach doctor
|
||||
```
|
||||
|
||||
Running `agent-reach doctor` (text mode) also auto-syncs the bundled skill
|
||||
(SKILL.md + references) into every detected agent skill directory — no
|
||||
separate skill-update step is needed.
|
||||
|
||||
Check the doctor output:
|
||||
- All channels should show ✅ or ⚠️ (with clear instructions)
|
||||
- If any channel that was working before now shows ❌, investigate:
|
||||
- Twitter: needs `twitter-cli` + browser cookies (run `xhs login` pattern)
|
||||
- Reddit: needs `rdt-cli` (should be installed in Step 3)
|
||||
- XiaoHongShu: needs `xhs-cli` + login (run `xhs login`)
|
||||
- WeChat: should work via Exa (zero config)
|
||||
|
||||
### Step 6: Update SKILL.md (if installed)
|
||||
- Every channel shows ✅ / [!] with a clear message, and multi-backend
|
||||
channels (小红书/Reddit/B站/Twitter) report `当前后端:…`
|
||||
- If a previously-working channel now shows [X]/error, the message contains
|
||||
the exact fix (e.g. a venv-reinstall prescription) — run it, then re-check
|
||||
- `--json` gives the same data machine-readably (`active_backend` per channel)
|
||||
|
||||
```bash
|
||||
agent-reach install --skill-only 2>/dev/null || python3 -c "
|
||||
import importlib.resources, os
|
||||
try:
|
||||
skill_pkg = importlib.resources.files('agent_reach').joinpath('skill')
|
||||
except Exception:
|
||||
from pathlib import Path
|
||||
skill_pkg = Path(os.path.dirname(os.path.abspath(__file__))).parent / 'agent_reach' / 'skill'
|
||||
|
||||
skill_md = skill_pkg.joinpath('SKILL.md').read_text(encoding='utf-8')
|
||||
refs_pkg = skill_pkg.joinpath('references')
|
||||
|
||||
for base in [os.path.expanduser('~/.openclaw/skills/agent-reach'),
|
||||
os.path.expanduser('~/.claude/skills/agent-reach'),
|
||||
os.path.expanduser('~/.agents/skills/agent-reach')]:
|
||||
if os.path.isdir(base):
|
||||
with open(os.path.join(base, 'SKILL.md'), 'w', encoding='utf-8') as f:
|
||||
f.write(skill_md)
|
||||
refs_dir = os.path.join(base, 'references')
|
||||
os.makedirs(refs_dir, exist_ok=True)
|
||||
for ref in refs_pkg.iterdir():
|
||||
name = ref.name if hasattr(ref, 'name') else str(ref).split('/')[-1]
|
||||
if name.endswith('.md'):
|
||||
with open(os.path.join(refs_dir, name), 'w', encoding='utf-8') as f:
|
||||
f.write(ref.read_text(encoding='utf-8'))
|
||||
print(f'✅ Updated skill: {base}')
|
||||
"
|
||||
```
|
||||
|
||||
### Step 7: Report to user
|
||||
### Step 6: Report to user
|
||||
|
||||
Tell the user:
|
||||
1. What version they're now on (`agent-reach version`)
|
||||
2. How many channels are available (count ✅ in doctor output)
|
||||
3. Any channels that need their action (e.g., `xhs login` for XiaoHongShu, browser cookies for Twitter)
|
||||
4. What changed in this update (major: Twitter/Reddit/XiaoHongShu upstream tools migrated for better stability)
|
||||
|
||||
1. What version they're on now (`agent-reach version`)
|
||||
2. How many channels are available, and which backend each multi-backend
|
||||
platform is using (from doctor)
|
||||
3. Anything that needs their action (e.g. Chrome extension click, `xhs login`,
|
||||
QR scan for xiaohongshu-mcp on servers)
|
||||
4. What changed in this update (release notes shown by `check-update`)
|
||||
|
||||
Done.
|
||||
|
||||
+1
-7
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "agent-reach"
|
||||
version = "1.4.0"
|
||||
version = "1.5.0"
|
||||
description = "Give your AI Agent eyes to see the entire internet. Search + Read 10+ platforms."
|
||||
readme = "README.md"
|
||||
license = {text = "MIT"}
|
||||
@@ -64,12 +64,6 @@ build-backend = "hatchling.build"
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["agent_reach"]
|
||||
|
||||
[tool.hatch.build.targets.wheel.force-include]
|
||||
"agent_reach/guides" = "agent_reach/guides"
|
||||
# Keep the whole skill directory so SKILL.md, SKILL_en.md, and references/ ship together.
|
||||
"agent_reach/skill" = "agent_reach/skill"
|
||||
"agent_reach/scripts" = "agent_reach/scripts"
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py310"
|
||||
line-length = 100
|
||||
|
||||
@@ -1,10 +1,17 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Contract tests for channel adapters."""
|
||||
|
||||
import subprocess
|
||||
|
||||
from agent_reach.channels import get_all_channels
|
||||
from agent_reach.config import Config
|
||||
|
||||
|
||||
def _fake_run_ok(cmd, **kwargs):
|
||||
"""Pretend any probed CLI executes fine and prints a version."""
|
||||
return subprocess.CompletedProcess(cmd, 0, "2026.06.09", "")
|
||||
|
||||
|
||||
def test_channel_registry_contract():
|
||||
channels = get_all_channels()
|
||||
assert channels, "channel registry must not be empty"
|
||||
@@ -29,6 +36,67 @@ def test_channel_check_contract_with_minimal_runtime(monkeypatch, tmp_path):
|
||||
assert isinstance(message, str) and message.strip()
|
||||
|
||||
|
||||
def test_channel_active_backend_attribute_contract():
|
||||
"""Every channel exposes active_backend (default None, or str once set)."""
|
||||
for ch in get_all_channels():
|
||||
assert hasattr(ch, "active_backend")
|
||||
# Fresh instances must default to None / str (class attribute on base)
|
||||
fresh = type(ch)()
|
||||
assert fresh.active_backend is None or isinstance(fresh.active_backend, str)
|
||||
|
||||
|
||||
def test_channel_active_backend_set_by_check(monkeypatch, tmp_path):
|
||||
"""After check(), active_backend is None or a str — never anything else."""
|
||||
monkeypatch.setattr("shutil.which", lambda _cmd: None)
|
||||
|
||||
# Keep the network-based channels (V2EX/Xueqiu/Bilibili API) deterministic.
|
||||
import urllib.request
|
||||
from urllib.error import URLError
|
||||
|
||||
def _no_net(*_a, **_k):
|
||||
raise URLError("offline")
|
||||
|
||||
monkeypatch.setattr(urllib.request, "urlopen", _no_net)
|
||||
import agent_reach.channels.xueqiu as xueqiu_mod
|
||||
monkeypatch.setattr(xueqiu_mod, "_cookies_initialized", True)
|
||||
monkeypatch.setattr(xueqiu_mod._opener, "open", _no_net)
|
||||
|
||||
config = Config(config_path=tmp_path / "config.yaml")
|
||||
for ch in get_all_channels():
|
||||
ch.check(config)
|
||||
assert ch.active_backend is None or isinstance(ch.active_backend, str), (
|
||||
f"{ch.name}: active_backend must be None or str after check()"
|
||||
)
|
||||
|
||||
|
||||
def test_ordered_backends_contract(tmp_path):
|
||||
"""ordered_backends(config) is a reordering (same multiset) of backends."""
|
||||
config = Config(config_path=tmp_path / "config.yaml")
|
||||
for ch in get_all_channels():
|
||||
ordered = ch.ordered_backends(config)
|
||||
assert isinstance(ordered, list)
|
||||
assert sorted(ordered) == sorted(ch.backends), (
|
||||
f"{ch.name}: ordered_backends must be a permutation of backends"
|
||||
)
|
||||
# And without any config at all
|
||||
ordered_none = ch.ordered_backends(None)
|
||||
assert sorted(ordered_none) == sorted(ch.backends)
|
||||
|
||||
|
||||
def test_ordered_backends_override_moves_backend_to_front():
|
||||
"""Config key <channel>_backend promotes the named backend to front."""
|
||||
from agent_reach.channels.twitter import TwitterChannel
|
||||
|
||||
ch = TwitterChannel()
|
||||
ordered = ch.ordered_backends({"twitter_backend": "bird"})
|
||||
assert ordered[0] == "bird CLI (legacy)"
|
||||
assert sorted(ordered) == sorted(ch.backends)
|
||||
|
||||
# Unknown override is ignored — never hides working backends
|
||||
ordered_unknown = ch.ordered_backends({"twitter_backend": "no-such-tool"})
|
||||
assert ordered_unknown == list(ch.backends)
|
||||
|
||||
|
||||
def test_youtube_warns_when_node_only_and_no_config(monkeypatch, tmp_path):
|
||||
"""YouTube should warn when only Node.js is installed but no yt-dlp config exists."""
|
||||
from agent_reach.channels.youtube import YouTubeChannel
|
||||
@@ -41,6 +109,7 @@ def test_youtube_warns_when_node_only_and_no_config(monkeypatch, tmp_path):
|
||||
return None # deno not installed
|
||||
|
||||
monkeypatch.setattr("shutil.which", fake_which)
|
||||
monkeypatch.setattr("subprocess.run", _fake_run_ok) # yt-dlp probe really executes now
|
||||
# Point to a non-existent config file
|
||||
monkeypatch.setattr("os.path.expanduser", lambda p: str(tmp_path / ".config/yt-dlp/config"))
|
||||
|
||||
@@ -48,6 +117,7 @@ def test_youtube_warns_when_node_only_and_no_config(monkeypatch, tmp_path):
|
||||
status, message = ch.check()
|
||||
assert status == "warn"
|
||||
assert "--js-runtimes" in message
|
||||
assert ch.active_backend == "yt-dlp" # 本体活着,warn 只关乎 JS runtime
|
||||
|
||||
|
||||
def test_youtube_warns_with_windows_specific_fix_command(monkeypatch, tmp_path):
|
||||
@@ -62,6 +132,7 @@ def test_youtube_warns_with_windows_specific_fix_command(monkeypatch, tmp_path):
|
||||
return None
|
||||
|
||||
monkeypatch.setattr("shutil.which", fake_which)
|
||||
monkeypatch.setattr("subprocess.run", _fake_run_ok) # yt-dlp probe really executes now
|
||||
monkeypatch.setattr("agent_reach.utils.paths.sys.platform", "win32")
|
||||
monkeypatch.setenv("APPDATA", str(tmp_path / "AppData" / "Roaming"))
|
||||
|
||||
@@ -84,53 +155,12 @@ def test_youtube_ok_when_deno_installed(monkeypatch):
|
||||
return None
|
||||
|
||||
monkeypatch.setattr("shutil.which", fake_which)
|
||||
monkeypatch.setattr("subprocess.run", _fake_run_ok) # yt-dlp probe really executes now
|
||||
|
||||
ch = YouTubeChannel()
|
||||
status, _msg = ch.check()
|
||||
assert status == "ok"
|
||||
|
||||
|
||||
def test_douyin_check_does_not_call_with_invalid_url(monkeypatch, tmp_path):
|
||||
"""Douyin check should use 'mcporter list' instead of calling with a hardcoded URL."""
|
||||
import subprocess
|
||||
|
||||
from agent_reach.channels.douyin import DouyinChannel
|
||||
|
||||
calls = []
|
||||
original_run = subprocess.run
|
||||
|
||||
def tracking_run(cmd, **kwargs):
|
||||
calls.append(cmd)
|
||||
# Simulate mcporter config list returning douyin
|
||||
if "config" in cmd and "list" in cmd:
|
||||
|
||||
class R:
|
||||
stdout = "douyin http://localhost:18070/mcp"
|
||||
returncode = 0
|
||||
|
||||
return R()
|
||||
# Simulate mcporter list douyin returning tools
|
||||
if "list" in cmd and "douyin" in cmd:
|
||||
|
||||
class R:
|
||||
stdout = "parse_douyin_video_info"
|
||||
returncode = 0
|
||||
|
||||
return R()
|
||||
return original_run(cmd, **kwargs)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"shutil.which", lambda cmd: "/usr/bin/mcporter" if cmd == "mcporter" else None
|
||||
)
|
||||
monkeypatch.setattr("subprocess.run", tracking_run)
|
||||
|
||||
ch = DouyinChannel()
|
||||
status, _msg = ch.check()
|
||||
|
||||
# Should NOT contain any hardcoded douyin.com URL in subprocess calls
|
||||
for call in calls:
|
||||
call_str = " ".join(call) if isinstance(call, list) else str(call)
|
||||
assert "https://www.douyin.com" not in call_str
|
||||
assert ch.active_backend == "yt-dlp"
|
||||
|
||||
|
||||
def test_channel_can_handle_contract():
|
||||
@@ -141,9 +171,7 @@ def test_channel_can_handle_contract():
|
||||
"reddit": "https://reddit.com/r/python",
|
||||
"bilibili": "https://www.bilibili.com/video/BV1xx411",
|
||||
"xiaohongshu": "https://www.xiaohongshu.com/explore/123",
|
||||
"douyin": "https://www.douyin.com/video/123",
|
||||
"linkedin": "https://www.linkedin.com/in/test",
|
||||
"weibo": "https://weibo.com/u/1749127163",
|
||||
"rss": "https://example.com/feed.xml",
|
||||
"xueqiu": "https://xueqiu.com/S/SH600519",
|
||||
"exa_search": "https://example.com",
|
||||
|
||||
+481
-14
@@ -647,15 +647,36 @@ class TestXueqiuChannel:
|
||||
|
||||
|
||||
class TestRedditChannel:
|
||||
def test_reports_off_when_not_installed(self, monkeypatch):
|
||||
"""多后端:OpenCLI > rdt-cli,没有零配置路径。"""
|
||||
|
||||
@staticmethod
|
||||
def _isolate(monkeypatch, opencli=None):
|
||||
"""隔离 OpenCLI 候选(None = 未安装),聚焦 rdt-cli 路径。"""
|
||||
from agent_reach.channels.reddit import RedditChannel
|
||||
monkeypatch.setattr(RedditChannel, "_check_opencli", lambda self: opencli)
|
||||
|
||||
def test_reports_off_when_nothing_installed(self, monkeypatch):
|
||||
self._isolate(monkeypatch)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: None)
|
||||
from agent_reach.channels.reddit import RedditChannel
|
||||
status, msg = RedditChannel().check()
|
||||
assert status == "off"
|
||||
assert "rdt-cli" in msg
|
||||
assert "public-clis/rdt-cli" in msg
|
||||
# 诚实口径:明说没有零配置路径,推荐 OpenCLI + rdt git 源
|
||||
assert "零配置" in msg
|
||||
assert "opencli" in msg
|
||||
assert "git+https://github.com/public-clis/rdt-cli.git" in msg
|
||||
|
||||
def test_opencli_ready_wins(self, monkeypatch):
|
||||
self._isolate(monkeypatch, opencli=("ok", "OpenCLI 可用(复用浏览器登录态)"))
|
||||
monkeypatch.setattr(shutil, "which", lambda _: None)
|
||||
from agent_reach.channels.reddit import RedditChannel
|
||||
ch = RedditChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert ch.active_backend == "OpenCLI"
|
||||
|
||||
def test_reports_ok_when_authenticated(self, monkeypatch):
|
||||
self._isolate(monkeypatch)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/rdt")
|
||||
fake_output = json.dumps({
|
||||
"ok": True,
|
||||
@@ -668,11 +689,14 @@ class TestRedditChannel:
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.reddit import RedditChannel
|
||||
status, msg = RedditChannel().check()
|
||||
ch = RedditChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert "testuser" in msg
|
||||
assert ch.active_backend == "rdt-cli"
|
||||
|
||||
def test_reports_warn_when_not_authenticated(self, monkeypatch):
|
||||
self._isolate(monkeypatch)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/rdt")
|
||||
fake_output = json.dumps({
|
||||
"ok": True,
|
||||
@@ -685,14 +709,19 @@ class TestRedditChannel:
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.reddit import RedditChannel
|
||||
status, msg = RedditChannel().check()
|
||||
ch = RedditChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "warn"
|
||||
assert "403" in msg
|
||||
assert "rdt login" in msg
|
||||
assert "Cookie-Editor" in msg
|
||||
assert "chromewebstore.google.com" in msg
|
||||
# 未登录是业务态:进程活着,后端仍然算可用
|
||||
assert ch.active_backend == "rdt-cli"
|
||||
|
||||
def test_reports_warn_when_status_check_fails(self, monkeypatch):
|
||||
def test_reports_error_when_status_check_fails(self, monkeypatch):
|
||||
"""rdt 非零退出且输出不可解析 → 工具异常(error),不再算 warn。"""
|
||||
self._isolate(monkeypatch)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/rdt")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
@@ -700,8 +729,45 @@ class TestRedditChannel:
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.reddit import RedditChannel
|
||||
status, msg = RedditChannel().check()
|
||||
assert status == "warn"
|
||||
ch = RedditChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "error"
|
||||
assert "rdt 异常退出" in msg
|
||||
assert ch.active_backend is None
|
||||
|
||||
def test_reports_error_with_reinstall_hint_when_broken(self, monkeypatch):
|
||||
"""which 命中但 exec 抛 FileNotFoundError(venv 断链)→ error + 重装处方。"""
|
||||
self._isolate(monkeypatch)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/rdt")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
raise FileNotFoundError("/usr/local/bin/rdt")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.reddit import RedditChannel
|
||||
ch = RedditChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "error"
|
||||
assert "无法执行" in msg
|
||||
assert "pipx install --force" in msg # rdt 专用 git 源重装处方
|
||||
assert "git+https://github.com/public-clis/rdt-cli.git" in msg
|
||||
assert ch.active_backend is None
|
||||
|
||||
def test_reports_error_with_reinstall_hint_on_exit_127(self, monkeypatch):
|
||||
"""退出码 127(找到但跑不动)同样按断链处理。"""
|
||||
self._isolate(monkeypatch)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/rdt")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 127, "", "")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.reddit import RedditChannel
|
||||
ch = RedditChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "error"
|
||||
assert "pipx install --force" in msg
|
||||
assert ch.active_backend is None
|
||||
|
||||
def test_can_handle_reddit_urls(self):
|
||||
from agent_reach.channels.reddit import RedditChannel
|
||||
@@ -713,7 +779,25 @@ class TestRedditChannel:
|
||||
|
||||
|
||||
class TestXiaoHongShuChannel:
|
||||
"""多后端选择逻辑:OpenCLI > xiaohongshu-mcp > xhs-cli,第一个完整可用者获胜。"""
|
||||
|
||||
@staticmethod
|
||||
def _isolate(monkeypatch, opencli=None, mcp_reachable=False):
|
||||
"""隔离 OpenCLI / mcp 候选,让测试聚焦目标后端。
|
||||
|
||||
opencli: None 表示未安装;否则传入 (status, message) 二元组。
|
||||
"""
|
||||
import agent_reach.channels.xiaohongshu as xhs_mod
|
||||
|
||||
monkeypatch.setattr(
|
||||
XiaoHongShuChannel, "_check_opencli", lambda self: opencli
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
xhs_mod, "_mcp_service_reachable", lambda timeout=3: mcp_reachable
|
||||
)
|
||||
|
||||
def test_reports_ok_when_cli_authenticated(self, monkeypatch):
|
||||
self._isolate(monkeypatch)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/xhs")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
@@ -721,11 +805,14 @@ class TestXiaoHongShuChannel:
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
|
||||
status, msg = XiaoHongShuChannel().check()
|
||||
ch = XiaoHongShuChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert "完整可用" in msg
|
||||
assert "xhs-cli 可用" in msg
|
||||
assert ch.active_backend == "xhs-cli (xiaohongshu-cli)"
|
||||
|
||||
def test_reports_warn_when_not_authenticated(self, monkeypatch):
|
||||
self._isolate(monkeypatch)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/xhs")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
@@ -733,12 +820,392 @@ class TestXiaoHongShuChannel:
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
|
||||
status, msg = XiaoHongShuChannel().check()
|
||||
ch = XiaoHongShuChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "warn"
|
||||
assert "xhs login" in msg
|
||||
# 未登录是业务态:工具进程活着,后端仍可用
|
||||
assert ch.active_backend == "xhs-cli (xiaohongshu-cli)"
|
||||
|
||||
def test_reports_off_when_not_installed(self, monkeypatch):
|
||||
def test_reports_off_when_nothing_installed(self, monkeypatch):
|
||||
self._isolate(monkeypatch)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: None)
|
||||
status, msg = XiaoHongShuChannel().check()
|
||||
ch = XiaoHongShuChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "off"
|
||||
assert "xiaohongshu-cli" in msg
|
||||
# off 指引推荐当代后端,而非停更的 xhs-cli
|
||||
assert "opencli" in msg
|
||||
assert "xiaohongshu-mcp" in msg
|
||||
assert ch.active_backend is None
|
||||
|
||||
def test_reports_error_with_reinstall_hint_when_broken(self, monkeypatch):
|
||||
"""which 命中但 exec 抛 FileNotFoundError(venv 断链)→ error + 重装处方。"""
|
||||
self._isolate(monkeypatch)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/xhs")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
raise FileNotFoundError("/usr/local/bin/xhs")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
|
||||
ch = XiaoHongShuChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "error"
|
||||
assert "无法执行" in msg
|
||||
assert "uv tool install --force xiaohongshu-cli" in msg
|
||||
assert "pipx reinstall xiaohongshu-cli" in msg
|
||||
assert ch.active_backend is None
|
||||
|
||||
def test_opencli_ready_wins_over_cli(self, monkeypatch):
|
||||
"""OpenCLI 完整可用时按序获胜,即使 xhs-cli 也完整可用。"""
|
||||
self._isolate(monkeypatch, opencli=("ok", "OpenCLI 可用(复用浏览器登录态)"))
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/xhs")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 0, "ok: true\n", "")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
|
||||
ch = XiaoHongShuChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert ch.active_backend == "OpenCLI"
|
||||
|
||||
def test_opencli_warn_loses_to_usable_cli(self, monkeypatch):
|
||||
"""OpenCLI 装了但扩展未连(warn)时,完整可用的 xhs-cli 获胜。"""
|
||||
self._isolate(monkeypatch, opencli=("warn", "扩展未连接"))
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/xhs")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 0, "ok: true\n", "")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
|
||||
ch = XiaoHongShuChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert ch.active_backend == "xhs-cli (xiaohongshu-cli)"
|
||||
|
||||
def test_mcp_service_wins_when_opencli_absent(self, monkeypatch):
|
||||
"""服务器场景:OpenCLI 未装、mcp 服务可达且 mcporter 已接入 → mcp 获胜。"""
|
||||
self._isolate(monkeypatch, mcp_reachable=True)
|
||||
|
||||
def fake_which(name):
|
||||
return "/usr/local/bin/mcporter" if name == "mcporter" else None
|
||||
|
||||
monkeypatch.setattr(shutil, "which", fake_which)
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 0, "exa\nxiaohongshu\n", "")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
|
||||
ch = XiaoHongShuChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert ch.active_backend == "xiaohongshu-mcp"
|
||||
assert "search_feeds" in msg
|
||||
|
||||
def test_mcp_reachable_but_mcporter_unconfigured_warns(self, monkeypatch):
|
||||
self._isolate(monkeypatch, mcp_reachable=True)
|
||||
|
||||
def fake_which(name):
|
||||
return "/usr/local/bin/mcporter" if name == "mcporter" else None
|
||||
|
||||
monkeypatch.setattr(shutil, "which", fake_which)
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 0, "exa\n", "")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
|
||||
ch = XiaoHongShuChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "warn"
|
||||
assert "mcporter config add xiaohongshu" in msg
|
||||
assert ch.active_backend == "xiaohongshu-mcp"
|
||||
|
||||
def test_backend_override_prefers_cli(self, monkeypatch):
|
||||
"""config xiaohongshu_backend=xhs-cli 时,即使 OpenCLI ready 也用 xhs-cli。"""
|
||||
self._isolate(monkeypatch, opencli=("ok", "OpenCLI 可用"))
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/xhs")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 0, "ok: true\n", "")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
|
||||
class _Cfg:
|
||||
def get(self, key, default=None):
|
||||
return "xhs-cli" if key == "xiaohongshu_backend" else default
|
||||
|
||||
ch = XiaoHongShuChannel()
|
||||
status, _ = ch.check(_Cfg())
|
||||
assert status == "ok"
|
||||
assert ch.active_backend == "xhs-cli (xiaohongshu-cli)"
|
||||
|
||||
|
||||
class TestBilibiliChannel:
|
||||
"""多后端:bili-cli > OpenCLI > 搜索 API。yt-dlp 已退出 B站(412 实锤)。"""
|
||||
|
||||
@staticmethod
|
||||
def _isolate(monkeypatch, opencli=None, api_ok=False):
|
||||
import agent_reach.channels.bilibili as bilibili_mod
|
||||
monkeypatch.setattr(
|
||||
bilibili_mod.BilibiliChannel, "_check_opencli", lambda self: opencli
|
||||
)
|
||||
monkeypatch.setattr(bilibili_mod, "_search_api_ok", lambda: api_ok)
|
||||
|
||||
def test_bili_cli_ok_is_active_backend(self, monkeypatch):
|
||||
self._isolate(monkeypatch)
|
||||
monkeypatch.setattr(
|
||||
shutil, "which",
|
||||
lambda cmd: "/usr/local/bin/bili" if cmd == "bili" else None,
|
||||
)
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 0, "bili, version 0.6.2", "")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.bilibili import BilibiliChannel
|
||||
ch = BilibiliChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert "bili-cli 可用" in msg
|
||||
assert ch.active_backend == "bili-cli"
|
||||
|
||||
def test_bili_broken_falls_back_to_api_with_hint_kept(self, monkeypatch):
|
||||
"""bili 断链 → API 兜底获胜,但重装处方必须保留在消息里。"""
|
||||
self._isolate(monkeypatch, api_ok=True)
|
||||
monkeypatch.setattr(
|
||||
shutil, "which",
|
||||
lambda cmd: "/usr/local/bin/bili" if cmd == "bili" else None,
|
||||
)
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
raise FileNotFoundError(cmd[0])
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.bilibili import BilibiliChannel
|
||||
ch = BilibiliChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok" # 搜索 API 兜底
|
||||
assert ch.active_backend == "B站搜索 API"
|
||||
assert "备选后端异常" in msg
|
||||
assert "pipx reinstall bilibili-cli" in msg
|
||||
|
||||
def test_bili_broken_and_no_fallback_reports_error(self, monkeypatch):
|
||||
self._isolate(monkeypatch, api_ok=False)
|
||||
monkeypatch.setattr(
|
||||
shutil, "which",
|
||||
lambda cmd: "/usr/local/bin/bili" if cmd == "bili" else None,
|
||||
)
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
raise FileNotFoundError(cmd[0])
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.bilibili import BilibiliChannel
|
||||
ch = BilibiliChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "error"
|
||||
assert "uv tool install --force bilibili-cli" in msg
|
||||
assert ch.active_backend is None
|
||||
|
||||
def test_opencli_serves_when_bili_missing(self, monkeypatch):
|
||||
self._isolate(monkeypatch, opencli=("ok", "OpenCLI 可用(字幕)"), api_ok=True)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: None)
|
||||
from agent_reach.channels.bilibili import BilibiliChannel
|
||||
ch = BilibiliChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert ch.active_backend == "OpenCLI"
|
||||
|
||||
def test_api_only_still_ok_with_install_nudge(self, monkeypatch):
|
||||
self._isolate(monkeypatch, api_ok=True)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: None)
|
||||
from agent_reach.channels.bilibili import BilibiliChannel
|
||||
ch = BilibiliChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert ch.active_backend == "B站搜索 API"
|
||||
assert "bilibili-cli" in msg
|
||||
|
||||
def test_off_when_everything_unreachable(self, monkeypatch):
|
||||
self._isolate(monkeypatch, api_ok=False)
|
||||
monkeypatch.setattr(shutil, "which", lambda _: None)
|
||||
from agent_reach.channels.bilibili import BilibiliChannel
|
||||
ch = BilibiliChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "off"
|
||||
assert ch.active_backend is None
|
||||
|
||||
|
||||
class TestYouTubeChannel:
|
||||
def test_reports_error_with_reinstall_hint_when_broken(self, monkeypatch):
|
||||
"""yt-dlp which 命中但 exec 抛 FileNotFoundError → error + 重装处方。"""
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/yt-dlp")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
raise FileNotFoundError(cmd[0])
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.youtube import YouTubeChannel
|
||||
ch = YouTubeChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "error"
|
||||
assert "无法执行" in msg
|
||||
assert "uv tool install --force yt-dlp" in msg
|
||||
assert ch.active_backend is None
|
||||
|
||||
|
||||
class TestGitHubChannel:
|
||||
def test_reports_error_with_reinstall_hint_when_broken(self, monkeypatch):
|
||||
"""gh which 命中但 exec 失败 → error + brew 重装处方(gh 不是 pip 包)。"""
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/gh")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
raise FileNotFoundError(cmd[0])
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.github import GitHubChannel
|
||||
ch = GitHubChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "error"
|
||||
assert "无法执行" in msg
|
||||
assert "brew reinstall gh" in msg
|
||||
assert ch.active_backend is None
|
||||
|
||||
def test_active_backend_set_when_authenticated(self, monkeypatch):
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/gh")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 0, "Logged in to github.com", "")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.github import GitHubChannel
|
||||
ch = GitHubChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert ch.active_backend == "gh CLI"
|
||||
|
||||
def test_active_backend_set_when_unauthenticated(self, monkeypatch):
|
||||
"""gh auth status 非零退出是正常业务态(未登录):warn 但后端可用。"""
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/gh")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 1, "", "You are not logged in")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.github import GitHubChannel
|
||||
ch = GitHubChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "warn"
|
||||
assert "gh auth login" in msg
|
||||
assert ch.active_backend == "gh CLI"
|
||||
|
||||
|
||||
class TestLinkedInChannel:
|
||||
def test_reports_error_with_reinstall_hint_when_broken(self, monkeypatch):
|
||||
"""mcporter which 命中但 exec 失败 → error + npm 重装处方。"""
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/mcporter")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
raise FileNotFoundError(cmd[0])
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.linkedin import LinkedInChannel
|
||||
ch = LinkedInChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "error"
|
||||
assert "npm install -g mcporter" in msg
|
||||
assert ch.active_backend is None
|
||||
|
||||
def test_active_backend_set_when_linkedin_configured(self, monkeypatch):
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/mcporter")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 0, "linkedin http://localhost:3000/mcp", "")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.linkedin import LinkedInChannel
|
||||
ch = LinkedInChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert ch.active_backend == "linkedin-scraper-mcp"
|
||||
|
||||
def test_off_without_backend_when_linkedin_not_configured(self, monkeypatch):
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/mcporter")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 0, "exa https://mcp.exa.ai/mcp", "")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.linkedin import LinkedInChannel
|
||||
ch = LinkedInChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "off"
|
||||
assert ch.active_backend is None
|
||||
|
||||
|
||||
class TestExaSearchChannel:
|
||||
def test_reports_error_with_reinstall_hint_when_broken(self, monkeypatch):
|
||||
"""mcporter which 命中但 exec 失败 → error + npm 重装处方。"""
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/mcporter")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
raise FileNotFoundError(cmd[0])
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.exa_search import ExaSearchChannel
|
||||
ch = ExaSearchChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "error"
|
||||
assert "npm install -g mcporter" in msg
|
||||
assert ch.active_backend is None
|
||||
|
||||
def test_active_backend_set_when_exa_configured(self, monkeypatch):
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/mcporter")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 0, "exa https://mcp.exa.ai/mcp", "")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.exa_search import ExaSearchChannel
|
||||
ch = ExaSearchChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert ch.active_backend == "Exa via mcporter"
|
||||
|
||||
|
||||
class TestXiaoyuzhouChannel:
|
||||
def test_reports_error_with_reinstall_hint_when_ffmpeg_broken(self, monkeypatch):
|
||||
"""ffmpeg which 命中但 exec 失败(pip 假 ffmpeg 断链)→ error + 重装处方。"""
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/ffmpeg")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
raise FileNotFoundError(cmd[0])
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
from agent_reach.channels.xiaoyuzhou import XiaoyuzhouChannel
|
||||
ch = XiaoyuzhouChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "error"
|
||||
assert "无法执行" in msg
|
||||
assert "brew install ffmpeg" in msg
|
||||
assert ch.active_backend is None
|
||||
|
||||
def test_active_backend_set_when_fully_configured(self, monkeypatch):
|
||||
monkeypatch.setattr(shutil, "which", lambda _: "/usr/local/bin/ffmpeg")
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
return subprocess.CompletedProcess(cmd, 0, "ffmpeg version 7.0", "")
|
||||
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
monkeypatch.setattr("os.path.isfile", lambda p: True) # transcribe.sh 已安装
|
||||
monkeypatch.setenv("GROQ_API_KEY", "gsk_test")
|
||||
from agent_reach.channels.xiaoyuzhou import XiaoyuzhouChannel
|
||||
ch = XiaoyuzhouChannel()
|
||||
status, msg = ch.check()
|
||||
assert status == "ok"
|
||||
assert ch.active_backend == "groq-whisper"
|
||||
|
||||
+100
-2
@@ -1,9 +1,12 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for Agent Reach CLI."""
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
import requests
|
||||
from unittest.mock import patch
|
||||
import agent_reach.cli as cli
|
||||
from agent_reach.cli import main
|
||||
|
||||
@@ -30,6 +33,21 @@ class TestCLI:
|
||||
assert "Agent Reach" in captured.out
|
||||
assert "✅" in captured.out
|
||||
|
||||
def test_transcribe_command_prints_text(self, capsys):
|
||||
with patch("agent_reach.transcribe.transcribe", return_value="hello transcript"):
|
||||
with patch("sys.argv", ["agent-reach", "transcribe", "audio.mp3"]):
|
||||
main()
|
||||
captured = capsys.readouterr()
|
||||
assert "hello transcript" in captured.out
|
||||
|
||||
def test_transcribe_command_writes_output_file(self, capsys, tmp_path):
|
||||
out_file = tmp_path / "t.txt"
|
||||
with patch("agent_reach.transcribe.transcribe", return_value="saved text"):
|
||||
with patch("sys.argv", ["agent-reach", "transcribe", "audio.mp3", "-o", str(out_file)]):
|
||||
main()
|
||||
assert out_file.read_text(encoding="utf-8").strip() == "saved text"
|
||||
assert "Transcript written" in capsys.readouterr().out
|
||||
|
||||
def test_parse_twitter_cookie_input_separate_values(self):
|
||||
auth_token, ct0 = cli._parse_twitter_cookie_input("token123 ct0abc")
|
||||
assert auth_token == "token123"
|
||||
@@ -42,6 +60,47 @@ class TestCLI:
|
||||
assert auth_token == "token123"
|
||||
assert ct0 == "ct0abc"
|
||||
|
||||
def test_install_rdt_cli_prefers_github_source(self, monkeypatch, capsys):
|
||||
state = {"rdt_installed": False}
|
||||
commands = []
|
||||
|
||||
def fake_which(name):
|
||||
if name == "rdt":
|
||||
return "/usr/local/bin/rdt" if state["rdt_installed"] else None
|
||||
if name == "pipx":
|
||||
return "/usr/local/bin/pipx"
|
||||
return None
|
||||
|
||||
def fake_run(cmd, **kwargs):
|
||||
commands.append(cmd)
|
||||
state["rdt_installed"] = True
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr(shutil, "which", fake_which)
|
||||
monkeypatch.setattr(subprocess, "run", fake_run)
|
||||
|
||||
cli._install_rdt_cli()
|
||||
|
||||
out = capsys.readouterr().out
|
||||
assert commands == [["pipx", "install", cli._RDT_GIT_SOURCE]]
|
||||
assert "✅ rdt-cli installed" in out
|
||||
|
||||
def test_install_reddit_deps_routes_by_environment(self, monkeypatch):
|
||||
"""桌面 → OpenCLI;服务器 → rdt-cli(钉 git 源)。"""
|
||||
calls = []
|
||||
monkeypatch.setattr(cli, "_install_opencli_deps", lambda: calls.append("opencli"))
|
||||
monkeypatch.setattr(cli, "_install_rdt_cli", lambda: calls.append("rdt"))
|
||||
monkeypatch.setattr(shutil, "which", lambda _: None)
|
||||
|
||||
monkeypatch.setattr(cli, "_detect_environment", lambda: "local")
|
||||
cli._install_reddit_deps()
|
||||
assert calls == ["opencli"]
|
||||
|
||||
calls.clear()
|
||||
monkeypatch.setattr(cli, "_detect_environment", lambda: "server")
|
||||
cli._install_reddit_deps()
|
||||
assert calls == ["rdt"]
|
||||
|
||||
|
||||
class TestCheckUpdateRetry:
|
||||
def test_retry_timeout_classification(self):
|
||||
@@ -89,7 +148,7 @@ class TestCheckUpdateRetry:
|
||||
|
||||
sequence = [
|
||||
R(429, headers={"Retry-After": "3"}),
|
||||
R(200, payload={"tag_name": "v1.4.0"}),
|
||||
R(200, payload={"tag_name": "v1.5.0"}),
|
||||
]
|
||||
|
||||
with patch("requests.get", side_effect=sequence):
|
||||
@@ -124,3 +183,42 @@ class TestCheckUpdateRetry:
|
||||
assert result == "error"
|
||||
assert "网络超时" in captured.out
|
||||
assert "已重试 3 次" in captured.out
|
||||
|
||||
|
||||
class TestVersionCompare:
|
||||
def test_newer_remote_triggers_update(self):
|
||||
assert cli._is_newer_version("1.5.0", "1.4.2") is True
|
||||
|
||||
def test_equal_versions_no_update(self):
|
||||
assert cli._is_newer_version("1.5.0", "1.5.0") is False
|
||||
|
||||
def test_local_ahead_of_release_no_downgrade_prompt(self):
|
||||
"""发版窗口期本地装了 main(更新)时,不能提示"有更新"诱导降级。"""
|
||||
assert cli._is_newer_version("1.4.2", "1.5.0") is False
|
||||
|
||||
def test_unparseable_falls_back_to_inequality(self):
|
||||
assert cli._is_newer_version("2026.06-beta", "1.5.0") is True
|
||||
assert cli._is_newer_version("1.5.0", "1.5.0-dev") is True
|
||||
|
||||
|
||||
class TestWatchVersionCompare:
|
||||
def test_watch_does_not_prompt_downgrade(self, monkeypatch, capsys):
|
||||
"""watch 与 check-update 同语义:本地领先远端 release 时不提示更新。"""
|
||||
class R:
|
||||
status_code = 200
|
||||
headers = {}
|
||||
|
||||
@staticmethod
|
||||
def json():
|
||||
return {"tag_name": "v1.4.2", "body": ""}
|
||||
|
||||
monkeypatch.setattr(cli, "_github_get_with_retry", lambda *a, **k: (R(), None, 1))
|
||||
monkeypatch.setattr(
|
||||
"agent_reach.doctor.check_all",
|
||||
lambda config: {"web": {"status": "ok", "name": "任意网页", "message": "ok",
|
||||
"tier": 0, "backends": ["Jina Reader"], "active_backend": "Jina Reader"}},
|
||||
)
|
||||
cli._cmd_watch()
|
||||
out = capsys.readouterr().out
|
||||
assert "新版本可用" not in out
|
||||
assert "全部正常" in out
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Verify credential files written by cookie sync helpers and CLI helpers
|
||||
are owner-only (0o600) and that values containing shell metacharacters do
|
||||
not break the shell-sourceable env file produced by _sync_bird_env().
|
||||
|
||||
Companion to tests/test_config.py::test_save_creates_file_with_restricted_permissions —
|
||||
the same threat-model claim ("Cookie/Token only stored locally, 600
|
||||
permissions") covers these paths.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import stat
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
import pytest
|
||||
|
||||
from agent_reach.cookie_extract import _sync_bird_env, _sync_xfetch_session
|
||||
|
||||
|
||||
def _owner_only(path: str) -> bool:
|
||||
mode = os.stat(path).st_mode
|
||||
return not (mode & (stat.S_IRGRP | stat.S_IWGRP | stat.S_IROTH | stat.S_IWOTH))
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="POSIX perm semantics only")
|
||||
def test_sync_xfetch_session_writes_0600(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
_sync_xfetch_session("auth_xxx", "ct0_yyy")
|
||||
session_path = tmp_path / ".config" / "xfetch" / "session.json"
|
||||
assert session_path.exists(), "expected ~/.config/xfetch/session.json"
|
||||
assert _owner_only(str(session_path)), "session.json must be 0o600"
|
||||
# Round-trip the content so we know we didn't accidentally corrupt JSON.
|
||||
data = json.loads(session_path.read_text(encoding="utf-8"))
|
||||
assert data["authToken"] == "auth_xxx"
|
||||
assert data["ct0"] == "ct0_yyy"
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="POSIX perm semantics only")
|
||||
def test_sync_bird_env_writes_0600(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
_sync_bird_env("auth_xxx", "ct0_yyy")
|
||||
env_path = tmp_path / ".config" / "bird" / "credentials.env"
|
||||
assert env_path.exists(), "expected ~/.config/bird/credentials.env"
|
||||
assert _owner_only(str(env_path)), "credentials.env must be 0o600"
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="POSIX sh needed for sourcing")
|
||||
def test_sync_bird_env_quotes_shell_metachars(tmp_path, monkeypatch):
|
||||
"""Tokens containing ", $, `, ; etc. must not break out of the assignment.
|
||||
|
||||
Prior implementation used `f'AUTH_TOKEN="{auth_token}"'` which an attacker-
|
||||
controlled cookie containing a literal `"` could break out of, turning a
|
||||
later `source ~/.config/bird/credentials.env` into arbitrary shell.
|
||||
"""
|
||||
monkeypatch.setenv("HOME", str(tmp_path))
|
||||
# Side-effect markers live under tmp_path (auto-cleaned by pytest) rather
|
||||
# than a shared absolute /tmp path — otherwise one vulnerable run leaves a
|
||||
# marker behind that fails every later run on the same machine/CI runner.
|
||||
pwn_auth = tmp_path / "pwn-auth"
|
||||
pwn_ct0 = tmp_path / "pwn-ct0"
|
||||
hostile_auth = f'inj"; touch {pwn_auth}; #'
|
||||
hostile_ct0 = f"ct0_$(touch {pwn_ct0})"
|
||||
_sync_bird_env(hostile_auth, hostile_ct0)
|
||||
env_path = tmp_path / ".config" / "bird" / "credentials.env"
|
||||
|
||||
# Sourcing the file must NOT execute the injected payload. Read back the
|
||||
# exported values from a subshell instead — they should equal the originals.
|
||||
probe = (
|
||||
f". {env_path}; "
|
||||
f'printf "AUTH=%s\\nCT0=%s\\n" "$AUTH_TOKEN" "$CT0"'
|
||||
)
|
||||
result = subprocess.run(
|
||||
["sh", "-c", probe],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=5,
|
||||
)
|
||||
assert result.returncode == 0, result.stderr
|
||||
lines = dict(
|
||||
line.split("=", 1) for line in result.stdout.strip().splitlines() if "=" in line
|
||||
)
|
||||
assert lines["AUTH"] == hostile_auth, "auth_token round-trip broke — injection possible"
|
||||
assert lines["CT0"] == hostile_ct0, "ct0 round-trip broke — injection possible"
|
||||
# And no side-effect files materialised.
|
||||
assert not pwn_auth.exists()
|
||||
assert not pwn_ct0.exists()
|
||||
+28
-2
@@ -8,13 +8,15 @@ from agent_reach.config import Config
|
||||
|
||||
|
||||
class _StubChannel:
|
||||
def __init__(self, name, description, tier, status, message, backends=None):
|
||||
def __init__(self, name, description, tier, status, message, backends=None,
|
||||
active_backend=None):
|
||||
self.name = name
|
||||
self.description = description
|
||||
self.tier = tier
|
||||
self._status = status
|
||||
self._message = message
|
||||
self.backends = backends or []
|
||||
self.active_backend = active_backend
|
||||
|
||||
def check(self, config=None):
|
||||
return self._status, self._message
|
||||
@@ -31,7 +33,8 @@ class TestDoctor:
|
||||
doctor,
|
||||
"get_all_channels",
|
||||
lambda: [
|
||||
_StubChannel("web", "网页", 0, "ok", "可抓取网页", ["requests"]),
|
||||
_StubChannel("web", "网页", 0, "ok", "可抓取网页", ["requests"],
|
||||
active_backend="requests"),
|
||||
_StubChannel("github", "GitHub", 0, "warn", "gh 未安装", ["gh"]),
|
||||
_StubChannel("exa_search", "全网语义搜索", 1, "off", "mcporter 未配置", ["Exa"]),
|
||||
],
|
||||
@@ -46,6 +49,7 @@ class TestDoctor:
|
||||
"message": "可抓取网页",
|
||||
"tier": 0,
|
||||
"backends": ["requests"],
|
||||
"active_backend": "requests",
|
||||
},
|
||||
"github": {
|
||||
"status": "warn",
|
||||
@@ -53,6 +57,7 @@ class TestDoctor:
|
||||
"message": "gh 未安装",
|
||||
"tier": 0,
|
||||
"backends": ["gh"],
|
||||
"active_backend": None,
|
||||
},
|
||||
"exa_search": {
|
||||
"status": "off",
|
||||
@@ -60,6 +65,7 @@ class TestDoctor:
|
||||
"message": "mcporter 未配置",
|
||||
"tier": 1,
|
||||
"backends": ["Exa"],
|
||||
"active_backend": None,
|
||||
},
|
||||
}
|
||||
|
||||
@@ -98,3 +104,23 @@ class TestDoctor:
|
||||
assert "1/3 个渠道可用" in plain
|
||||
# Inactive optional channels should be summarized in one line
|
||||
assert "可选渠道可以解锁" in plain
|
||||
|
||||
|
||||
def test_stale_active_backend_does_not_leak_into_errored_result(monkeypatch):
|
||||
"""渠道单例上一轮的 active_backend 不得泄漏进本轮异常结果(Codex review 发现)。"""
|
||||
from agent_reach import doctor
|
||||
|
||||
class _ExplodingChannel:
|
||||
name = "boom"
|
||||
description = "爆炸渠道"
|
||||
tier = 0
|
||||
backends = ["a", "b"]
|
||||
active_backend = "a" # 上一轮成功的残留
|
||||
|
||||
def check(self, config=None):
|
||||
raise RuntimeError("boom")
|
||||
|
||||
monkeypatch.setattr(doctor, "get_all_channels", lambda: [_ExplodingChannel()])
|
||||
results = doctor.check_all(config=None)
|
||||
assert results["boom"]["status"] == "error"
|
||||
assert results["boom"]["active_backend"] is None
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for the OpenCLI cross-channel backend probing."""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
from agent_reach.backends import opencli_status, opencli_summary
|
||||
from agent_reach.probe import ProbeResult
|
||||
|
||||
|
||||
def _status_with(version_probe, daemon_probe=None, ext_on_disk=False):
|
||||
"""Run opencli_status with probe_command and disk check patched."""
|
||||
calls = []
|
||||
|
||||
def fake_probe(cmd, args=("--version",), **kwargs):
|
||||
calls.append(list(args))
|
||||
if list(args) == ["--version"]:
|
||||
return version_probe
|
||||
return daemon_probe or ProbeResult("missing")
|
||||
|
||||
with patch("agent_reach.backends.opencli.probe_command", side_effect=fake_probe), \
|
||||
patch(
|
||||
"agent_reach.backends.opencli._extension_installed_on_disk",
|
||||
return_value=ext_on_disk,
|
||||
):
|
||||
return opencli_status(), calls
|
||||
|
||||
|
||||
def test_not_installed():
|
||||
st, _ = _status_with(ProbeResult("missing"))
|
||||
assert not st.installed
|
||||
assert not st.ready
|
||||
assert "未安装" in opencli_summary(st)
|
||||
|
||||
|
||||
def test_broken_node_env_gives_npm_hint():
|
||||
st, _ = _status_with(ProbeResult("broken", hint="x"))
|
||||
assert st.installed and st.broken
|
||||
assert "npm install -g @jackwener/opencli" in st.hint
|
||||
assert not st.ready
|
||||
|
||||
|
||||
def test_daemon_running_extension_connected_is_ready():
|
||||
daemon_out = "Daemon: running (PID 37389)\nVersion: v1.8.3\nExtension: connected\n"
|
||||
st, _ = _status_with(
|
||||
ProbeResult("ok", output="1.8.3"),
|
||||
ProbeResult("ok", output=daemon_out),
|
||||
)
|
||||
assert st.installed and st.daemon_running and st.extension_connected
|
||||
assert st.ready
|
||||
assert "1.8.3" in opencli_summary(st)
|
||||
|
||||
|
||||
def test_extension_never_installed_not_ready_with_store_guide():
|
||||
daemon_out = "Daemon: running (PID 1)\nExtension: disconnected\n"
|
||||
st, _ = _status_with(
|
||||
ProbeResult("ok", output="1.8.3"),
|
||||
ProbeResult("ok", output=daemon_out),
|
||||
ext_on_disk=False,
|
||||
)
|
||||
assert st.daemon_running and not st.extension_connected
|
||||
assert not st.ready
|
||||
assert "chromewebstore.google.com" in st.hint
|
||||
|
||||
|
||||
def test_sleeping_extension_counts_as_ready():
|
||||
"""实测:扩展 service worker 睡眠时 daemon status 报 disconnected,
|
||||
但任何真实命令会唤醒它——装在磁盘上即视为可用。"""
|
||||
daemon_out = "Daemon: running (PID 1)\nExtension: disconnected\n"
|
||||
st, _ = _status_with(
|
||||
ProbeResult("ok", output="1.8.3"),
|
||||
ProbeResult("ok", output=daemon_out),
|
||||
ext_on_disk=True,
|
||||
)
|
||||
assert not st.extension_connected
|
||||
assert st.extension_installed
|
||||
assert st.ready
|
||||
assert "唤醒" in opencli_summary(st)
|
||||
assert st.hint == ""
|
||||
|
||||
|
||||
def test_daemon_not_running_parsed_correctly():
|
||||
st, _ = _status_with(
|
||||
ProbeResult("ok", output="1.8.3"),
|
||||
ProbeResult("ok", output="Daemon: not running\n"),
|
||||
)
|
||||
assert st.installed
|
||||
assert not st.daemon_running
|
||||
assert not st.extension_connected
|
||||
assert "自动启动" in opencli_summary(st)
|
||||
|
||||
|
||||
def test_probe_uses_daemon_status_not_doctor():
|
||||
"""`opencli doctor` auto-starts the daemon (side effect) — health checks
|
||||
must only ever call `daemon status`."""
|
||||
_, calls = _status_with(
|
||||
ProbeResult("ok", output="1.8.3"),
|
||||
ProbeResult("ok", output="Daemon: not running\n"),
|
||||
)
|
||||
assert ["daemon", "status"] in calls
|
||||
assert ["doctor"] not in calls
|
||||
@@ -0,0 +1,90 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for agent_reach.probe — real-execution probing and failure classification."""
|
||||
|
||||
import os
|
||||
import stat
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from agent_reach.probe import ProbeResult, probe_command, reinstall_hint
|
||||
|
||||
|
||||
def _make_executable(path, content):
|
||||
path.write_text(content)
|
||||
path.chmod(path.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)
|
||||
return str(path)
|
||||
|
||||
|
||||
def test_missing_command():
|
||||
r = probe_command("definitely-not-a-real-command-xyz")
|
||||
assert r.status == "missing"
|
||||
assert not r.ok
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="shebang semantics are POSIX-only")
|
||||
def test_broken_shebang_detected_as_broken(tmp_path, monkeypatch):
|
||||
"""A stale venv shim: which() finds it, exec raises FileNotFoundError."""
|
||||
script = _make_executable(
|
||||
tmp_path / "stale-tool", "#!/nonexistent/venv/bin/python\nprint('hi')\n"
|
||||
)
|
||||
monkeypatch.setenv("PATH", str(tmp_path) + os.pathsep + os.environ.get("PATH", ""))
|
||||
|
||||
r = probe_command("stale-tool", package="stale-tool-pkg")
|
||||
assert r.status == "broken"
|
||||
assert "uv tool install --force stale-tool-pkg" in r.hint
|
||||
assert "pipx reinstall stale-tool-pkg" in r.hint
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="shell script fixture is POSIX-only")
|
||||
def test_healthy_command_returns_ok_with_output(tmp_path, monkeypatch):
|
||||
script = _make_executable(
|
||||
tmp_path / "healthy-tool", "#!/bin/sh\necho 'healthy-tool 1.2.3'\n"
|
||||
)
|
||||
monkeypatch.setenv("PATH", str(tmp_path) + os.pathsep + os.environ.get("PATH", ""))
|
||||
|
||||
r = probe_command("healthy-tool")
|
||||
assert r.ok
|
||||
assert "1.2.3" in r.output
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="shell script fixture is POSIX-only")
|
||||
def test_nonzero_exit_classified_as_error(tmp_path, monkeypatch):
|
||||
script = _make_executable(
|
||||
tmp_path / "failing-tool", "#!/bin/sh\necho 'boom' >&2\nexit 3\n"
|
||||
)
|
||||
monkeypatch.setenv("PATH", str(tmp_path) + os.pathsep + os.environ.get("PATH", ""))
|
||||
|
||||
r = probe_command("failing-tool")
|
||||
assert r.status == "error"
|
||||
assert "boom" in r.output
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="shell script fixture is POSIX-only")
|
||||
def test_exit_127_classified_as_broken(tmp_path, monkeypatch):
|
||||
script = _make_executable(tmp_path / "wrapper-tool", "#!/bin/sh\nexit 127\n")
|
||||
monkeypatch.setenv("PATH", str(tmp_path) + os.pathsep + os.environ.get("PATH", ""))
|
||||
|
||||
r = probe_command("wrapper-tool", package="wrapper-pkg")
|
||||
assert r.status == "broken"
|
||||
assert "wrapper-pkg" in r.hint
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform == "win32", reason="shell script fixture is POSIX-only")
|
||||
def test_retries_help_transient_failures(tmp_path, monkeypatch):
|
||||
"""First call fails (exit 1), second succeeds — retries=1 should return ok."""
|
||||
marker = tmp_path / "ran-once"
|
||||
script = _make_executable(
|
||||
tmp_path / "flaky-tool",
|
||||
f"#!/bin/sh\nif [ -f {marker} ]; then echo ok; exit 0; fi\ntouch {marker}\nexit 1\n",
|
||||
)
|
||||
monkeypatch.setenv("PATH", str(tmp_path) + os.pathsep + os.environ.get("PATH", ""))
|
||||
|
||||
r = probe_command("flaky-tool", retries=1)
|
||||
assert r.ok
|
||||
|
||||
|
||||
def test_reinstall_hint_mentions_both_installers():
|
||||
hint = reinstall_hint("some-pkg")
|
||||
assert "uv tool install --force some-pkg" in hint
|
||||
assert "pipx reinstall some-pkg" in hint
|
||||
@@ -0,0 +1,18 @@
|
||||
from agent_reach.utils.process import mcporter_utf8_env_args, utf8_subprocess_env
|
||||
|
||||
|
||||
def test_utf8_subprocess_env_forces_python_utf8():
|
||||
env = utf8_subprocess_env({"PYTHONUTF8": "0", "OTHER": "value"})
|
||||
|
||||
assert env["PYTHONUTF8"] == "1"
|
||||
assert env["PYTHONIOENCODING"] == "utf-8"
|
||||
assert env["OTHER"] == "value"
|
||||
|
||||
|
||||
def test_mcporter_utf8_env_args():
|
||||
assert mcporter_utf8_env_args() == [
|
||||
"--env",
|
||||
"PYTHONUTF8=1",
|
||||
"--env",
|
||||
"PYTHONIOENCODING=utf-8",
|
||||
]
|
||||
@@ -39,14 +39,11 @@ class TestSkillCommand(unittest.TestCase):
|
||||
with patch.dict(os.environ, env, clear=True):
|
||||
_install_skill()
|
||||
|
||||
target = os.path.join(skill_dir, "agent-reach", "SKILL.md")
|
||||
# Check at least one known skill dir pattern
|
||||
found = False
|
||||
for dirpath, _, filenames in os.walk(tmpdir):
|
||||
if "SKILL.md" in filenames:
|
||||
found = True
|
||||
# Verify content is non-empty
|
||||
with open(os.path.join(dirpath, "SKILL.md")) as f:
|
||||
with open(os.path.join(dirpath, "SKILL.md"), encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
self.assertIn("Agent Reach", content)
|
||||
# _install_skill may or may not find dirs depending on mock; just ensure no crash
|
||||
@@ -58,7 +55,7 @@ class TestSkillCommand(unittest.TestCase):
|
||||
# Create a fake skill installation
|
||||
skill_path = os.path.join(tmpdir, ".openclaw", "skills", "agent-reach")
|
||||
os.makedirs(skill_path)
|
||||
with open(os.path.join(skill_path, "SKILL.md"), "w") as f:
|
||||
with open(os.path.join(skill_path, "SKILL.md"), "w", encoding="utf-8") as f:
|
||||
f.write("test")
|
||||
|
||||
self.assertTrue(os.path.exists(skill_path))
|
||||
@@ -92,7 +89,7 @@ class TestSkillCommand(unittest.TestCase):
|
||||
|
||||
target = os.path.join(skill_parent, "agent-reach", "SKILL.md")
|
||||
self.assertTrue(os.path.exists(target))
|
||||
with open(target) as f:
|
||||
with open(target, encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
self.assertIn("Agent Reach", content)
|
||||
|
||||
@@ -114,10 +111,10 @@ class TestSkillCommand(unittest.TestCase):
|
||||
|
||||
target = os.path.join(skill_parent, "agent-reach", "SKILL.md")
|
||||
self.assertTrue(os.path.exists(target))
|
||||
with open(target) as f:
|
||||
with open(target, encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
self.assertTrue(content.strip())
|
||||
self.assertIn("Give your AI agent eyes to see the entire internet.", content)
|
||||
self.assertIn("Xiaoyuzhou Podcast, LinkedIn", content)
|
||||
self.assertNotIn("搜推特", content)
|
||||
self.assertTrue(
|
||||
os.path.exists(os.path.join(skill_parent, "agent-reach", "references"))
|
||||
|
||||
@@ -0,0 +1,262 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for agent_reach.transcribe — provider routing, fallback, and errors."""
|
||||
|
||||
from typing import List
|
||||
|
||||
import pytest
|
||||
|
||||
from agent_reach import transcribe as tr
|
||||
from agent_reach.config import Config
|
||||
|
||||
# --- Fixtures ----------------------------------------------------------- #
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_config(tmp_path, monkeypatch):
|
||||
"""A Config that writes to a temp dir and never touches the user's HOME."""
|
||||
cfg_path = tmp_path / "config.yaml"
|
||||
monkeypatch.setattr(Config, "CONFIG_DIR", tmp_path)
|
||||
monkeypatch.setattr(Config, "CONFIG_FILE", cfg_path)
|
||||
cfg = Config(config_path=cfg_path)
|
||||
return cfg
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def chunk_file(tmp_path):
|
||||
p = tmp_path / "chunk.m4a"
|
||||
p.write_bytes(b"\x00fake-m4a-bytes")
|
||||
return p
|
||||
|
||||
|
||||
class FakeResponse:
|
||||
def __init__(self, status_code: int, text: str = ""):
|
||||
self.status_code = status_code
|
||||
self.text = text
|
||||
|
||||
@property
|
||||
def ok(self) -> bool:
|
||||
return 200 <= self.status_code < 300
|
||||
|
||||
|
||||
# --- transcribe_chunk: provider routing -------------------------------- #
|
||||
|
||||
|
||||
class TestTranscribeChunk:
|
||||
def test_routes_to_groq_endpoint(self, monkeypatch, fake_config, chunk_file):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
captured = {}
|
||||
|
||||
def fake_post(url, headers=None, files=None, data=None, timeout=None):
|
||||
captured["url"] = url
|
||||
captured["headers"] = headers
|
||||
captured["model"] = data["model"]
|
||||
return FakeResponse(200, "hello world")
|
||||
|
||||
monkeypatch.setattr(tr.requests, "post", fake_post)
|
||||
text = tr.transcribe_chunk(chunk_file, "groq", config=fake_config)
|
||||
assert text == "hello world"
|
||||
assert captured["url"] == tr.PROVIDERS["groq"]["endpoint"]
|
||||
assert captured["model"] == "whisper-large-v3"
|
||||
assert captured["headers"]["Authorization"] == "Bearer gsk_test"
|
||||
|
||||
def test_routes_to_openai_endpoint(self, monkeypatch, fake_config, chunk_file):
|
||||
fake_config.set("openai_api_key", "sk-test")
|
||||
captured = {}
|
||||
|
||||
def fake_post(url, headers=None, files=None, data=None, timeout=None):
|
||||
captured["url"] = url
|
||||
captured["model"] = data["model"]
|
||||
return FakeResponse(200, "openai output")
|
||||
|
||||
monkeypatch.setattr(tr.requests, "post", fake_post)
|
||||
text = tr.transcribe_chunk(chunk_file, "openai", config=fake_config)
|
||||
assert text == "openai output"
|
||||
assert captured["url"] == tr.PROVIDERS["openai"]["endpoint"]
|
||||
assert captured["model"] == "whisper-1"
|
||||
|
||||
def test_raises_when_key_missing(self, fake_config, chunk_file):
|
||||
with pytest.raises(tr.NoProviderConfigured):
|
||||
tr.transcribe_chunk(chunk_file, "groq", config=fake_config)
|
||||
|
||||
def test_raises_on_http_error(self, monkeypatch, fake_config, chunk_file):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
monkeypatch.setattr(
|
||||
tr.requests,
|
||||
"post",
|
||||
lambda *a, **k: FakeResponse(429, "rate limited"),
|
||||
)
|
||||
with pytest.raises(tr.TranscribeError, match="HTTP 429"):
|
||||
tr.transcribe_chunk(chunk_file, "groq", config=fake_config)
|
||||
|
||||
def test_unknown_provider(self, fake_config, chunk_file):
|
||||
with pytest.raises(tr.TranscribeError, match="unknown provider"):
|
||||
tr.transcribe_chunk(chunk_file, "azure", config=fake_config)
|
||||
|
||||
|
||||
# --- _transcribe_with_fallback ----------------------------------------- #
|
||||
|
||||
|
||||
class TestFallback:
|
||||
def test_groq_succeeds_no_openai_call(self, monkeypatch, fake_config, chunk_file):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
fake_config.set("openai_api_key", "sk-test")
|
||||
calls: List[str] = []
|
||||
|
||||
def fake_post(url, headers=None, files=None, data=None, timeout=None):
|
||||
calls.append(url)
|
||||
return FakeResponse(200, "from-groq")
|
||||
|
||||
monkeypatch.setattr(tr.requests, "post", fake_post)
|
||||
text = tr._transcribe_with_fallback(chunk_file, ["groq", "openai"], fake_config)
|
||||
assert text == "from-groq"
|
||||
assert calls == [tr.PROVIDERS["groq"]["endpoint"]]
|
||||
|
||||
def test_groq_429_falls_back_to_openai(self, monkeypatch, fake_config, chunk_file):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
fake_config.set("openai_api_key", "sk-test")
|
||||
calls: List[str] = []
|
||||
|
||||
def fake_post(url, headers=None, files=None, data=None, timeout=None):
|
||||
calls.append(url)
|
||||
if url == tr.PROVIDERS["groq"]["endpoint"]:
|
||||
return FakeResponse(429, "rate limited")
|
||||
return FakeResponse(200, "from-openai")
|
||||
|
||||
monkeypatch.setattr(tr.requests, "post", fake_post)
|
||||
text = tr._transcribe_with_fallback(chunk_file, ["groq", "openai"], fake_config)
|
||||
assert text == "from-openai"
|
||||
assert calls == [
|
||||
tr.PROVIDERS["groq"]["endpoint"],
|
||||
tr.PROVIDERS["openai"]["endpoint"],
|
||||
]
|
||||
|
||||
def test_skip_unconfigured_provider(self, monkeypatch, fake_config, chunk_file):
|
||||
# Only openai key configured — fallback should skip groq silently.
|
||||
fake_config.set("openai_api_key", "sk-test")
|
||||
calls: List[str] = []
|
||||
|
||||
def fake_post(url, headers=None, files=None, data=None, timeout=None):
|
||||
calls.append(url)
|
||||
return FakeResponse(200, "via-openai")
|
||||
|
||||
monkeypatch.setattr(tr.requests, "post", fake_post)
|
||||
text = tr._transcribe_with_fallback(chunk_file, ["groq", "openai"], fake_config)
|
||||
assert text == "via-openai"
|
||||
assert calls == [tr.PROVIDERS["openai"]["endpoint"]]
|
||||
|
||||
def test_all_fail_raises_with_last_error(self, monkeypatch, fake_config, chunk_file):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
fake_config.set("openai_api_key", "sk-test")
|
||||
monkeypatch.setattr(
|
||||
tr.requests,
|
||||
"post",
|
||||
lambda *a, **k: FakeResponse(500, "boom"),
|
||||
)
|
||||
with pytest.raises(tr.TranscribeError, match="all providers failed"):
|
||||
tr._transcribe_with_fallback(chunk_file, ["groq", "openai"], fake_config)
|
||||
|
||||
|
||||
# --- transcribe (orchestrator) ---------------------------------------- #
|
||||
|
||||
|
||||
class TestOrchestrator:
|
||||
def test_local_file_skips_yt_dlp(self, monkeypatch, fake_config, tmp_path, chunk_file):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
|
||||
def boom_download(*a, **k):
|
||||
raise AssertionError("yt-dlp must not be called for local files")
|
||||
|
||||
# Stub heavy external steps to no-ops that keep file paths valid.
|
||||
compressed = tmp_path / "compressed.m4a"
|
||||
compressed.write_bytes(b"x" * 1024)
|
||||
|
||||
def fake_compress(src, out_dir):
|
||||
return compressed
|
||||
|
||||
monkeypatch.setattr(tr, "download_audio", boom_download)
|
||||
monkeypatch.setattr(tr, "compress_audio", fake_compress)
|
||||
monkeypatch.setattr(
|
||||
tr.requests,
|
||||
"post",
|
||||
lambda *a, **k: FakeResponse(200, "transcript text"),
|
||||
)
|
||||
|
||||
text = tr.transcribe(
|
||||
str(chunk_file),
|
||||
out_dir=tmp_path / "work",
|
||||
config=fake_config,
|
||||
)
|
||||
assert text == "transcript text"
|
||||
|
||||
def test_chunks_concatenated_with_newlines(
|
||||
self, monkeypatch, fake_config, tmp_path, chunk_file
|
||||
):
|
||||
fake_config.set("groq_api_key", "gsk_test")
|
||||
# Force the "needs chunking" path by writing a file above the size limit.
|
||||
big = tmp_path / "compressed.m4a"
|
||||
big.write_bytes(b"x" * (tr.SIZE_LIMIT_BYTES + 1))
|
||||
monkeypatch.setattr(tr, "compress_audio", lambda src, out_dir: big)
|
||||
c1 = tmp_path / "chunk_001.m4a"
|
||||
c2 = tmp_path / "chunk_002.m4a"
|
||||
c1.write_bytes(b"a")
|
||||
c2.write_bytes(b"b")
|
||||
monkeypatch.setattr(tr, "chunk_audio", lambda src, out_dir: [c1, c2])
|
||||
|
||||
responses = iter(["part one ", "part two "])
|
||||
monkeypatch.setattr(
|
||||
tr.requests,
|
||||
"post",
|
||||
lambda *a, **k: FakeResponse(200, next(responses)),
|
||||
)
|
||||
|
||||
text = tr.transcribe(
|
||||
str(chunk_file),
|
||||
out_dir=tmp_path / "work",
|
||||
config=fake_config,
|
||||
)
|
||||
assert text == "part one\npart two"
|
||||
|
||||
def test_no_provider_configured_fails_fast(self, fake_config, chunk_file):
|
||||
with pytest.raises(tr.NoProviderConfigured):
|
||||
tr.transcribe(str(chunk_file), config=fake_config)
|
||||
|
||||
def test_invalid_provider_string(self, fake_config, chunk_file):
|
||||
with pytest.raises(tr.TranscribeError, match="unknown provider"):
|
||||
tr.transcribe(str(chunk_file), provider="azure", config=fake_config)
|
||||
|
||||
|
||||
# --- YouTubeChannel integration --------------------------------------- #
|
||||
|
||||
|
||||
class TestYouTubeChannelTranscribe:
|
||||
def test_delegates_to_transcribe(self, monkeypatch, fake_config):
|
||||
from agent_reach.channels.youtube import YouTubeChannel
|
||||
|
||||
captured = {}
|
||||
|
||||
def fake_transcribe(source, *, provider="auto", out_dir=None, config=None):
|
||||
captured["source"] = source
|
||||
captured["provider"] = provider
|
||||
captured["config"] = config
|
||||
return "delegated text"
|
||||
|
||||
monkeypatch.setattr(tr, "transcribe", fake_transcribe)
|
||||
out = YouTubeChannel().transcribe(
|
||||
"https://youtu.be/abc", provider="groq", config=fake_config
|
||||
)
|
||||
assert out == "delegated text"
|
||||
assert captured["source"] == "https://youtu.be/abc"
|
||||
assert captured["provider"] == "groq"
|
||||
assert captured["config"] is fake_config
|
||||
|
||||
|
||||
# --- Config feature requirement --------------------------------------- #
|
||||
|
||||
|
||||
class TestConfigOpenAIWhisper:
|
||||
def test_openai_whisper_feature_registered(self, fake_config):
|
||||
assert "openai_whisper" in Config.FEATURE_REQUIREMENTS
|
||||
assert Config.FEATURE_REQUIREMENTS["openai_whisper"] == ["openai_api_key"]
|
||||
assert not fake_config.is_configured("openai_whisper")
|
||||
fake_config.set("openai_api_key", "sk-test")
|
||||
assert fake_config.is_configured("openai_whisper")
|
||||
@@ -26,6 +26,7 @@ def test_check_twitter_cli_found_and_auth_ok():
|
||||
assert status == "ok"
|
||||
assert "twitter-cli" in message
|
||||
assert "完整可用" in message
|
||||
assert channel.active_backend == "twitter-cli"
|
||||
|
||||
|
||||
def test_check_twitter_cli_found_auth_missing():
|
||||
@@ -41,6 +42,8 @@ def test_check_twitter_cli_found_auth_missing():
|
||||
status, message = channel.check()
|
||||
assert status == "warn"
|
||||
assert "未认证" in message
|
||||
# 未认证是业务态:工具进程活着,后端仍可用
|
||||
assert channel.active_backend == "twitter-cli"
|
||||
|
||||
|
||||
# --- bird CLI fallback tests ---
|
||||
@@ -59,6 +62,7 @@ def test_check_bird_fallback_auth_ok():
|
||||
status, message = channel.check()
|
||||
assert status == "ok"
|
||||
assert "bird" in message
|
||||
assert channel.active_backend == "bird CLI (legacy)"
|
||||
|
||||
|
||||
def test_check_bird_fallback_auth_missing():
|
||||
@@ -86,6 +90,7 @@ def test_check_nothing_installed():
|
||||
status, message = channel.check()
|
||||
assert status == "warn"
|
||||
assert "twitter-cli" in message
|
||||
assert channel.active_backend is None
|
||||
|
||||
|
||||
# --- twitter-cli preferred over bird ---
|
||||
@@ -106,3 +111,74 @@ def test_twitter_cli_preferred_over_bird():
|
||||
status, message = channel.check()
|
||||
assert status == "ok"
|
||||
assert "twitter-cli" in message
|
||||
assert channel.active_backend == "twitter-cli"
|
||||
|
||||
|
||||
# --- broken install (stale venv shim) ---
|
||||
|
||||
def test_check_twitter_cli_broken_reports_error_with_reinstall_hint():
|
||||
"""which 命中但 exec 抛 FileNotFoundError(venv 断链)→ error + 重装处方。"""
|
||||
channel = TwitterChannel()
|
||||
with patch(
|
||||
"shutil.which",
|
||||
side_effect=lambda name: "/usr/local/bin/twitter" if name == "twitter" else None,
|
||||
), patch("subprocess.run", side_effect=FileNotFoundError("/usr/local/bin/twitter")):
|
||||
status, message = channel.check()
|
||||
assert status == "error"
|
||||
assert "无法执行" in message
|
||||
assert "uv tool install --force twitter-cli" in message
|
||||
assert "pipx reinstall twitter-cli" in message
|
||||
assert channel.active_backend is None
|
||||
|
||||
|
||||
def test_check_twitter_cli_broken_falls_back_to_bird():
|
||||
"""twitter-cli 断链但 bird 健康 → 回退到 bird,后端正确归属。"""
|
||||
channel = TwitterChannel()
|
||||
|
||||
def which_side_effect(name):
|
||||
if name in ("twitter", "bird"):
|
||||
return f"/usr/local/bin/{name}"
|
||||
return None
|
||||
|
||||
def run_side_effect(cmd, **kwargs):
|
||||
if "twitter" in cmd[0]:
|
||||
raise FileNotFoundError(cmd[0])
|
||||
return _cp(stdout="Authenticated as @user\n", returncode=0)
|
||||
|
||||
with patch("shutil.which", side_effect=which_side_effect), patch(
|
||||
"subprocess.run", side_effect=run_side_effect
|
||||
):
|
||||
status, message = channel.check()
|
||||
assert status == "ok"
|
||||
assert "bird" in message
|
||||
assert channel.active_backend == "bird CLI (legacy)"
|
||||
|
||||
|
||||
def test_unauthenticated_twitter_cli_does_not_block_working_opencli():
|
||||
"""warn 候选不得屏蔽排在后面的 ok 候选(Codex review 发现)。"""
|
||||
channel = TwitterChannel()
|
||||
with patch.object(
|
||||
TwitterChannel, "_check_twitter_cli",
|
||||
return_value=("warn", "twitter-cli 已安装但未认证"),
|
||||
), patch.object(
|
||||
TwitterChannel, "_check_opencli",
|
||||
return_value=("ok", "OpenCLI 可用(复用浏览器登录态)"),
|
||||
), patch.object(TwitterChannel, "_check_bird", return_value=None):
|
||||
status, msg = channel.check()
|
||||
assert status == "ok"
|
||||
assert channel.active_backend == "OpenCLI"
|
||||
|
||||
|
||||
def test_all_warn_falls_back_to_first_warn():
|
||||
channel = TwitterChannel()
|
||||
with patch.object(
|
||||
TwitterChannel, "_check_twitter_cli",
|
||||
return_value=("warn", "twitter-cli 未认证"),
|
||||
), patch.object(
|
||||
TwitterChannel, "_check_opencli",
|
||||
return_value=("warn", "扩展未连接"),
|
||||
), patch.object(TwitterChannel, "_check_bird", return_value=None):
|
||||
status, msg = channel.check()
|
||||
assert status == "warn"
|
||||
assert channel.active_backend == "twitter-cli"
|
||||
assert "未认证" in msg
|
||||
|
||||
Reference in New Issue
Block a user