sawz/Agent-Reach
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
154 Commits 2 Branches 7 Tags
db20c5d5d36147ea6e314f84416ffff36dd8a6ca
Commit Graph

31 Commits

Author SHA1 Message Date
AICodeLion db20c5d5d3 feat: add Weibo channel via mcp-server-weibo (#107) 
Adds WeiboChannel integrating qinyuanpei/mcp-server-weibo (34+ stars,
MIT, 10 tools) via mcporter. Addresses #75.

Files changed (3 files, +57 lines):
- agent_reach/channels/weibo.py — new channel (54 lines)
- agent_reach/channels/__init__.py — register WeiboChannel
- tests/test_channel_contracts.py — add weibo URL sample

Co-authored-by: AICodeLion <AICodeLion@users.noreply.github.com>
 2026-03-08 21:54:35 +08:00
Shawn 3a3d38acce fix: YouTube JS runtime check, Douyin health check, cli bare except, config permission race (#104) 
- YouTube: warn when only Node.js is installed but yt-dlp config file
  is missing (previously returned "ok" incorrectly)
- Douyin: use `mcporter list` instead of calling with a hardcoded
  invalid share URL that always fails
- cli: replace bare `except:` with `except Exception:` in
  `_detect_environment` to avoid catching KeyboardInterrupt/SystemExit
- cli: fix unclosed file handle for cloud VM detection
- config: use `os.open()` with 0o600 mode to eliminate permission
  race window when saving credentials
 2026-03-08 21:24:05 +08:00
Pnant 476935be92 feat: add WeChat search via miku_ai (sogou) (#68) 
- wechat channel now supports both search and read
- Search: miku_ai (sogou weixin search) → article list with URLs
- Read: camoufox (stealth Firefox) → full Markdown content
- Doctor shows granular status (search only / read only / both)
- SKILL.md updated with search + read workflow examples

Co-authored-by: Panniantong <panniantong@users.noreply.github.com>
 2026-03-04 18:02:05 +08:00
Pnant 4f4ad99d49 feat: add WeChat Official Account channel (#54) (#67) 
New channel: wechat (微信公众号文章)
- Backend: wechat-article-for-ai (Camoufox stealth browser)
- Can read mp.weixin.qq.com article URLs → clean Markdown
- Bypasses WeChat anti-bot detection via stealth Firefox
- Doctor checks for camoufox package availability
- SKILL.md updated with usage examples

Co-authored-by: Panniantong <panniantong@users.noreply.github.com>
 2026-03-04 17:53:39 +08:00
Pnant a91a74d880 fix: Windows encoding + .cmd subprocess resolution (#64) (#66) 
- Replace text=True with encoding='utf-8', errors='replace' in all
  subprocess.run calls (channels + cli.py) to fix GBK decode errors
  on Chinese Windows systems
- Use shutil.which() resolved paths in channel subprocess calls to
  handle Windows .cmd wrappers (npm global installs)

Co-authored-by: Panniantong <panniantong@users.noreply.github.com>
 2026-03-04 17:30:11 +08:00
Panniantong 0d1a33236e fix: YouTube requires JS runtime — auto-configure Node.js in install 
yt-dlp now requires an external JS runtime (deno or Node.js) for YouTube.
Since agent-reach already installs Node.js, we configure yt-dlp to use it:

1. cli.py: install writes '--js-runtimes node' to ~/.config/yt-dlp/config
2. youtube.py: doctor checks JS runtime availability and config
3. SKILL.md: updated YouTube section with accurate troubleshooting

Bilibili note: 412 errors are IP-based (needs proxy), not cookie-related.

Fixes #47
 2026-02-27 11:45:39 +01:00
Pnant 15a2961dfa Merge pull request #46 from Panniantong/migrate-bird-to-xreach 
feat: 迁移 Twitter 后端 bird → xreach CLI
 2026-02-27 15:21:25 +08:00
Panniantong 62aacf38b5 feat: migrate Twitter backend from bird CLI to xreach CLI 
bird CLI (@steipete/bird) is deprecated and no longer maintained.
xreach CLI (xreach-cli on npm) is our maintained fork with:
- Fixed SearchTimeline (POST + updated query ID)
- Built-in proxy rotation support
- Additional features (DMs, notifications, lists)

Changes across 11 files:
- channels/twitter.py: detect xreach instead of bird/birdx
- cli.py: install/doctor/uninstall all reference xreach-cli
- SKILL.md: updated command examples (bird read → xreach tweet)
- guides/setup-twitter.md: rewritten for xreach
- docs/troubleshooting.md: updated proxy guidance
- README.md + README_en.md: all references updated
- config.py: twitter_bird → twitter_xreach
- core.py, mcp_server.py: comment updates

npm package: https://www.npmjs.com/package/xreach-cli
Source: https://github.com/Panniantong/xfetch
 2026-02-27 08:17:51 +01:00
Panniantong d4e437bf2c fix: add ARM64/Apple Silicon support for xiaohongshu-mcp Docker setup 
- Detect ARM64 architecture and auto-add --platform linux/amd64 to
  docker run hints in doctor/check output
- Add ARM64 FAQ section to setup guide with two options:
  Rosetta emulation (--platform linux/amd64) or build from source
- Closes #44
 2026-02-27 08:06:06 +01:00
Your Name 4b2e6f2ffb fix: split security hardening and setup mcporter config checks 2026-02-27 12:16:00 +08:00
Panniantong 50f2181fcc feat: 新增抖音（Douyin）渠道 - 基于 douyin-mcp-server 
- 新增 agent_reach/channels/douyin.py
- 更新 doctor 检测逻辑
- 更新 install.md/SKILL.md/README 文档
- 通过 mcporter 调用 douyin-mcp-server
- 支持视频解析、下载链接获取
- 无需登录即可使用
 2026-02-26 13:25:57 +01:00
Panniantong a37e9aa190 refactor: strip to installer + doctor + docs, remove read/search wrapper layer 
BREAKING CHANGE: Remove all `agent-reach read` and `agent-reach search-*` commands.

Agent Reach is now an installer, configuration tool, and doctor —
not a wrapper layer. After installation, agents call upstream tools
directly (bird CLI, yt-dlp, mcporter, gh CLI, Jina Reader, etc.).

What's kept:
- agent-reach install: one-shot installer
- agent-reach doctor: channel status overview
- agent-reach configure: cookies, proxy, credentials
- agent-reach setup: interactive wizard
- SKILL.md: complete guide for agents to use upstream tools directly

What's removed:
- agent-reach read URL (and all channel read() methods)
- agent-reach search-* commands (and all channel search() methods)
- ReadResult / SearchResult data classes
- URL routing system (get_channel_for_url)
- All parsing/conversion logic (VTT, Reddit JSON, bird text parser, etc.)
- MCP server read/search tools (kept only get_status)

Net change: -1790 lines. Less code = fewer bugs.
 2026-02-26 08:15:56 +01:00
Pnant 1cbf6a7b9c fix(xiaohongshu): 修复 xsec_token 丢失导致无法读取笔记详情 (#17) 
问题：搜索结果包含 xsecToken 但被丢弃，read() 只从首页 feed
查找 token，几乎不可能匹配到目标笔记。

修复：
1. search() 返回的 URL 携带 xsec_token 查询参数
2. read() 三级 token 查找：URL 参数 → 首页 feed → 搜索反查
3. 全部失败时回退到 Jina Reader

Fixes #15

Co-authored-by: Panniantong <panniantong@users.noreply.github.com>
 2026-02-26 14:36:36 +08:00
Panniantong f70711e75e remove(instagram): 移除 Instagram 渠道 
Instagram 反爬封杀导致所有开源工具（instaloader 等）失效，
无论有无 cookies 都无法正常使用。

- 删除 instagram.py 渠道文件
- 移除 CLI 中 search-instagram、configure instagram-cookies 等命令
- 移除 setup/doctor 中 instaloader 依赖检查
- 更新 README、docs、SKILL.md、pyproject.toml

上游 issue: instaloader#2585, instaloader#2648
Relates to: #13
 2026-02-26 07:20:13 +01:00
Panniantong 12bd727e30 fix: Twitter bird CLI fetch failed — 自动代理注入 + 连通性检测 + Exa fallback 
问题：bird CLI 用 Node.js 原生 fetch()，不走 HTTP_PROXY，
导致需要代理的用户（如 Windows + 本地代理）完全无法使用。

修复：
1. 自动代理注入：检测到 HTTP_PROXY/HTTPS_PROXY 时，通过 undici 的
   EnvHttpProxyAgent 让 Node.js fetch 自动走代理，用户无需任何额外配置
2. doctor 检测：实际跑 bird whoami 测连通性，失败时给出具体原因
3. search fallback：bird 搜索失败时自动 fallback 到 Exa
4. install：自动安装 undici（npm install -g undici）
5. 新增 troubleshooting.md：常见问题解决方案

Fixes #9
 2026-02-25 21:35:21 +01:00
Panniantong 31eb6db537 fix: use encoding='utf-8' with errors='replace' for bird subprocess calls 
Fixes non-UTF-8 character handling in Twitter content.
Replaces text=True with explicit encoding parameters.

Co-authored-by: dev-starlight <dev-starlight@users.noreply.github.com>
Closes #3
 2026-02-25 13:44:47 +01:00
Panniantong 5906d4ee3f fix: LinkedIn MCP 参数适配 — 用 company_name/linkedin_username 替代 url 
linkedin-scraper-mcp 的 API 参数是 company_name 和 linkedin_username，
不是 url。从 URL 中提取 slug 传给 MCP。
timeout 增加到 60 秒（浏览器自动化需要时间）。
search 去掉不支持的 limit 参数。
 2026-02-25 13:16:00 +01:00
Panniantong e8fe33c80c feat: Instagram 支持 Cookie-Editor 导入 + 超时 fallback 优化 
- agent-reach configure instagram-cookies 支持 Cookie-Editor Header String
- Instagram cookie 保存到 ~/.agent-reach/instagram-cookies.txt
- instaloader 429 限速时 15 秒自动超时 fallback 到 Jina Reader
- LinkedIn/Boss直聘 保持使用各自 MCP 的原生登录流程（不侵入）
- doctor 提示信息更新
 2026-02-25 11:13:52 +01:00
Panniantong aafd6839e9 fix: Instagram 429 限速时 15 秒超时后自动 fallback 到 Jina 
instaloader 遇到 429 会自动等 30 分钟重试导致卡住。
改为在 executor 里跑，15 秒超时后自动切 Jina Reader fallback。
 2026-02-25 10:32:51 +01:00
Panniantong e3804108fe feat: 新增 Instagram、LinkedIn、Boss直聘 三个渠道 
新增渠道：
- Instagram: 基于 instaloader (⭐9.8K)，读取帖子/Profile，Cookie 登录
- LinkedIn: 基于 linkedin-scraper-mcp (⭐900+) MCP 服务，Jina Reader fallback
- Boss直聘: 基于 mcp-bosszp MCP 服务，Jina Reader fallback

代码改动：
- 新建 channels/instagram.py, linkedin.py, bosszhipin.py
- 注册到 channels/__init__.py
- cli.py 添加 search-instagram/linkedin/bosszhipin 子命令
- cli.py 安装逻辑添加 instaloader 自动安装
- core.py 添加 search_instagram/linkedin/bosszhipin 方法
- README.md + docs/README_en.md 更新平台表格和选型表格
- docs/install.md 添加三个新渠道的配置说明和 Quick Reference
 2026-02-25 10:25:30 +01:00
Panniantong a0ecd438ee fix: 完善所有渠道安装指引，修复文档错误 
- 小红书: 添加 Docker 安装命令 (xpzouying/xiaohongshu-mcp)，替换模糊的 'install xiaohongshu-mcp'
- Exa: 修正文档错误（实际免费无需 API Key，之前写要 Key）
- CLI: install 命令输出添加小红书 Docker 安装指引
- channel: xiaohongshu.py 所有错误信息添加完整安装步骤
- install.md: 添加小红书配置段落
- README: 修正小红书 GitHub 链接，Exa 描述改为免费无需 Key

Fixes user-reported issue: 安装后不知道怎么装小红书 MCP
 2026-02-25 05:10:56 +01:00
Panniantong 93ad9c5722 fix: birdx → bird CLI (npm @steipete/bird) 
birdx 从来不是 PyPI 包，pip install birdx 必然失败。
实际工具是 npm 包 @steipete/bird，一个 Twitter GraphQL CLI。

变更：
- 安装器改用 npm install -g @steipete/bird
- twitter.py 直接调 bird，通过环境变量传 AUTH_TOKEN/CT0
- 兼容已有的 birdx wrapper（shutil.which 回退）
- 更新所有文档引用
- 重写 setup-twitter.md 指南
 2026-02-25 04:02:42 +01:00
Panniantong 7d0da09222 feat: 新增 YouTube/Bilibili/XHS 搜索 CLI 命令 + 修复 B站搜索 
- cli.py: 新增 search-youtube / search-bilibili / search-xhs 子命令
- core.py: 新增 search_youtube / search_bilibili / search_xhs 方法
- bilibili.py: 搜索策略改为 yt-dlp → Exa fallback（服务器 IP 被 B站 412）
  - 本地机器走 yt-dlp bilisearch（有 title/uploader）
  - 服务器自动 fallback 到 Exa site:bilibili.com
  - 修复 mcporter 输出解析（Title:/URL: 格式）

测试结果: 15/15 全通（8 read + 7 search）
 2026-02-24 13:39:21 +01:00
Panniantong 88eaebe263 refactor: 统一所有渠道后端，对齐 research 技能 
GitHub: REST API → gh CLI（官方工具，认证后完整能力）
Bilibili: 自写 API → yt-dlp（和 YouTube 统一后端，支持搜索 bilisearch）
YouTube: 新增搜索功能（ytsearch via yt-dlp）

README 中英文同步更新：
- 平台表格：小红书/Exa/GitHub/YouTube/B站 描述全部更新
- 选型表格：新增 gh CLI、xiaohongshu-mcp，更新 yt-dlp/Exa 描述
- 按需解锁：去掉 Exa Key 注册步骤（已自动配置）
- 配置难度说明：新增「自动配置」「mcporter」级别
 2026-02-24 13:25:57 +01:00
Panniantong bb9ce6f7f1 refactor: XHS + Exa 全面切换到 mcporter MCP 后端 
- xiaohongshu.py: 删除网页爬取代码，100% 走 mcporter + xiaohongshu-mcp
- exa_search.py: 删除直接 API 调用，100% 走 mcporter + exa MCP（免 Key）
- cli.py install: 新增 _install_mcporter()，自动装 mcporter + 配置 exa/xhs
- cli.py configure: 移除 exa-key 和 xhs-cookie（不再需要）
- doctor.py: 更新 tier 1 描述

净减 166 行代码，架构更简单：
之前: 每个渠道两套后端代码（mcporter + fallback）
现在: 每个渠道一套后端（mcporter only）
 2026-02-24 13:20:23 +01:00
Panniantong 4e0c008d42 feat: 小红书 + Exa 搜索支持 mcporter MCP 后端 
小红书：
- 优先通过 mcporter + xiaohongshu MCP server（内部 API，不受反爬限制）
- Fallback 到 Cookie 网页爬取（原方案，服务器端可能被拦截）
- 新增 search() 方法支持搜索小红书内容
- doctor 检测 MCP 连接状态和登录状态

Exa 搜索：
- 优先通过 mcporter + Exa MCP server（OAuth 免 API Key）
- Fallback 到直接 API 调用（需 EXA_API_KEY）
- 解析 mcporter 输出的 Title/URL/Text 格式

两个渠道都从「必须配置才能用」变成「有 mcporter 就自动能用」
9/9 渠道全部可用
 2026-02-24 13:04:55 +01:00
Panniantong 3efc8d1d55 fix: Reddit 子版块列表页支持 + --version 参数 
- reddit.py: 新增 _parse_listing() 处理 /r/sub/hot|new|top 等列表页
  之前只能读单个帖子，列表页直接报 ValueError
- cli.py: 添加 --version flag (标准 CLI 惯例)
 2026-02-24 12:49:58 +01:00
Panniantong 30f11bb6e4 fix: remove dead code (unreachable return) in twitter.py check() 2026-02-24 12:39:00 +01:00
Panniantong 2d1b8658b6 fix: handle curly quotes in Twitter Jina fallback detection 
Jina Reader returns smart/curly quotes (U+2019) instead of ASCII
apostrophes. Use apostrophe-free substrings for detection.
 2026-02-24 12:38:25 +01:00
Panniantong a5b7b93b1f fix: improve error handling for Twitter Jina fallback and invalid URL inputs 
- Twitter _read_jina now detects unusable X.com responses (JS-required pages)
  and shows a friendly error instead of garbage HTML
- CLI read command now shows user-friendly messages for invalid URLs
  and connection errors instead of raw HTTP exception traces
 2026-02-24 12:36:57 +01:00
Panniantong 5c62a21f32 rename: Agent Eyes → Agent Reach 
全局重命名：
- 包名: agent_eyes → agent_reach
- CLI: agent-eyes → agent-reach
- 类名: AgentEyes → AgentReach
- 显示名: Agent Eyes → Agent Reach
- GitHub: Panniantong/agent-eyes → Panniantong/Agent-Reach

所有 36 个测试通过，CLI/doctor/read/search 全部正常。
 2026-02-24 10:25:46 +01:00