fix(xueqiu): fix 400 errors by using browser cookies and correct headers

The Xueqiu stock API requires a login session token (xq_a_token) that
is generated by Xueqiu's frontend JavaScript and cannot be obtained by
simply visiting the homepage. This caused persistent HTTP 400 (error
code 400016) for all users.

Changes:
- _ensure_cookies(): add three-level priority — config file (saved by
  --from-browser) → live Chrome cookies via browser_cookie3 → homepage
  fallback. The homepage-only approach only ever got acw_tc (anti-DDoS
  token), never xq_a_token.
- _get_json(): switch User-Agent from "agent-reach/1.0" to a real Chrome
  UA, and add Referer: https://xueqiu.com/ to all API requests.
- get_hot_posts(): replace the defunct /statuses/hot/listV3.json endpoint
  (returns empty body) with the v4 public timeline endpoint; correctly
  parse item.data as a JSON string to extract author, text, and likes.
- cookie_extract.py: add Xueqiu to PLATFORM_SPECS and configure_from_browser
  so that `agent-reach configure --from-browser chrome` now also saves
  Xueqiu cookies (only when xq_a_token is present).
- check(): improve error message to direct users to --from-browser instead
  of suggesting a proxy.
- Fix urllib.parse.quote usage (was using urllib.request.quote).
- Update tier and backends description to reflect cookie requirement.
- Add 2 new tests: cookie loading from config, Referer/UA header verification.
- Update docs: README, install guide, troubleshooting, SKILL.md, CHANGELOG.
This commit is contained in:
fernando_jacob
2026-03-27 11:55:14 +08:00
parent ca2e85520b
commit d793afaba6
9 changed files with 293 additions and 70 deletions
+17
View File
@@ -6,6 +6,23 @@ All notable changes to this project will be documented in this file.
---
## [1.3.1] - 2026-03-27
### 🐛 Bug Fixes / 修复
#### 📈 Xueqiu (雪球) — 全面修复
- **修复 400 错误根本原因:** `_ensure_cookies()` 仅访问首页只能获取 `acw_tc`(防 DDoS token),`xq_a_token` 由雪球前端 JS 动态生成,无法通过纯 HTTP 请求获取。新增三级 cookie 加载策略:① 读取 config 文件(`--from-browser` 保存的)→ ② 自动从本地 Chrome 浏览器提取(需安装 browser-cookie3)→ ③ homepage fallback
- **修复 User-Agent** `"agent-reach/1.0"` 被雪球反爬系统识别拒绝,改为真实 Chrome UA
- **修复缺失 `Referer` 头:** 所有 API 请求加上 `Referer: https://xueqiu.com/`
- **修复 `get_hot_posts()` 端点:** 原端点 `/statuses/hot/listV3.json` 已废弃(返回空 body),改为 `/v4/statuses/public_timeline_by_category.json`,正确解析 `item.data` JSON 字符串获取 author/likes/text
- **修复 `urllib.request.quote``urllib.parse.quote`** 明确使用正确模块
- **修复 `configure --from-browser` 不提取雪球 Cookie** `PLATFORM_SPECS` 加入 Xueqiu,检测 `xq_a_token` 存在才保存
- **修正文档误导:** README/SKILL.md 中"无需配置"/"public API, no login required" → 准确描述需要 browser cookie
- **改善错误信息:** `check()` 失败时提示 `configure --from-browser chrome` 而非"可能需要代理"
---
## [1.3.0] - 2026-03-12
### 🆕 New Channels / 新增渠道
+1 -1
View File
@@ -80,7 +80,7 @@ AI Agent 已经能帮你写代码、改文档、管项目——但你让它去
| 💬 **微信公众号** | 搜索 + 阅读公众号文章(全文 Markdown) | — | 无需配置 |
| 📰 **微博** | 热搜、搜索内容/用户/话题、用户动态、评论 | — | 无需配置 |
| 💻 **V2EX** | 热门帖子、节点帖子、帖子详情+回复、用户信息 | — | 无需配置 |
| 📈 **雪球** | 股票行情、搜索股票、热门帖子、热门股票排行 | — | 无需配置 |
| 📈 **雪球** | 股票行情、搜索股票、热门帖子、热门股票排行 | — | 告诉 Agent「帮我配雪球」 |
| 🎙️ **小宇宙播客** | — | 播客音频转文字(Whisper 转录,免费 Key) | 告诉 Agent「帮我配小宇宙播客」 |
> **不知道怎么配?不用查文档。** 直接告诉 Agent「帮我配 XXX」,它知道需要什么、会一步一步引导你。
+134 -24
View File
@@ -4,12 +4,18 @@
import http.cookiejar
import json
import re
import urllib.parse
import urllib.request
from typing import Any
from .base import Channel
_UA = "agent-reach/1.0"
_UA = (
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/120.0.0.0 Safari/537.36"
)
_REFERER = "https://xueqiu.com/"
_TIMEOUT = 10
_XUEQIU_HOME = "https://xueqiu.com"
@@ -22,11 +28,90 @@ _opener = urllib.request.build_opener(
_cookies_initialized = False
def _inject_cookie_string(cookie_str: str) -> None:
"""Parse a 'name=value; name2=value2' string and inject into the cookie jar."""
for pair in cookie_str.split(";"):
pair = pair.strip()
if "=" not in pair:
continue
name, _, value = pair.partition("=")
cookie = http.cookiejar.Cookie(
version=0,
name=name.strip(),
value=value.strip(),
port=None,
port_specified=False,
domain=".xueqiu.com",
domain_specified=True,
domain_initial_dot=True,
path="/",
path_specified=True,
secure=True,
expires=None,
discard=True,
comment=None,
comment_url=None,
rest={},
)
_cookie_jar.set_cookie(cookie)
def _load_cookies_from_config() -> bool:
"""Try to load Xueqiu cookies from agent-reach config file (xueqiu_cookie key)."""
try:
from ..config import Config
cfg = Config()
cookie_str = cfg.get("xueqiu_cookie")
if not cookie_str:
return False
_inject_cookie_string(cookie_str)
return True
except Exception:
return False
def _load_cookies_from_browser() -> bool:
"""Try to silently load Xueqiu cookies from the local Chrome browser.
Only succeeds when browser_cookie3 is installed AND the user is logged in
(xq_a_token present). Failures are silently ignored so that agents without
a local browser keep working.
"""
try:
import browser_cookie3
cookies = list(browser_cookie3.chrome(domain_name=".xueqiu.com"))
if not any(c.name == "xq_a_token" for c in cookies):
return False
for c in cookies:
_cookie_jar.set_cookie(c)
return True
except Exception:
return False
def _ensure_cookies() -> None:
"""Visit xueqiu.com homepage once to obtain session cookies."""
"""Populate session cookies using the best available source.
Priority order:
1. Saved cookie string in ~/.agent-reach/config.yaml (set by configure --from-browser)
2. Live Chrome browser cookies via browser_cookie3 (if installed + logged in)
3. Homepage visit fallback (only yields anti-DDoS acw_tc,
not enough for stock APIs)
"""
global _cookies_initialized
if _cookies_initialized:
return
if _load_cookies_from_config():
_cookies_initialized = True
return
if _load_cookies_from_browser():
_cookies_initialized = True
return
# Fallback: visit homepage to pick up acw_tc anti-DDoS cookie.
# This is not sufficient for authenticated APIs but avoids hard failures
# on public endpoints that only need the session cookie.
req = urllib.request.Request(_XUEQIU_HOME, headers={"User-Agent": _UA})
_opener.open(req, timeout=_TIMEOUT)
_cookies_initialized = True
@@ -35,7 +120,9 @@ def _ensure_cookies() -> None:
def _get_json(url: str) -> Any:
"""Fetch *url* with Xueqiu session cookies and return parsed JSON."""
_ensure_cookies()
req = urllib.request.Request(url, headers={"User-Agent": _UA})
req = urllib.request.Request(
url, headers={"User-Agent": _UA, "Referer": _REFERER}
)
with _opener.open(req, timeout=_TIMEOUT) as resp:
return json.loads(resp.read().decode("utf-8"))
@@ -51,17 +138,15 @@ def _strip_html(text: str) -> str:
class XueqiuChannel(Channel):
name = "xueqiu"
description = "雪球股票行情与社区动态"
backends = ["Xueqiu API (public)"]
tier = 0
backends = ["Xueqiu API (需要登录 Cookie)"]
tier = 1
# ------------------------------------------------------------------ #
# URL routing
# ------------------------------------------------------------------ #
def can_handle(self, url: str) -> bool:
from urllib.parse import urlparse
d = urlparse(url).netloc.lower()
d = urllib.parse.urlparse(url).netloc.lower()
return "xueqiu.com" in d
# ------------------------------------------------------------------ #
@@ -70,13 +155,18 @@ class XueqiuChannel(Channel):
def check(self, config=None):
try:
data = _get_json("https://stock.xueqiu.com/v5/stock/batch/quote.json?symbol=SH000001")
data = _get_json(
"https://stock.xueqiu.com/v5/stock/batch/quote.json?symbol=SH000001"
)
items = (data.get("data") or {}).get("items") or []
if items:
return "ok", "公开 API 可用(行情、搜索、热帖、热股)"
return "warn", "API 响应异常(返回数据为空)"
except Exception as e:
return "warn", f"Xueqiu API 连接失败(可能需要代理):{e}"
return "warn", (
f"Xueqiu API 连接失败:{e}"
"请先登录雪球后运行:agent-reach configure --from-browser chrome"
)
# ------------------------------------------------------------------ #
# Data-fetching methods
@@ -92,7 +182,9 @@ class XueqiuChannel(Channel):
symbol, name, current, percent, chg, high, low, open, last_close,
volume, amount, market_capital, turnover_rate, pe_ttm, timestamp
"""
data = _get_json(f"https://stock.xueqiu.com/v5/stock/batch/quote.json?symbol={symbol}")
data = _get_json(
f"https://stock.xueqiu.com/v5/stock/batch/quote.json?symbol={symbol}"
)
items = (data.get("data") or {}).get("items") or []
q = (items[0].get("quote") or {}) if items else {}
return {
@@ -124,7 +216,8 @@ class XueqiuChannel(Channel):
symbol, name, exchange
"""
data = _get_json(
f"https://xueqiu.com/stock/search.json?code={urllib.request.quote(query)}&size={limit}"
f"https://xueqiu.com/stock/search.json"
f"?code={urllib.parse.quote(query)}&size={limit}"
)
stocks = data.get("stocks") or []
results = []
@@ -141,29 +234,45 @@ class XueqiuChannel(Channel):
def get_hot_posts(self, limit: int = 20) -> list:
"""获取雪球热门帖子。
Uses the v4 public timeline endpoint which returns posts in a `list`
array. Each item carries a JSON-encoded `data` field containing the
actual post payload (title, description, user, like_count, target).
Args:
limit: 最多返回条数(上限 50)
Returns a list of dicts with keys:
id, title, text, author, likes, url
"""
data = _get_json("https://xueqiu.com/statuses/hot/listV3.json?source=hot&page=1")
items = (data.get("data") or {}).get("items") or []
data = _get_json(
"https://xueqiu.com/v4/statuses/public_timeline_by_category.json"
"?since_id=-1&max_id=-1&count=20&category=-1"
)
items = data.get("list") or []
results = []
for item in items[:limit]:
original = item.get("original_status") or item
text = _strip_html(original.get("text") or original.get("description") or "")
user = original.get("user") or {}
# Each item.data is a JSON string containing the real post payload
try:
post = (
json.loads(item["data"])
if isinstance(item.get("data"), str)
else {}
)
except (json.JSONDecodeError, KeyError):
post = {}
user = post.get("user") or {}
text = _strip_html(
post.get("text") or post.get("description") or ""
)
target = post.get("target", "")
results.append(
{
"id": original.get("id", 0),
"title": original.get("title") or "",
"id": post.get("id", 0),
"title": post.get("title") or "",
"text": text[:200],
"author": user.get("screen_name", ""),
"likes": original.get("like_count", 0),
"url": f"https://xueqiu.com{original['target']}"
if original.get("target")
else "",
"likes": post.get("like_count", 0),
"url": f"https://xueqiu.com{target}" if target else "",
}
)
return results
@@ -179,7 +288,8 @@ class XueqiuChannel(Channel):
symbol, name, current, percent, rank
"""
data = _get_json(
f"https://stock.xueqiu.com/v5/stock/hot_stock/list.json?size={limit}&type={stock_type}"
f"https://stock.xueqiu.com/v5/stock/hot_stock/list.json"
f"?size={limit}&type={stock_type}"
)
items = (data.get("data") or {}).get("items") or []
results = []
+18
View File
@@ -32,6 +32,12 @@ PLATFORM_SPECS = [
"cookies": ["SESSDATA", "bili_jct"],
"config_key": "bilibili",
},
{
"name": "Xueqiu",
"domains": [".xueqiu.com", "xueqiu.com"],
"cookies": None, # grab all — xq_a_token + session cookies required
"config_key": "xueqiu",
},
]
@@ -217,4 +223,16 @@ def configure_from_browser(browser: str, config) -> List[Tuple[str, bool, str]]:
results_list.append(("Bilibili", False,
f"No SESSDATA found. Make sure you're logged into bilibili.com in {browser}."))
if "xueqiu" in extracted:
cookie_str = extracted["xueqiu"].get("cookie_string", "")
# Only save if xq_a_token is present — anonymous cookies are useless
if cookie_str and "xq_a_token" in cookie_str:
config.set("xueqiu_cookie", cookie_str)
n_cookies = len(cookie_str.split(";"))
results_list.append(("Xueqiu", True, f"{n_cookies} cookies (含 xq_a_token)"))
elif cookie_str:
results_list.append(("Xueqiu", False,
f"找到 {len(cookie_str.split(';'))} 个 Cookie 但缺少 xq_a_token"
f"请先在 {browser} 中登录 xueqiu.com"))
return results_list
+3 -3
View File
@@ -255,7 +255,7 @@ print(result[0]["error"]) # 提示使用站内搜索或 Exa channel
> No auth required. Results are public JSON. V2EX 节点名见 https://www.v2ex.com/planes
## 雪球 / Xueqiu (public API)
## 雪球 / Xueqiu (需要登录 Cookie)
```python
from agent_reach.channels.xueqiu import XueqiuChannel
@@ -274,7 +274,7 @@ stocks = ch.search_stock("茅台", limit=5)
for s in stocks:
print(f"{s['name']} ({s['symbol']}) - {s['exchange']}")
# 热门帖子
# 热门帖子v4 timeline,含作者、点赞数、正文)
# 返回字段:id, title, text(前200字), author, likes, url
posts = ch.get_hot_posts(limit=10)
for p in posts:
@@ -287,7 +287,7 @@ for s in hot:
print(f"#{s['rank']} {s['name']} ({s['symbol']}): {s['current']} ({s['percent']}%)")
```
> 无需登录。自动获取会话 Cookie,所有公开 API 均可直接使用
> 需要登录 Cookie。先在 Chrome 登录 xueqiu.com,再运行 `agent-reach configure --from-browser chrome`
## RSS (feedparser)
+1 -1
View File
@@ -69,7 +69,7 @@ Update Agent Reach: https://raw.githubusercontent.com/Panniantong/agent-reach/ma
| 💬 **WeChat Articles** | Search + Read | Zero config | Search + read WeChat Official Account articles (full Markdown) ([wechat-article-for-ai](https://github.com/Panniantong/wechat-article-for-ai) + [miku_ai](https://github.com/GobinFan/Miku_Spider)) |
| 📰 **Weibo** | Trending · Search · Feeds · Comments | Zero config | Hot search, content/user/topic search, feeds, comments ([mcp-server-weibo](https://github.com/Panniantong/mcp-server-weibo)) |
| 💻 **V2EX** | Hot topics · Node topics · Topic detail + replies · User profile | Zero config | Public JSON API, no auth required. Great for tech community content |
| 📈 **Xueqiu (雪球)** | Stock quotes · Search · Hot posts · Hot stocks | Zero config | Public API with auto session cookies, no login required |
| 📈 **Xueqiu (雪球)** | Stock quotes · Search · Hot posts · Hot stocks | Browser cookie | Tell your Agent "help me set up Xueqiu" |
| 🎙️ **Xiaoyuzhou Podcast** | Transcription | Free API key | Podcast audio → full text transcript via Groq Whisper (free) |
| 🔍 **Web Search** | Search | Auto-configured | Auto-configured during install, free, no API key ([Exa](https://exa.ai) via [mcporter](https://github.com/nicepkg/mcporter)) |
| 📦 **GitHub** | Read · Search | Zero config | [gh CLI](https://cli.github.com) powered. Public repos work immediately. `gh auth login` unlocks Fork, Issue, PR |
+11 -2
View File
@@ -112,13 +112,13 @@ Some channels need credentials only the user can provide. Based on the doctor ou
> 🍪 **Cookie 导入(所有需要登录的平台通用):**
>
> 所有需要 Cookie 的平台(Twitter、小红书等),**优先使用 Cookie-Editor 导入**,这是最简单最可靠的方式:
> 所有需要 Cookie 的平台(Twitter、小红书、雪球等),**优先使用 Cookie-Editor 导入**,这是最简单最可靠的方式:
> 1. 用户在自己的浏览器上登录对应平台
> 2. 安装 [Cookie-Editor](https://chromewebstore.google.com/detail/cookie-editor/hlkenndednhfkekhgcdicdfddnkalmdm) Chrome 插件
> 3. 点击插件 → Export → Header String
> 4. 把导出的字符串发给 Agent
>
> **本地电脑用户**也可以用 `agent-reach configure --from-browser chrome` 一键自动提取(支持 Twitter + 小红书)。
> **本地电脑用户**也可以用 `agent-reach configure --from-browser chrome` 一键自动提取(支持 Twitter + 小红书 + 雪球)。
**Twitter search & posting:**
> "To unlock Twitter search, I need your Twitter cookies. Install the Cookie-Editor Chrome extension, go to x.com/twitter.com, click the extension → Export → Header String, and paste it to me."
@@ -185,6 +185,15 @@ mcporter config add weibo --command 'mcp-server-weibo'
> 无需登录、无需 Cookie、无需代理。海外服务器也可以直接访问。
**雪球 / Xueqiu (股票行情 + 热门帖子):**
> "雪球需要登录后的 Cookie。请先在 Chrome 里登录 xueqiu.com,然后运行:"
```bash
agent-reach configure --from-browser chrome
```
> Cookie 会随其他平台一起自动提取。
**小宇宙播客 / Xiaoyuzhou Podcast (Groq Whisper):**
> "小宇宙播客转文字已默认安装,只需要一个免费的 Groq API Key。"
+16
View File
@@ -1,5 +1,21 @@
# 常见问题排查
## 雪球 / Xueqiu: API 返回 400
**症状:** `agent-reach doctor` 显示雪球 ⚠️,报 `HTTP Error 400`
**原因:** 雪球 API 需要登录 Cookie,无法通过匿名访问获取。
**解决方案:** 在 Chrome 里登录 xueqiu.com,然后运行:
```bash
agent-reach configure --from-browser chrome
```
再次运行 `agent-reach doctor` 确认恢复 ✅。Cookie 过期后重新运行即可。
---
## Twitter/X: bird CLI 连接失败
**症状:** `bird search` 或其他命令返回错误
+92 -39
View File
@@ -473,31 +473,23 @@ class TestXueqiuChannel:
monkeypatch.setattr(xueqiu_mod, "_cookies_initialized", True)
fake_data = {
"data": {
"items": [
{
"original_status": {
"id": 111,
"title": "市场分析",
"text": "<p>今天大盘走势&amp;分析</p>",
"user": {"screen_name": "投资者A"},
"like_count": 42,
"target": "/1234567890/111",
}
},
{
"original_status": {
"id": 222,
"title": "",
"text": "短评",
"user": {"screen_name": "投资者B"},
"like_count": 10,
"target": "/9876543210/222",
}
},
]
# v4 timeline: each item has a JSON-encoded `data` field
def make_item(id_, title, text, author, likes, target):
post = {
"id": id_,
"title": title,
"text": text,
"user": {"screen_name": author},
"like_count": likes,
"target": target,
}
return {"data": json.dumps(post), "original_status": None}
fake_data = {
"list": [
make_item(111, "市场分析", "<p>今天大盘走势&amp;分析</p>", "投资者A", 42, "/1234567890/111"),
make_item(222, "", "短评", "投资者B", 10, "/9876543210/222"),
]
}
class FakeResponse:
@@ -526,21 +518,20 @@ class TestXueqiuChannel:
monkeypatch.setattr(xueqiu_mod, "_cookies_initialized", True)
fake_data = {
"data": {
"items": [
{
"original_status": {
"id": i,
"title": f"Post {i}",
"text": f"Content {i}",
"user": {"screen_name": f"User {i}"},
"like_count": i,
"target": f"/user/{i}",
}
}
for i in range(10)
]
}
"list": [
{
"data": json.dumps({
"id": i,
"title": f"Post {i}",
"text": f"Content {i}",
"user": {"screen_name": f"User {i}"},
"like_count": i,
"target": f"/user/{i}",
}),
"original_status": None,
}
for i in range(10)
]
}
class FakeResponse:
@@ -594,6 +585,68 @@ class TestXueqiuChannel:
assert stocks[1]["percent"] == -0.8
assert stocks[2]["rank"] == 3
# ------------------------------------------------------------------ #
# Cookie loading
# ------------------------------------------------------------------ #
def test_ensure_cookies_loads_from_config(self, monkeypatch, tmp_path):
"""_ensure_cookies() should inject cookies from the config file."""
import agent_reach.channels.xueqiu as xueqiu_mod
monkeypatch.setattr(xueqiu_mod, "_cookies_initialized", False)
# Provide a fake Config that returns a cookie string with xq_a_token
class FakeConfig:
def get(self, key, default=None):
if key == "xueqiu_cookie":
return "xq_a_token=TESTTOKEN; xq_is_login=1"
return default
import agent_reach.channels.xueqiu as xq_mod
monkeypatch.setattr(
xq_mod,
"_load_cookies_from_config",
lambda: (xq_mod._inject_cookie_string("xq_a_token=TESTTOKEN; xq_is_login=1") or True),
)
monkeypatch.setattr(xq_mod, "_load_cookies_from_browser", lambda: False)
# Patch opener so no real HTTP call is made
class FakeResp:
def __enter__(self): return self
def __exit__(self, *_): pass
def read(self): return b'{"data":{"items":[]}}'
monkeypatch.setattr(xq_mod._opener, "open", lambda req, timeout=None: FakeResp())
xq_mod._ensure_cookies()
assert xq_mod._cookies_initialized is True
cookie_names = {c.name for c in xq_mod._cookie_jar}
assert "xq_a_token" in cookie_names
def test_get_json_sends_referer_and_browser_ua(self, monkeypatch):
"""_get_json() must send Referer and a browser-like User-Agent."""
import agent_reach.channels.xueqiu as xueqiu_mod
monkeypatch.setattr(xueqiu_mod, "_cookies_initialized", True)
captured = {}
class FakeResp:
def __enter__(self): return self
def __exit__(self, *_): pass
def read(self): return b'{"data":{"items":[]}}'
def fake_open(req, timeout=None):
captured["ua"] = req.get_header("User-agent")
captured["referer"] = req.get_header("Referer")
return FakeResp()
monkeypatch.setattr(xueqiu_mod._opener, "open", fake_open)
xueqiu_mod._get_json("https://stock.xueqiu.com/v5/stock/batch/quote.json?symbol=SH000001")
assert captured["referer"] == "https://xueqiu.com/"
assert "Mozilla" in captured["ua"]
assert "agent-reach" not in captured["ua"]
class TestXiaoHongShuChannel:
def test_reports_ok_when_server_health_is_ok(self, monkeypatch):