fix(bluesky): make Bluesky opt-in with app password auth
searchPosts endpoint now returns 403 for unauthenticated requests. Add session auth via createSession, gate on BSKY_HANDLE + BSKY_APP_PASSWORD env vars. When unconfigured, Bluesky is completely invisible (no error). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -24,6 +24,8 @@ metadata:
|
||||
- APIFY_API_TOKEN
|
||||
- AUTH_TOKEN
|
||||
- CT0
|
||||
- BSKY_HANDLE
|
||||
- BSKY_APP_PASSWORD
|
||||
bins:
|
||||
- node
|
||||
- python3
|
||||
@@ -46,7 +48,7 @@ metadata:
|
||||
|
||||
# last30days v2.9.5: Research Any Topic from the Last 30 Days
|
||||
|
||||
> **Permissions overview:** Reads public web/platform data and optionally saves research briefings to `~/Documents/Last30Days/`. X/Twitter search uses optional user-provided tokens (AUTH_TOKEN/CT0 env vars) — no browser session access. All credential usage and data writes are documented in the [Security & Permissions](#security--permissions) section.
|
||||
> **Permissions overview:** Reads public web/platform data and optionally saves research briefings to `~/Documents/Last30Days/`. X/Twitter search uses optional user-provided tokens (AUTH_TOKEN/CT0 env vars). Bluesky search uses optional app password (BSKY_HANDLE/BSKY_APP_PASSWORD env vars - create at bsky.app/settings/app-passwords). All credential usage and data writes are documented in the [Security & Permissions](#security--permissions) section.
|
||||
|
||||
Research ANY topic across Reddit, X, Bluesky, YouTube, TikTok, Hacker News, Polymarket, and the web. Surface what people are actually discussing, recommending, betting on, and debating right now.
|
||||
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
---
|
||||
title: "fix: Make Bluesky opt-in with auth (searchPosts now requires authentication)"
|
||||
type: fix
|
||||
status: completed
|
||||
date: 2026-03-09
|
||||
---
|
||||
|
||||
# fix: Make Bluesky opt-in with auth (searchPosts now requires authentication)
|
||||
|
||||
## Problem
|
||||
|
||||
Bluesky's `app.bsky.feed.searchPosts` endpoint now returns **403 Forbidden** for unauthenticated requests. This broke today (2026-03-09), likely related to the CEO transition. The `searchActors` endpoint still works without auth, but `searchPosts` does not.
|
||||
|
||||
We built Bluesky as an always-on source (like HN and Polymarket) because the AT Protocol public API was documented as free. That's no longer true for post search.
|
||||
|
||||
**Current behavior:** Bluesky silently returns 0 results (403 caught, empty list returned). No stats line appears, no error shown. Users don't know it exists.
|
||||
|
||||
**Desired behavior:** Bluesky is opt-in via env vars. When not configured, completely invisible (no error, no stats line, no mention). When configured with auth, it works as before.
|
||||
|
||||
## Approach
|
||||
|
||||
Follow the same pattern as X/Twitter auth: env vars for credentials, availability check gates dispatch.
|
||||
|
||||
AT Protocol auth flow:
|
||||
1. User creates an App Password at `bsky.app/settings/app-passwords`
|
||||
2. User sets `BSKY_HANDLE` and `BSKY_APP_PASSWORD` env vars
|
||||
3. `bluesky.py` calls `com.atproto.server.createSession` to get a bearer token
|
||||
4. Subsequent `searchPosts` calls include `Authorization: Bearer {token}`
|
||||
|
||||
## Files to Change
|
||||
|
||||
| File | Change | Lines (est.) |
|
||||
|------|--------|-------------|
|
||||
| `scripts/lib/bluesky.py` | Add `_create_session()` auth, pass bearer token to search | ~25 |
|
||||
| `scripts/lib/env.py` | Update `is_bluesky_available()` to check env vars; add env var loading | ~10 |
|
||||
| `scripts/last30days.py` | Gate `do_bluesky` on `is_bluesky_available(config)`; update diag dicts | ~8 |
|
||||
| `tests/test_bluesky.py` | Add auth session tests, update existing tests | ~15 |
|
||||
| `SKILL.md` | Document `BSKY_HANDLE` / `BSKY_APP_PASSWORD` env vars in setup section | ~5 |
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
### 1. `scripts/lib/env.py`
|
||||
|
||||
- Add `BSKY_HANDLE` and `BSKY_APP_PASSWORD` to the env var loading in `get_config()`
|
||||
- Update `is_bluesky_available()`:
|
||||
```python
|
||||
def is_bluesky_available(config: Dict[str, Any]) -> bool:
|
||||
return bool(config.get('BSKY_HANDLE') and config.get('BSKY_APP_PASSWORD'))
|
||||
```
|
||||
|
||||
### 2. `scripts/lib/bluesky.py`
|
||||
|
||||
- Add `_create_session(handle, app_password)` that POSTs to `https://bsky.social/xrpc/com.atproto.server.createSession`
|
||||
- Cache the access token for the lifetime of the search (module-level or passed through)
|
||||
- Update `search_bluesky()` to accept config dict, extract creds, create session, add `Authorization: Bearer {token}` header
|
||||
- On auth failure, return `{"posts": [], "error": "Bluesky auth failed"}` (don't raise)
|
||||
|
||||
### 3. `scripts/last30days.py`
|
||||
|
||||
- Change `do_bluesky` default from `True` to checking `is_bluesky_available(config)`:
|
||||
```python
|
||||
has_bluesky = env.is_bluesky_available(config)
|
||||
```
|
||||
- Gate the `bluesky_future` submit on `has_bluesky` (already gated on `do_bluesky`, just wire it)
|
||||
- Update both diagnostic dicts: `"bluesky": True` -> `"bluesky": has_bluesky`
|
||||
- Pass `config` to `_search_bluesky()` so it can extract auth creds
|
||||
|
||||
### 4. `tests/test_bluesky.py`
|
||||
|
||||
- Test `_create_session` with mocked HTTP response
|
||||
- Test `search_bluesky` with auth header injection
|
||||
- Existing parse/normalize tests don't change (they don't hit the network)
|
||||
|
||||
### 5. `SKILL.md`
|
||||
|
||||
- Add Bluesky to the optional env vars section (near AUTH_TOKEN/CT0):
|
||||
```
|
||||
BSKY_HANDLE=your-handle.bsky.social
|
||||
BSKY_APP_PASSWORD=xxxx-xxxx-xxxx-xxxx
|
||||
```
|
||||
- Keep Bluesky in source lists but note "(requires app password)" in setup
|
||||
|
||||
## What NOT to Change
|
||||
|
||||
- `render.py` - already handles empty bluesky gracefully (hides stats line when 0 items)
|
||||
- `schema.py`, `normalize.py`, `score.py`, `dedupe.py` - no changes needed, they work on items already
|
||||
- No error display when unconfigured - completely silent, same as how TikTok/Instagram behave when SCRAPECREATORS_API_KEY is missing
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] `is_bluesky_available()` returns False when env vars not set
|
||||
- [x] No Bluesky stats line, no error, no mention when unconfigured
|
||||
- [x] With valid `BSKY_HANDLE` + `BSKY_APP_PASSWORD`, posts are returned
|
||||
- [x] Auth failure returns empty results gracefully (no crash)
|
||||
- [x] SKILL.md documents the env vars
|
||||
- [x] All existing Bluesky tests still pass
|
||||
- [x] New auth tests added
|
||||
- [ ] `bash scripts/sync.sh` deploys successfully
|
||||
|
||||
## Sources
|
||||
|
||||
- AT Protocol auth: `POST https://bsky.social/xrpc/com.atproto.server.createSession` with `{identifier, password}`
|
||||
- App passwords: `bsky.app/settings/app-passwords`
|
||||
- Existing pattern: X auth with AUTH_TOKEN/CT0 in `env.py:256-257`
|
||||
- Existing pattern: TikTok/Instagram opt-in via `is_tiktok_available()` in `env.py:499`
|
||||
@@ -519,6 +519,7 @@ def _search_bluesky(
|
||||
from_date: str,
|
||||
to_date: str,
|
||||
depth: str,
|
||||
config: dict = None,
|
||||
) -> tuple:
|
||||
"""Search Bluesky via AT Protocol (runs in thread).
|
||||
|
||||
@@ -529,7 +530,7 @@ def _search_bluesky(
|
||||
|
||||
try:
|
||||
response = bluesky.search_bluesky(
|
||||
topic, from_date, to_date, depth=depth,
|
||||
topic, from_date, to_date, depth=depth, config=config,
|
||||
)
|
||||
except Exception as e:
|
||||
return [], f"{type(e).__name__}: {e}"
|
||||
@@ -1062,7 +1063,7 @@ def run_research(
|
||||
|
||||
if do_bluesky:
|
||||
bluesky_future = executor.submit(
|
||||
_search_bluesky, topic, from_date, to_date, depth
|
||||
_search_bluesky, topic, from_date, to_date, depth, config
|
||||
)
|
||||
|
||||
if do_polymarket:
|
||||
@@ -1497,6 +1498,9 @@ def main():
|
||||
# Auto-detect Xiaohongshu HTTP API (requires service + login)
|
||||
has_xiaohongshu = env.is_xiaohongshu_available(config)
|
||||
|
||||
# Auto-detect Bluesky (requires BSKY_HANDLE + BSKY_APP_PASSWORD)
|
||||
has_bluesky = env.is_bluesky_available(config)
|
||||
|
||||
# --diagnose: show source availability and exit
|
||||
if args.diagnose:
|
||||
web_source = env.get_web_search_source(config)
|
||||
@@ -1514,7 +1518,7 @@ def main():
|
||||
"xiaohongshu": has_xiaohongshu,
|
||||
"xiaohongshu_api_base": env.get_xiaohongshu_api_base(config),
|
||||
"hackernews": True,
|
||||
"bluesky": True,
|
||||
"bluesky": has_bluesky,
|
||||
"polymarket": True,
|
||||
"web_search_backend": web_source,
|
||||
"parallel_ai": bool(config.get("PARALLEL_API_KEY")),
|
||||
@@ -1630,7 +1634,7 @@ def main():
|
||||
|
||||
# Apply --search flag: restrict sources to the specified subset
|
||||
search_do_hackernews = True
|
||||
search_do_bluesky = True
|
||||
search_do_bluesky = has_bluesky
|
||||
search_do_polymarket = True
|
||||
search_run_youtube = has_ytdlp
|
||||
search_run_tiktok = has_tiktok
|
||||
@@ -1641,7 +1645,7 @@ def main():
|
||||
has_reddit = "reddit" in search_sources
|
||||
has_x = "x" in search_sources
|
||||
search_do_hackernews = "hn" in search_sources
|
||||
search_do_bluesky = "bluesky" in search_sources or "bsky" in search_sources
|
||||
search_do_bluesky = ("bluesky" in search_sources or "bsky" in search_sources) and has_bluesky
|
||||
search_do_polymarket = "polymarket" in search_sources
|
||||
search_run_youtube = "youtube" in search_sources and has_ytdlp
|
||||
search_run_tiktok = "tiktok" in search_sources and has_tiktok
|
||||
|
||||
+60
-5
@@ -1,7 +1,7 @@
|
||||
"""Bluesky search via AT Protocol (free, no auth required).
|
||||
"""Bluesky search via AT Protocol (requires app password).
|
||||
|
||||
Uses public.api.bsky.app for post discovery.
|
||||
No API key needed - just HTTP calls via stdlib urllib.
|
||||
Uses bsky.social for auth and public.api.bsky.app for post search.
|
||||
Requires BSKY_HANDLE and BSKY_APP_PASSWORD env vars.
|
||||
"""
|
||||
|
||||
import math
|
||||
@@ -12,6 +12,7 @@ from typing import Any, Dict, List, Optional
|
||||
|
||||
from . import http
|
||||
|
||||
BSKY_SESSION_URL = "https://bsky.social/xrpc/com.atproto.server.createSession"
|
||||
BSKY_SEARCH_URL = "https://public.api.bsky.app/xrpc/app.bsky.feed.searchPosts"
|
||||
|
||||
DEPTH_CONFIG = {
|
||||
@@ -20,6 +21,9 @@ DEPTH_CONFIG = {
|
||||
"deep": 60,
|
||||
}
|
||||
|
||||
# Module-level token cache (valid for the lifetime of a single research run)
|
||||
_cached_token: Optional[str] = None
|
||||
|
||||
|
||||
def _log(msg: str):
|
||||
"""Log to stderr (only in TTY mode to avoid cluttering Claude Code output)."""
|
||||
@@ -28,6 +32,39 @@ def _log(msg: str):
|
||||
sys.stderr.flush()
|
||||
|
||||
|
||||
def _create_session(handle: str, app_password: str) -> Optional[str]:
|
||||
"""Create an AT Protocol session and return the access token.
|
||||
|
||||
Args:
|
||||
handle: Bluesky handle (e.g. user.bsky.social)
|
||||
app_password: App password from bsky.app/settings/app-passwords
|
||||
|
||||
Returns:
|
||||
Access JWT string, or None on failure.
|
||||
"""
|
||||
global _cached_token
|
||||
if _cached_token:
|
||||
return _cached_token
|
||||
|
||||
try:
|
||||
response = http.request(
|
||||
"POST",
|
||||
BSKY_SESSION_URL,
|
||||
json_data={"identifier": handle, "password": app_password},
|
||||
timeout=15,
|
||||
)
|
||||
token = response.get("accessJwt")
|
||||
if token:
|
||||
_cached_token = token
|
||||
_log("Session created successfully")
|
||||
return token
|
||||
_log("No accessJwt in session response")
|
||||
return None
|
||||
except Exception as e:
|
||||
_log(f"Session creation failed: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def _extract_core_subject(topic: str) -> str:
|
||||
"""Extract core subject from verbose query for Bluesky search."""
|
||||
text = topic.lower().strip()
|
||||
@@ -73,18 +110,32 @@ def search_bluesky(
|
||||
from_date: str,
|
||||
to_date: str,
|
||||
depth: str = "default",
|
||||
config: Optional[Dict[str, Any]] = None,
|
||||
) -> Dict[str, Any]:
|
||||
"""Search Bluesky via AT Protocol public API.
|
||||
"""Search Bluesky via AT Protocol API.
|
||||
|
||||
Args:
|
||||
topic: Search topic
|
||||
from_date: Start date (YYYY-MM-DD)
|
||||
to_date: End date (YYYY-MM-DD)
|
||||
depth: 'quick', 'default', or 'deep'
|
||||
config: Config dict with BSKY_HANDLE and BSKY_APP_PASSWORD
|
||||
|
||||
Returns:
|
||||
Dict with 'posts' list from AT Protocol response.
|
||||
"""
|
||||
config = config or {}
|
||||
handle = config.get("BSKY_HANDLE", "")
|
||||
app_password = config.get("BSKY_APP_PASSWORD", "")
|
||||
|
||||
if not handle or not app_password:
|
||||
return {"posts": [], "error": "Bluesky credentials not configured"}
|
||||
|
||||
# Authenticate
|
||||
token = _create_session(handle, app_password)
|
||||
if not token:
|
||||
return {"posts": [], "error": "Bluesky auth failed"}
|
||||
|
||||
count = DEPTH_CONFIG.get(depth, DEPTH_CONFIG["default"])
|
||||
core_topic = _extract_core_subject(topic)
|
||||
|
||||
@@ -99,7 +150,11 @@ def search_bluesky(
|
||||
url = f"{BSKY_SEARCH_URL}?{urlencode(params)}"
|
||||
|
||||
try:
|
||||
response = http.request("GET", url, timeout=30)
|
||||
response = http.request(
|
||||
"GET", url,
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
timeout=30,
|
||||
)
|
||||
except http.HTTPError as e:
|
||||
_log(f"Search failed: {e}")
|
||||
return {"posts": [], "error": str(e)}
|
||||
|
||||
+5
-3
@@ -255,6 +255,8 @@ def get_config() -> Dict[str, Any]:
|
||||
('APIFY_API_TOKEN', None),
|
||||
('AUTH_TOKEN', None),
|
||||
('CT0', None),
|
||||
('BSKY_HANDLE', None),
|
||||
('BSKY_APP_PASSWORD', None),
|
||||
]
|
||||
|
||||
for key, default in keys:
|
||||
@@ -480,12 +482,12 @@ def is_hackernews_available() -> bool:
|
||||
return True
|
||||
|
||||
|
||||
def is_bluesky_available() -> bool:
|
||||
def is_bluesky_available(config: Dict[str, Any]) -> bool:
|
||||
"""Check if Bluesky source is available.
|
||||
|
||||
Always returns True - AT Protocol search is free, no key needed.
|
||||
Requires BSKY_HANDLE and BSKY_APP_PASSWORD (app password from bsky.app/settings).
|
||||
"""
|
||||
return True
|
||||
return bool(config.get('BSKY_HANDLE') and config.get('BSKY_APP_PASSWORD'))
|
||||
|
||||
|
||||
def is_polymarket_available() -> bool:
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent / "scripts"))
|
||||
from lib import bluesky
|
||||
@@ -99,5 +100,79 @@ class TestDepthConfig(unittest.TestCase):
|
||||
self.assertGreater(deep, quick)
|
||||
|
||||
|
||||
class TestCreateSession(unittest.TestCase):
|
||||
def setUp(self):
|
||||
bluesky._cached_token = None
|
||||
|
||||
def tearDown(self):
|
||||
bluesky._cached_token = None
|
||||
|
||||
@patch("lib.bluesky.http.request")
|
||||
def test_returns_token(self, mock_request):
|
||||
mock_request.return_value = {"accessJwt": "tok123", "refreshJwt": "ref456"}
|
||||
token = bluesky._create_session("user.bsky.social", "app-pw")
|
||||
self.assertEqual(token, "tok123")
|
||||
mock_request.assert_called_once()
|
||||
|
||||
@patch("lib.bluesky.http.request")
|
||||
def test_caches_token(self, mock_request):
|
||||
mock_request.return_value = {"accessJwt": "tok123", "refreshJwt": "ref456"}
|
||||
bluesky._create_session("user.bsky.social", "app-pw")
|
||||
bluesky._create_session("user.bsky.social", "app-pw")
|
||||
mock_request.assert_called_once() # Only one HTTP call
|
||||
|
||||
@patch("lib.bluesky.http.request")
|
||||
def test_returns_none_on_failure(self, mock_request):
|
||||
mock_request.side_effect = Exception("connection refused")
|
||||
token = bluesky._create_session("user.bsky.social", "app-pw")
|
||||
self.assertIsNone(token)
|
||||
|
||||
@patch("lib.bluesky.http.request")
|
||||
def test_returns_none_on_missing_jwt(self, mock_request):
|
||||
mock_request.return_value = {"did": "did:plc:abc"}
|
||||
token = bluesky._create_session("user.bsky.social", "app-pw")
|
||||
self.assertIsNone(token)
|
||||
|
||||
|
||||
class TestSearchBlueskyAuth(unittest.TestCase):
|
||||
def setUp(self):
|
||||
bluesky._cached_token = None
|
||||
|
||||
def tearDown(self):
|
||||
bluesky._cached_token = None
|
||||
|
||||
def test_no_config_returns_error(self):
|
||||
result = bluesky.search_bluesky("test", "2026-01-01", "2026-03-09")
|
||||
self.assertEqual(result["posts"], [])
|
||||
self.assertIn("not configured", result["error"])
|
||||
|
||||
def test_empty_config_returns_error(self):
|
||||
result = bluesky.search_bluesky("test", "2026-01-01", "2026-03-09", config={})
|
||||
self.assertEqual(result["posts"], [])
|
||||
self.assertIn("not configured", result["error"])
|
||||
|
||||
@patch("lib.bluesky.http.request")
|
||||
def test_auth_failure_returns_error(self, mock_request):
|
||||
mock_request.side_effect = Exception("401 Unauthorized")
|
||||
config = {"BSKY_HANDLE": "user.bsky.social", "BSKY_APP_PASSWORD": "pw"}
|
||||
result = bluesky.search_bluesky("test", "2026-01-01", "2026-03-09", config=config)
|
||||
self.assertEqual(result["posts"], [])
|
||||
self.assertIn("auth failed", result["error"])
|
||||
|
||||
@patch("lib.bluesky.http.request")
|
||||
def test_successful_search_passes_bearer(self, mock_request):
|
||||
# First call: createSession, second call: searchPosts
|
||||
mock_request.side_effect = [
|
||||
{"accessJwt": "tok123", "refreshJwt": "ref456"},
|
||||
{"posts": [{"uri": "at://did/app.bsky.feed.post/abc", "author": {"handle": "u1"}, "record": {"text": "hi"}}]},
|
||||
]
|
||||
config = {"BSKY_HANDLE": "user.bsky.social", "BSKY_APP_PASSWORD": "pw"}
|
||||
result = bluesky.search_bluesky("test", "2026-01-01", "2026-03-09", config=config)
|
||||
self.assertEqual(len(result["posts"]), 1)
|
||||
# Verify the search call included the Bearer token
|
||||
search_call = mock_request.call_args_list[1]
|
||||
self.assertEqual(search_call.kwargs.get("headers", {}), {"Authorization": "Bearer tok123"})
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
Reference in New Issue
Block a user