fix(bluesky): make Bluesky opt-in with app password auth

searchPosts endpoint now returns 403 for unauthenticated requests.
Add session auth via createSession, gate on BSKY_HANDLE + BSKY_APP_PASSWORD
env vars. When unconfigured, Bluesky is completely invisible (no error).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Matt Van Horn
2026-03-09 23:27:39 -07:00
parent ecf90c0281
commit adb5a672d9
6 changed files with 257 additions and 14 deletions
+3 -1
View File
@@ -24,6 +24,8 @@ metadata:
- APIFY_API_TOKEN - APIFY_API_TOKEN
- AUTH_TOKEN - AUTH_TOKEN
- CT0 - CT0
- BSKY_HANDLE
- BSKY_APP_PASSWORD
bins: bins:
- node - node
- python3 - python3
@@ -46,7 +48,7 @@ metadata:
# last30days v2.9.5: Research Any Topic from the Last 30 Days # last30days v2.9.5: Research Any Topic from the Last 30 Days
> **Permissions overview:** Reads public web/platform data and optionally saves research briefings to `~/Documents/Last30Days/`. X/Twitter search uses optional user-provided tokens (AUTH_TOKEN/CT0 env vars) — no browser session access. All credential usage and data writes are documented in the [Security & Permissions](#security--permissions) section. > **Permissions overview:** Reads public web/platform data and optionally saves research briefings to `~/Documents/Last30Days/`. X/Twitter search uses optional user-provided tokens (AUTH_TOKEN/CT0 env vars). Bluesky search uses optional app password (BSKY_HANDLE/BSKY_APP_PASSWORD env vars - create at bsky.app/settings/app-passwords). All credential usage and data writes are documented in the [Security & Permissions](#security--permissions) section.
Research ANY topic across Reddit, X, Bluesky, YouTube, TikTok, Hacker News, Polymarket, and the web. Surface what people are actually discussing, recommending, betting on, and debating right now. Research ANY topic across Reddit, X, Bluesky, YouTube, TikTok, Hacker News, Polymarket, and the web. Surface what people are actually discussing, recommending, betting on, and debating right now.
@@ -0,0 +1,105 @@
---
title: "fix: Make Bluesky opt-in with auth (searchPosts now requires authentication)"
type: fix
status: completed
date: 2026-03-09
---
# fix: Make Bluesky opt-in with auth (searchPosts now requires authentication)
## Problem
Bluesky's `app.bsky.feed.searchPosts` endpoint now returns **403 Forbidden** for unauthenticated requests. This broke today (2026-03-09), likely related to the CEO transition. The `searchActors` endpoint still works without auth, but `searchPosts` does not.
We built Bluesky as an always-on source (like HN and Polymarket) because the AT Protocol public API was documented as free. That's no longer true for post search.
**Current behavior:** Bluesky silently returns 0 results (403 caught, empty list returned). No stats line appears, no error shown. Users don't know it exists.
**Desired behavior:** Bluesky is opt-in via env vars. When not configured, completely invisible (no error, no stats line, no mention). When configured with auth, it works as before.
## Approach
Follow the same pattern as X/Twitter auth: env vars for credentials, availability check gates dispatch.
AT Protocol auth flow:
1. User creates an App Password at `bsky.app/settings/app-passwords`
2. User sets `BSKY_HANDLE` and `BSKY_APP_PASSWORD` env vars
3. `bluesky.py` calls `com.atproto.server.createSession` to get a bearer token
4. Subsequent `searchPosts` calls include `Authorization: Bearer {token}`
## Files to Change
| File | Change | Lines (est.) |
|------|--------|-------------|
| `scripts/lib/bluesky.py` | Add `_create_session()` auth, pass bearer token to search | ~25 |
| `scripts/lib/env.py` | Update `is_bluesky_available()` to check env vars; add env var loading | ~10 |
| `scripts/last30days.py` | Gate `do_bluesky` on `is_bluesky_available(config)`; update diag dicts | ~8 |
| `tests/test_bluesky.py` | Add auth session tests, update existing tests | ~15 |
| `SKILL.md` | Document `BSKY_HANDLE` / `BSKY_APP_PASSWORD` env vars in setup section | ~5 |
## Implementation Steps
### 1. `scripts/lib/env.py`
- Add `BSKY_HANDLE` and `BSKY_APP_PASSWORD` to the env var loading in `get_config()`
- Update `is_bluesky_available()`:
```python
def is_bluesky_available(config: Dict[str, Any]) -> bool:
return bool(config.get('BSKY_HANDLE') and config.get('BSKY_APP_PASSWORD'))
```
### 2. `scripts/lib/bluesky.py`
- Add `_create_session(handle, app_password)` that POSTs to `https://bsky.social/xrpc/com.atproto.server.createSession`
- Cache the access token for the lifetime of the search (module-level or passed through)
- Update `search_bluesky()` to accept config dict, extract creds, create session, add `Authorization: Bearer {token}` header
- On auth failure, return `{"posts": [], "error": "Bluesky auth failed"}` (don't raise)
### 3. `scripts/last30days.py`
- Change `do_bluesky` default from `True` to checking `is_bluesky_available(config)`:
```python
has_bluesky = env.is_bluesky_available(config)
```
- Gate the `bluesky_future` submit on `has_bluesky` (already gated on `do_bluesky`, just wire it)
- Update both diagnostic dicts: `"bluesky": True` -> `"bluesky": has_bluesky`
- Pass `config` to `_search_bluesky()` so it can extract auth creds
### 4. `tests/test_bluesky.py`
- Test `_create_session` with mocked HTTP response
- Test `search_bluesky` with auth header injection
- Existing parse/normalize tests don't change (they don't hit the network)
### 5. `SKILL.md`
- Add Bluesky to the optional env vars section (near AUTH_TOKEN/CT0):
```
BSKY_HANDLE=your-handle.bsky.social
BSKY_APP_PASSWORD=xxxx-xxxx-xxxx-xxxx
```
- Keep Bluesky in source lists but note "(requires app password)" in setup
## What NOT to Change
- `render.py` - already handles empty bluesky gracefully (hides stats line when 0 items)
- `schema.py`, `normalize.py`, `score.py`, `dedupe.py` - no changes needed, they work on items already
- No error display when unconfigured - completely silent, same as how TikTok/Instagram behave when SCRAPECREATORS_API_KEY is missing
## Acceptance Criteria
- [x] `is_bluesky_available()` returns False when env vars not set
- [x] No Bluesky stats line, no error, no mention when unconfigured
- [x] With valid `BSKY_HANDLE` + `BSKY_APP_PASSWORD`, posts are returned
- [x] Auth failure returns empty results gracefully (no crash)
- [x] SKILL.md documents the env vars
- [x] All existing Bluesky tests still pass
- [x] New auth tests added
- [ ] `bash scripts/sync.sh` deploys successfully
## Sources
- AT Protocol auth: `POST https://bsky.social/xrpc/com.atproto.server.createSession` with `{identifier, password}`
- App passwords: `bsky.app/settings/app-passwords`
- Existing pattern: X auth with AUTH_TOKEN/CT0 in `env.py:256-257`
- Existing pattern: TikTok/Instagram opt-in via `is_tiktok_available()` in `env.py:499`
+9 -5
View File
@@ -519,6 +519,7 @@ def _search_bluesky(
from_date: str, from_date: str,
to_date: str, to_date: str,
depth: str, depth: str,
config: dict = None,
) -> tuple: ) -> tuple:
"""Search Bluesky via AT Protocol (runs in thread). """Search Bluesky via AT Protocol (runs in thread).
@@ -529,7 +530,7 @@ def _search_bluesky(
try: try:
response = bluesky.search_bluesky( response = bluesky.search_bluesky(
topic, from_date, to_date, depth=depth, topic, from_date, to_date, depth=depth, config=config,
) )
except Exception as e: except Exception as e:
return [], f"{type(e).__name__}: {e}" return [], f"{type(e).__name__}: {e}"
@@ -1062,7 +1063,7 @@ def run_research(
if do_bluesky: if do_bluesky:
bluesky_future = executor.submit( bluesky_future = executor.submit(
_search_bluesky, topic, from_date, to_date, depth _search_bluesky, topic, from_date, to_date, depth, config
) )
if do_polymarket: if do_polymarket:
@@ -1497,6 +1498,9 @@ def main():
# Auto-detect Xiaohongshu HTTP API (requires service + login) # Auto-detect Xiaohongshu HTTP API (requires service + login)
has_xiaohongshu = env.is_xiaohongshu_available(config) has_xiaohongshu = env.is_xiaohongshu_available(config)
# Auto-detect Bluesky (requires BSKY_HANDLE + BSKY_APP_PASSWORD)
has_bluesky = env.is_bluesky_available(config)
# --diagnose: show source availability and exit # --diagnose: show source availability and exit
if args.diagnose: if args.diagnose:
web_source = env.get_web_search_source(config) web_source = env.get_web_search_source(config)
@@ -1514,7 +1518,7 @@ def main():
"xiaohongshu": has_xiaohongshu, "xiaohongshu": has_xiaohongshu,
"xiaohongshu_api_base": env.get_xiaohongshu_api_base(config), "xiaohongshu_api_base": env.get_xiaohongshu_api_base(config),
"hackernews": True, "hackernews": True,
"bluesky": True, "bluesky": has_bluesky,
"polymarket": True, "polymarket": True,
"web_search_backend": web_source, "web_search_backend": web_source,
"parallel_ai": bool(config.get("PARALLEL_API_KEY")), "parallel_ai": bool(config.get("PARALLEL_API_KEY")),
@@ -1630,7 +1634,7 @@ def main():
# Apply --search flag: restrict sources to the specified subset # Apply --search flag: restrict sources to the specified subset
search_do_hackernews = True search_do_hackernews = True
search_do_bluesky = True search_do_bluesky = has_bluesky
search_do_polymarket = True search_do_polymarket = True
search_run_youtube = has_ytdlp search_run_youtube = has_ytdlp
search_run_tiktok = has_tiktok search_run_tiktok = has_tiktok
@@ -1641,7 +1645,7 @@ def main():
has_reddit = "reddit" in search_sources has_reddit = "reddit" in search_sources
has_x = "x" in search_sources has_x = "x" in search_sources
search_do_hackernews = "hn" in search_sources search_do_hackernews = "hn" in search_sources
search_do_bluesky = "bluesky" in search_sources or "bsky" in search_sources search_do_bluesky = ("bluesky" in search_sources or "bsky" in search_sources) and has_bluesky
search_do_polymarket = "polymarket" in search_sources search_do_polymarket = "polymarket" in search_sources
search_run_youtube = "youtube" in search_sources and has_ytdlp search_run_youtube = "youtube" in search_sources and has_ytdlp
search_run_tiktok = "tiktok" in search_sources and has_tiktok search_run_tiktok = "tiktok" in search_sources and has_tiktok
+60 -5
View File
@@ -1,7 +1,7 @@
"""Bluesky search via AT Protocol (free, no auth required). """Bluesky search via AT Protocol (requires app password).
Uses public.api.bsky.app for post discovery. Uses bsky.social for auth and public.api.bsky.app for post search.
No API key needed - just HTTP calls via stdlib urllib. Requires BSKY_HANDLE and BSKY_APP_PASSWORD env vars.
""" """
import math import math
@@ -12,6 +12,7 @@ from typing import Any, Dict, List, Optional
from . import http from . import http
BSKY_SESSION_URL = "https://bsky.social/xrpc/com.atproto.server.createSession"
BSKY_SEARCH_URL = "https://public.api.bsky.app/xrpc/app.bsky.feed.searchPosts" BSKY_SEARCH_URL = "https://public.api.bsky.app/xrpc/app.bsky.feed.searchPosts"
DEPTH_CONFIG = { DEPTH_CONFIG = {
@@ -20,6 +21,9 @@ DEPTH_CONFIG = {
"deep": 60, "deep": 60,
} }
# Module-level token cache (valid for the lifetime of a single research run)
_cached_token: Optional[str] = None
def _log(msg: str): def _log(msg: str):
"""Log to stderr (only in TTY mode to avoid cluttering Claude Code output).""" """Log to stderr (only in TTY mode to avoid cluttering Claude Code output)."""
@@ -28,6 +32,39 @@ def _log(msg: str):
sys.stderr.flush() sys.stderr.flush()
def _create_session(handle: str, app_password: str) -> Optional[str]:
"""Create an AT Protocol session and return the access token.
Args:
handle: Bluesky handle (e.g. user.bsky.social)
app_password: App password from bsky.app/settings/app-passwords
Returns:
Access JWT string, or None on failure.
"""
global _cached_token
if _cached_token:
return _cached_token
try:
response = http.request(
"POST",
BSKY_SESSION_URL,
json_data={"identifier": handle, "password": app_password},
timeout=15,
)
token = response.get("accessJwt")
if token:
_cached_token = token
_log("Session created successfully")
return token
_log("No accessJwt in session response")
return None
except Exception as e:
_log(f"Session creation failed: {e}")
return None
def _extract_core_subject(topic: str) -> str: def _extract_core_subject(topic: str) -> str:
"""Extract core subject from verbose query for Bluesky search.""" """Extract core subject from verbose query for Bluesky search."""
text = topic.lower().strip() text = topic.lower().strip()
@@ -73,18 +110,32 @@ def search_bluesky(
from_date: str, from_date: str,
to_date: str, to_date: str,
depth: str = "default", depth: str = "default",
config: Optional[Dict[str, Any]] = None,
) -> Dict[str, Any]: ) -> Dict[str, Any]:
"""Search Bluesky via AT Protocol public API. """Search Bluesky via AT Protocol API.
Args: Args:
topic: Search topic topic: Search topic
from_date: Start date (YYYY-MM-DD) from_date: Start date (YYYY-MM-DD)
to_date: End date (YYYY-MM-DD) to_date: End date (YYYY-MM-DD)
depth: 'quick', 'default', or 'deep' depth: 'quick', 'default', or 'deep'
config: Config dict with BSKY_HANDLE and BSKY_APP_PASSWORD
Returns: Returns:
Dict with 'posts' list from AT Protocol response. Dict with 'posts' list from AT Protocol response.
""" """
config = config or {}
handle = config.get("BSKY_HANDLE", "")
app_password = config.get("BSKY_APP_PASSWORD", "")
if not handle or not app_password:
return {"posts": [], "error": "Bluesky credentials not configured"}
# Authenticate
token = _create_session(handle, app_password)
if not token:
return {"posts": [], "error": "Bluesky auth failed"}
count = DEPTH_CONFIG.get(depth, DEPTH_CONFIG["default"]) count = DEPTH_CONFIG.get(depth, DEPTH_CONFIG["default"])
core_topic = _extract_core_subject(topic) core_topic = _extract_core_subject(topic)
@@ -99,7 +150,11 @@ def search_bluesky(
url = f"{BSKY_SEARCH_URL}?{urlencode(params)}" url = f"{BSKY_SEARCH_URL}?{urlencode(params)}"
try: try:
response = http.request("GET", url, timeout=30) response = http.request(
"GET", url,
headers={"Authorization": f"Bearer {token}"},
timeout=30,
)
except http.HTTPError as e: except http.HTTPError as e:
_log(f"Search failed: {e}") _log(f"Search failed: {e}")
return {"posts": [], "error": str(e)} return {"posts": [], "error": str(e)}
+5 -3
View File
@@ -255,6 +255,8 @@ def get_config() -> Dict[str, Any]:
('APIFY_API_TOKEN', None), ('APIFY_API_TOKEN', None),
('AUTH_TOKEN', None), ('AUTH_TOKEN', None),
('CT0', None), ('CT0', None),
('BSKY_HANDLE', None),
('BSKY_APP_PASSWORD', None),
] ]
for key, default in keys: for key, default in keys:
@@ -480,12 +482,12 @@ def is_hackernews_available() -> bool:
return True return True
def is_bluesky_available() -> bool: def is_bluesky_available(config: Dict[str, Any]) -> bool:
"""Check if Bluesky source is available. """Check if Bluesky source is available.
Always returns True - AT Protocol search is free, no key needed. Requires BSKY_HANDLE and BSKY_APP_PASSWORD (app password from bsky.app/settings).
""" """
return True return bool(config.get('BSKY_HANDLE') and config.get('BSKY_APP_PASSWORD'))
def is_polymarket_available() -> bool: def is_polymarket_available() -> bool:
+75
View File
@@ -3,6 +3,7 @@
import sys import sys
import unittest import unittest
from pathlib import Path from pathlib import Path
from unittest.mock import patch, MagicMock
sys.path.insert(0, str(Path(__file__).parent.parent / "scripts")) sys.path.insert(0, str(Path(__file__).parent.parent / "scripts"))
from lib import bluesky from lib import bluesky
@@ -99,5 +100,79 @@ class TestDepthConfig(unittest.TestCase):
self.assertGreater(deep, quick) self.assertGreater(deep, quick)
class TestCreateSession(unittest.TestCase):
def setUp(self):
bluesky._cached_token = None
def tearDown(self):
bluesky._cached_token = None
@patch("lib.bluesky.http.request")
def test_returns_token(self, mock_request):
mock_request.return_value = {"accessJwt": "tok123", "refreshJwt": "ref456"}
token = bluesky._create_session("user.bsky.social", "app-pw")
self.assertEqual(token, "tok123")
mock_request.assert_called_once()
@patch("lib.bluesky.http.request")
def test_caches_token(self, mock_request):
mock_request.return_value = {"accessJwt": "tok123", "refreshJwt": "ref456"}
bluesky._create_session("user.bsky.social", "app-pw")
bluesky._create_session("user.bsky.social", "app-pw")
mock_request.assert_called_once() # Only one HTTP call
@patch("lib.bluesky.http.request")
def test_returns_none_on_failure(self, mock_request):
mock_request.side_effect = Exception("connection refused")
token = bluesky._create_session("user.bsky.social", "app-pw")
self.assertIsNone(token)
@patch("lib.bluesky.http.request")
def test_returns_none_on_missing_jwt(self, mock_request):
mock_request.return_value = {"did": "did:plc:abc"}
token = bluesky._create_session("user.bsky.social", "app-pw")
self.assertIsNone(token)
class TestSearchBlueskyAuth(unittest.TestCase):
def setUp(self):
bluesky._cached_token = None
def tearDown(self):
bluesky._cached_token = None
def test_no_config_returns_error(self):
result = bluesky.search_bluesky("test", "2026-01-01", "2026-03-09")
self.assertEqual(result["posts"], [])
self.assertIn("not configured", result["error"])
def test_empty_config_returns_error(self):
result = bluesky.search_bluesky("test", "2026-01-01", "2026-03-09", config={})
self.assertEqual(result["posts"], [])
self.assertIn("not configured", result["error"])
@patch("lib.bluesky.http.request")
def test_auth_failure_returns_error(self, mock_request):
mock_request.side_effect = Exception("401 Unauthorized")
config = {"BSKY_HANDLE": "user.bsky.social", "BSKY_APP_PASSWORD": "pw"}
result = bluesky.search_bluesky("test", "2026-01-01", "2026-03-09", config=config)
self.assertEqual(result["posts"], [])
self.assertIn("auth failed", result["error"])
@patch("lib.bluesky.http.request")
def test_successful_search_passes_bearer(self, mock_request):
# First call: createSession, second call: searchPosts
mock_request.side_effect = [
{"accessJwt": "tok123", "refreshJwt": "ref456"},
{"posts": [{"uri": "at://did/app.bsky.feed.post/abc", "author": {"handle": "u1"}, "record": {"text": "hi"}}]},
]
config = {"BSKY_HANDLE": "user.bsky.social", "BSKY_APP_PASSWORD": "pw"}
result = bluesky.search_bluesky("test", "2026-01-01", "2026-03-09", config=config)
self.assertEqual(len(result["posts"]), 1)
# Verify the search call included the Bearer token
search_call = mock_request.call_args_list[1]
self.assertEqual(search_call.kwargs.get("headers", {}), {"Authorization": "Bearer tok123"})
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()