diff --git a/docs/plans/2026-02-26-fix-polymarket-query-expansion-data-availability-plan.md b/docs/plans/2026-02-26-fix-polymarket-query-expansion-data-availability-plan.md new file mode 100644 index 0000000..e869b68 --- /dev/null +++ b/docs/plans/2026-02-26-fix-polymarket-query-expansion-data-availability-plan.md @@ -0,0 +1,235 @@ +--- +title: "fix: Polymarket query expansion - find markets where topic is an outcome, not just a title" +type: fix +status: active +date: 2026-02-26 +--- + +# fix: Polymarket Query Expansion & Data Availability + +## Overview + +The Polymarket module misses the most interesting markets when the search topic is an **outcome** in a broader market rather than appearing in the event title. For "Arizona Basketball", the NCAA Tournament Winner (30 open markets) and #1 Seed (20 open markets) are invisible because "Arizona" only appears as an outcome, never in the event title. The Gamma API only searches titles/slugs. + +The outcome-aware scoring (from the prior plan) works perfectly on fixture data - it correctly ranks markets where Arizona is an outcome. The problem is upstream: those markets never reach the scoring layer because the Gamma API never returns them. + +## Problem Statement + +### Root Cause: Gamma API is title/slug search only + +`gamma-api.polymarket.com/public-search?q=X` only matches against event titles and slugs. It does NOT search outcome names, market descriptions, or tags. + +**Live API verification** (2026-02-26): + +| Query | Events returned | Arizona-relevant open markets | +|-------|----------------|-------------------------------| +| "Arizona Basketball" | 5 | 2 (Big 12 Champion, NAU vs Idaho) | +| "Arizona" | 5 | 0 (all political/closed) | +| "NCAA Tournament" | 5 | 2 (Tournament Winner: 30 open, #1 Seed: 20 open) | +| "Basketball" | 5 | 5 (conference champions, no NCAA) | +| "college basketball" | 5 | 1 (#1 seed) | + +The championship and seeding markets that the user wants (`NCAA Tournament Winner`, `#1 Seed`) are only findable by searching "NCAA Tournament" or "NCAA" - terms that don't appear in "Arizona Basketball". + +### Contributing Factor 1: Query expansion too narrow + +`_expand_queries("Arizona Basketball")` generates only `["Arizona Basketball", "Arizona"]`. + +It only tries the **first word** as a standalone query. The second word "Basketball" is never searched independently. This means conference-adjacent and tournament markets are invisible. + +### Contributing Factor 2: No domain bridging + +Even searching "Basketball" (all individual words) only returns conference champions. The leap from "Basketball" to "NCAA Tournament" requires discovering the domain context from initial results. Currently there is no second-pass expansion. + +### Contributing Factor 3: Shallow default depth + +`DEPTH_CONFIG["default"] = 2` pages (10 events per query). With 3 queries that's 30 raw events, but heavy dedup and closed-event filtering reduces this to 2-5 usable results. + +### What works (don't break it) + +- "Iran War" returned 9 perfect markets because "Iran" and "War" appear directly in event titles +- Outcome-aware scoring correctly ranks Arizona-outcome markets when they reach the scoring layer +- Topic-matching outcome reordering surfaces the right outcome first + +## Proposed Solution + +Three changes, all generic (no hardcoded domain knowledge): + +### 1. Search ALL individual words, not just the first + +Currently `_expand_queries()` only adds `words[0]` as a standalone query. Change to add **every word** as a standalone query, then dedupe. + +For "Arizona Basketball": `["Arizona Basketball", "Arizona", "Basketball"]` +For "Iran War": `["Iran War", "Iran", "War"]` (same as before since both are short) +For "AI video generation tools": `["AI video generation tools", "AI", "video", "generation", "tools"]` (capped at 6) + +Raise query cap from 4 to 6 to accommodate. + +```python +def _expand_queries(topic: str) -> List[str]: + core = _extract_core_subject(topic) + queries = [core] + + words = core.split() + if len(words) >= 2: + # Add ALL individual words (not just first) + for word in words: + if len(word) > 2: # skip very short words ("AI", "vs") + queries.append(word) + + if topic.lower().strip() != core.lower(): + queries.append(topic.strip()) + + # Dedupe, cap at 6 + seen = set() + unique = [] + for q in queries: + q_lower = q.lower().strip() + if q_lower and q_lower not in seen: + seen.add(q_lower) + unique.append(q.strip()) + return unique[:6] +``` + +Note: "AI" is only 2 chars but is meaningful. Lower the threshold to `len(word) > 1` to catch it. Single-char words (rare) get filtered. + +### 2. Second-pass context expansion from first-pass results + +After the first-pass search, extract domain-indicator terms from event titles and run a focused second-pass search. + +Algorithm: +1. Collect ALL event titles from first-pass results (including closed events) +2. Tokenize titles into bigrams (two-word sequences) +3. Count bigrams across events, filter out bigrams containing topic words +4. Take the top 1-2 most frequent non-topic bigrams as "domain indicators" +5. Search each domain indicator (1 page each) +6. Merge with first-pass results, dedupe, re-rank + +**Example for "Arizona Basketball":** + +First-pass titles include: +- "Big 12 Men's College Basketball 2025-2026 Regular Season Champion" +- "SEC Men's College Basketball 2025-2026 Regular Season Champion" +- "ACC Men's College Basketball 2025-2026 Regular Season Champion" +- "Big East Men's College Basketball 2025-2026 Regular Season Champion" + +Frequent bigrams (excluding topic words): "college basketball" (4x), "regular season" (4x), "season champion" (4x) + +Top domain indicator: **"college basketball"** + +Searching "college basketball" returns: **"#1 seed in NCAA Tournament"** (20 open markets with Arizona as outcome!) + +**Example for "Iran War":** First-pass already finds everything via title matches. Second-pass bigrams would be things like "iran strikes", "khamenei out" - searching these finds the same events (deduped). No regression. + +```python +def _extract_domain_queries(topic: str, events: List[Dict]) -> List[str]: + """Extract domain-indicator search terms from first-pass event titles.""" + topic_words = set(_extract_core_subject(topic).lower().split()) + + # Collect bigrams from all event titles + bigram_counts = {} + for event in events: + title = event.get("title", "").lower() + words = re.findall(r'[a-z]+', title) + for i in range(len(words) - 1): + bigram = f"{words[i]} {words[i+1]}" + # Skip if both words are in topic (we already search the topic) + if words[i] in topic_words and words[i+1] in topic_words: + continue + # Skip very common filler + if any(w in ("the", "of", "in", "to", "a", "and", "vs", "will", "be") for w in (words[i], words[i+1])): + continue + bigram_counts[bigram] = bigram_counts.get(bigram, 0) + 1 + + # Return bigrams appearing in 2+ event titles + domain_queries = [bg for bg, count in sorted(bigram_counts.items(), key=lambda x: -x[1]) if count >= 2] + return domain_queries[:2] +``` + +### 3. Increase depth and result caps + +```python +DEPTH_CONFIG = { + "quick": 1, + "default": 3, # was 2 (50% more raw results) + "deep": 4, # was 3 +} + +RESULT_CAP = { + "quick": 5, + "default": 15, # was 10 (more room for cross-domain markets) + "deep": 25, # was 20 +} +``` + +This gives default searches 3 queries x 3 pages = 45 raw events (up from 2 queries x 2 pages = 20), plus 2 domain-indicator queries x 1 page = 10 more. + +## Technical Approach + +### Implementation Plan + +#### Phase 1: Expand query generation + +- [x] `scripts/lib/polymarket.py` - Update `_expand_queries()` to add ALL individual words (len > 1), raise cap from 4 to 6 +- [x] `tests/test_polymarket.py` - Test: `_expand_queries("Arizona Basketball")` returns `["Arizona Basketball", "Arizona", "Basketball"]` +- [x] `tests/test_polymarket.py` - Test: `_expand_queries("Iran War")` returns `["Iran War", "Iran", "War"]` +- [x] `tests/test_polymarket.py` - Test: short words excluded, cap at 6 + +#### Phase 2: Second-pass context expansion (evolved: tags instead of bigrams) + +- [x] `scripts/lib/polymarket.py` - Add `_extract_domain_queries()` using event TAGS (not bigrams - tags are more reliable, contain "NCAA CBB" etc.) +- [x] `scripts/lib/polymarket.py` - Update `search_polymarket()` with `_run_queries_parallel()` helper for two-pass search +- [x] `scripts/lib/polymarket.py` - Add `_shorten_question()` to extract team names from neg-risk binary market questions +- [x] `scripts/lib/polymarket.py` - Synthesize outcome_prices from binary sub-market questions (detects Yes/No pattern, not just negRisk flag) +- [x] `scripts/lib/polymarket.py` - Updated outcome reordering to use token-based matching for long question strings +- [x] `scripts/lib/polymarket.py` - Also pass market questions to `_compute_text_similarity()` for neg-risk events +- [x] `tests/test_polymarket.py` - Tests for tag-based domain extraction: frequent tags, generic tag filtering, topic word filtering, min frequency, cap, empty events + +#### Phase 3: Increase depth and caps + +- [x] `scripts/lib/polymarket.py` - Update `DEPTH_CONFIG`: default 2 -> 3, deep 3 -> 4 +- [x] `scripts/lib/polymarket.py` - Update `RESULT_CAP`: default 10 -> 15, deep 20 -> 25 + +#### Phase 4: Tests and verification + +- [x] Run full test suite (238 passed, 5 pre-existing failures unrelated) +- [x] `bash scripts/sync.sh` to deploy to CROSS +- [x] Live API test: "Arizona Basketball" - finds NCAA Tournament Winner (Arizona: 12%), #1 Seed (Arizona: 88%), Big 12 (Arizona: 69%) +- [x] Live API test: "Iran War" - 15 markets, no regression (domain expansion found "Geopolitics", "Middle East") +- [ ] Manual test: `/last30daysCROSS "Arizona Basketball"` - end-to-end with LLM synthesis +- [ ] Manual test: `/last30daysCROSS "Duke Basketball"` - verify NCAA Tournament markets appear for another team + +## Acceptance Criteria + +- [x] `_expand_queries()` searches ALL individual words, not just the first +- [x] Query cap raised from 4 to 6 +- [x] Second-pass context expansion discovers domain-indicator terms from first-pass event tags +- [x] "Arizona Basketball" search finds NCAA Tournament Winner AND #1 Seed markets (via "NCAA" tag domain expansion) +- [x] "Iran War" search still returns 9+ markets (15 - no regression) +- [x] All existing tests pass + new query expansion tests pass (91 polymarket tests) +- [x] No hardcoded domain knowledge (uses event tags, not hardcoded terms) +- [x] Default depth increased to 3 pages per query +- [x] Default result cap increased to 15 +- [x] Neg-risk binary markets show team names instead of Yes/No (via question extraction) +- [x] Topic-matching team surfaced first in outcome display + +## Dependencies & Risks + +**No blockers.** Changes are internal to `polymarket.py` and don't affect other modules. + +**Risk: Second-pass adds latency.** 2 extra queries x 1 page each, at ~200ms per call = ~400ms additional latency. The queries run after first-pass completes (serial), so total Polymarket time goes from ~1.5s to ~2s. Acceptable since other sources (YouTube, X) take 10-30s. + +**Risk: Bigram extraction produces noise.** Common title patterns like "regular season" or "2025-2026" could become domain queries. Mitigation: filter filler words, require 2+ title appearances, and cap at 2 queries. A noisy domain query just returns irrelevant events that get scored low by the existing relevance ranker. + +**Risk: Individual word queries return unrelated events.** "Basketball" returns European basketball, "War" returns non-Iran conflicts. Mitigation: the outcome-aware scoring already handles this - events without topic-matching outcomes get low text_score (token overlap only) and sort to the bottom. + +**Risk: Rate limiting.** Adding 4-6 extra API calls per search. Gamma API allows 350 req/10s, so even aggressive searching stays well under limits. + +## Sources & References + +- Query expansion: `scripts/lib/polymarket.py:60` (`_expand_queries`) +- Search orchestration: `scripts/lib/polymarket.py:109` (`search_polymarket`) +- Event filtering: `scripts/lib/polymarket.py:302-306` +- Depth config: `scripts/lib/polymarket.py:20-31` +- Previous outcome-aware scoring plan: `docs/plans/2026-02-26-feat-polymarket-smarter-synthesis-plan.md` +- Live Gamma API test results from 2026-02-26 (documented above) diff --git a/scripts/lib/polymarket.py b/scripts/lib/polymarket.py index e2f4e04..ee9922a 100644 --- a/scripts/lib/polymarket.py +++ b/scripts/lib/polymarket.py @@ -19,15 +19,15 @@ GAMMA_SEARCH_URL = "https://gamma-api.polymarket.com/public-search" # Pages to fetch per query (API returns 5 events per page, limit param is a no-op) DEPTH_CONFIG = { "quick": 1, - "default": 2, - "deep": 3, + "default": 3, + "deep": 4, } # Max events to return after merge + dedup + re-ranking RESULT_CAP = { "quick": 5, - "default": 10, - "deep": 20, + "default": 15, + "deep": 25, } @@ -58,28 +58,29 @@ def _extract_core_subject(topic: str) -> str: def _expand_queries(topic: str) -> List[str]: - """Generate 2-4 search queries to cast a wider net. + """Generate search queries to cast a wider net. Strategy: - Always include the core subject - - Split multi-word topics into component searches + - Add ALL individual words as standalone searches (not just first) - Include the full topic if different from core - - Cap at 4 queries, dedupe + - Cap at 6 queries, dedupe """ core = _extract_core_subject(topic) queries = [core] - # Split multi-word topics into component searches + # Add ALL individual words as separate queries words = core.split() if len(words) >= 2: - # Try the first significant word alone (e.g., "Arizona" from "Arizona Basketball") - queries.append(words[0]) + for word in words: + if len(word) > 1: # skip single-char words + queries.append(word) # Add the full topic if different from core if topic.lower().strip() != core.lower(): queries.append(topic.strip()) - # Dedupe while preserving order, cap at 4 + # Dedupe while preserving order, cap at 6 seen = set() unique = [] for q in queries: @@ -87,7 +88,44 @@ def _expand_queries(topic: str) -> List[str]: if q_lower and q_lower not in seen: seen.add(q_lower) unique.append(q.strip()) - return unique[:4] + return unique[:6] + + +_GENERIC_TAGS = frozenset({"sports", "politics", "crypto", "science", "culture", "pop culture"}) + + +def _extract_domain_queries(topic: str, events: List[Dict]) -> List[str]: + """Extract domain-indicator search terms from first-pass event tags. + + Uses structured tag metadata from Gamma API events to discover broader + domain categories (e.g., 'NCAA CBB' from a Big 12 basketball event). + Falls back to frequent title bigrams if no useful tags exist. + """ + query_words = set(_extract_core_subject(topic).lower().split()) + + # Collect tag labels from all first-pass events, count occurrences + tag_counts: Dict[str, int] = {} + for event in events: + tags = event.get("tags") or [] + for tag in tags: + label = tag.get("label", "") if isinstance(tag, dict) else str(tag) + if not label: + continue + label_lower = label.lower() + # Skip generic category tags and tags matching existing queries + if label_lower in _GENERIC_TAGS: + continue + if label_lower in query_words: + continue + tag_counts[label] = tag_counts.get(label, 0) + 1 + + # Sort by frequency, take top 2 that appear in 2+ events + domain_queries = [ + label for label, count in sorted(tag_counts.items(), key=lambda x: -x[1]) + if count >= 2 + ][:2] + + return domain_queries def _search_single_query(query: str, page: int = 1) -> Dict[str, Any]: @@ -106,38 +144,13 @@ def _search_single_query(query: str, page: int = 1) -> Dict[str, Any]: return {"events": [], "error": str(e)} -def search_polymarket( - topic: str, - from_date: str, - to_date: str, - depth: str = "default", -) -> Dict[str, Any]: - """Search Polymarket via Gamma API with smart query expansion. - - Runs 2-4 expanded queries in parallel, merges and dedupes by event ID. - - Args: - topic: Search topic - from_date: Start date (YYYY-MM-DD) - used for activity filtering - to_date: End date (YYYY-MM-DD) - depth: 'quick', 'default', or 'deep' - - Returns: - Dict with 'events' list and optional 'error'. - """ - pages = DEPTH_CONFIG.get(depth, DEPTH_CONFIG["default"]) - cap = RESULT_CAP.get(depth, RESULT_CAP["default"]) - queries = _expand_queries(topic) - - _log(f"Searching for '{topic}' with queries: {queries} (pages={pages})") - - # Run all (query, page) combinations in parallel - all_events = {} # event_id -> (event_data, query_index) - errors = [] - +def _run_queries_parallel( + queries: List[str], pages: int, all_events: Dict, errors: List, start_idx: int = 0, +) -> None: + """Run (query, page) combinations in parallel, merging into all_events.""" with ThreadPoolExecutor(max_workers=min(8, len(queries) * pages)) as executor: futures = {} - for i, q in enumerate(queries): + for i, q in enumerate(queries, start=start_idx): for p in range(1, pages + 1): future = executor.submit(_search_single_query, q, p) futures[future] = i @@ -154,17 +167,59 @@ def search_polymarket( event_id = event.get("id", "") if not event_id: continue - # Keep the first occurrence (from highest-priority query) if event_id not in all_events: all_events[event_id] = (event, query_idx) elif query_idx < all_events[event_id][1]: - # Replace with higher-priority query result all_events[event_id] = (event, query_idx) except Exception as e: errors.append(str(e)) + +def search_polymarket( + topic: str, + from_date: str, + to_date: str, + depth: str = "default", +) -> Dict[str, Any]: + """Search Polymarket via Gamma API with two-pass query expansion. + + Pass 1: Run expanded queries in parallel, merge and dedupe by event ID. + Pass 2: Extract domain-indicator terms from first-pass titles, search those. + + Args: + topic: Search topic + from_date: Start date (YYYY-MM-DD) - used for activity filtering + to_date: End date (YYYY-MM-DD) + depth: 'quick', 'default', or 'deep' + + Returns: + Dict with 'events' list and optional 'error'. + """ + pages = DEPTH_CONFIG.get(depth, DEPTH_CONFIG["default"]) + cap = RESULT_CAP.get(depth, RESULT_CAP["default"]) + queries = _expand_queries(topic) + + _log(f"Searching for '{topic}' with queries: {queries} (pages={pages})") + + # Pass 1: run expanded queries in parallel + all_events: Dict[str, tuple] = {} + errors: List[str] = [] + _run_queries_parallel(queries, pages, all_events, errors) + + # Pass 2: extract domain-indicator terms from first-pass titles and search + first_pass_events = [ev for ev, _ in all_events.values()] + domain_queries = _extract_domain_queries(topic, first_pass_events) + # Filter out queries we already ran + seen_queries = {q.lower() for q in queries} + domain_queries = [dq for dq in domain_queries if dq.lower() not in seen_queries] + + if domain_queries: + _log(f"Domain expansion queries: {domain_queries}") + _run_queries_parallel(domain_queries, 1, all_events, errors, start_idx=len(queries)) + merged_events = [ev for ev, _ in sorted(all_events.values(), key=lambda x: x[1])] - _log(f"Found {len(merged_events)} unique events across {len(queries)} queries x {pages} pages") + total_queries = len(queries) + len(domain_queries) + _log(f"Found {len(merged_events)} unique events across {total_queries} queries") result = {"events": merged_events, "_cap": cap} if errors and not merged_events: @@ -233,6 +288,24 @@ def _parse_outcome_prices(market: Dict[str, Any]) -> List[tuple]: return result +def _shorten_question(question: str) -> str: + """Extract a short display name from a market question. + + 'Will Arizona win the 2026 NCAA Tournament?' -> 'Arizona' + 'Will Duke be a number 1 seed in the 2026 NCAA...' -> 'Duke' + """ + q = question.strip().rstrip("?") + # Common patterns: "Will X win/be/...", "X wins/loses..." + m = re.match(r"^Will\s+(.+?)\s+(?:win|be|make|reach|have|lose|qualify|advance|strike|agree|pass|sign|get|become|remain|stay|leave|survive|next)\b", q, re.IGNORECASE) + if m: + return m.group(1).strip() + m = re.match(r"^Will\s+(.+?)\s+", q, re.IGNORECASE) + if m and len(m.group(1).split()) <= 4: + return m.group(1).strip() + # Fallback: truncate + return question[:40] if len(question) > 40 else question + + def _compute_text_similarity(topic: str, title: str, outcomes: List[str] = None) -> float: """Score how well the event title (or outcome names) match the search topic. @@ -341,14 +414,39 @@ def parse_polymarket_response(response: Dict[str, Any], topic: str = "") -> List # Collect outcome names from ALL active markets (not just top) for similarity scoring # Filter to outcomes with price > 1% to avoid noise + # Also extract subjects from market questions for neg-risk events (outcomes are Yes/No) all_outcome_names = [] for m in active_markets: for name, price in _parse_outcome_prices(m): if price > 0.01 and name not in all_outcome_names: all_outcome_names.append(name) + # For neg-risk binary markets (Yes/No outcomes), the team/entity name + # lives in the question, e.g., "Will Arizona win the NCAA Tournament?" + question = m.get("question", "") + if question and question != title: + all_outcome_names.append(question) - # Parse outcome prices from top market + # Parse outcome prices - for multi-market events with Yes/No binary + # sub-markets, synthesize from market questions to show actual + # team/entity probabilities instead of a single market's Yes/No outcome_prices = _parse_outcome_prices(top_market) + top_outcomes_are_binary = ( + len(outcome_prices) == 2 + and {n.lower() for n, _ in outcome_prices} == {"yes", "no"} + ) + if top_outcomes_are_binary and len(active_markets) > 1: + synth_outcomes = [] + for m in active_markets: + q = m.get("question", "") + if not q: + continue + pairs = _parse_outcome_prices(m) + yes_price = next((p for name, p in pairs if name.lower() == "yes"), None) + if yes_price is not None and yes_price > 0.005: + synth_outcomes.append((q, yes_price)) + if synth_outcomes: + synth_outcomes.sort(key=lambda x: x[1], reverse=True) + outcome_prices = [(_shorten_question(q), p) for q, p in synth_outcomes] # Format price movement price_movement = _format_price_movement(top_market) @@ -412,11 +510,15 @@ def parse_polymarket_response(response: Dict[str, Any], topic: str = "") -> List # Surface the topic-matching outcome to the front before truncating if topic and outcome_prices: core = _extract_core_subject(topic).lower() + core_tokens = set(core.split()) reordered = [] rest = [] for pair in outcome_prices: name_lower = pair[0].lower() - if core in name_lower or name_lower in core: + # Match if full core is substring, or name is substring of core, + # or any core token appears in the name (handles long question strings) + if (core in name_lower or name_lower in core + or any(tok in name_lower for tok in core_tokens if len(tok) > 2)): reordered.append(pair) else: rest.append(pair) diff --git a/tests/test_polymarket.py b/tests/test_polymarket.py index 00ba267..8298740 100644 --- a/tests/test_polymarket.py +++ b/tests/test_polymarket.py @@ -45,7 +45,8 @@ class TestExpandQueries(unittest.TestCase): queries = polymarket._expand_queries("Arizona Basketball") self.assertIn("Arizona Basketball", queries) self.assertIn("Arizona", queries) - self.assertEqual(len(queries), 2) + self.assertIn("Basketball", queries) + self.assertEqual(len(queries), 3) def test_with_prefix_stripped(self): queries = polymarket._expand_queries("last 7 days Iran") @@ -58,9 +59,89 @@ class TestExpandQueries(unittest.TestCase): # Should not have duplicates self.assertEqual(len(queries), len(set(q.lower() for q in queries))) - def test_max_4_queries(self): + def test_max_6_queries(self): queries = polymarket._expand_queries("some really long topic with many words") - self.assertLessEqual(len(queries), 4) + self.assertLessEqual(len(queries), 6) + + def test_all_words_included(self): + queries = polymarket._expand_queries("Iran War") + self.assertIn("Iran War", queries) + self.assertIn("Iran", queries) + self.assertIn("War", queries) + self.assertEqual(len(queries), 3) + + def test_short_words_excluded(self): + """Single-char words should not become standalone queries.""" + queries = polymarket._expand_queries("A new idea") + # "A" is single char, should be excluded + self.assertNotIn("A", queries) + self.assertIn("new", queries) + self.assertIn("idea", queries) + + +class TestExtractDomainQueries(unittest.TestCase): + def _make_tag(self, label): + return {"id": "1", "label": label, "slug": label.lower().replace(" ", "-")} + + def test_finds_frequent_tags(self): + tag_ncaa = self._make_tag("NCAA CBB") + tag_sport = self._make_tag("Sports") + tag_bball = self._make_tag("Basketball") + events = [ + {"title": "SEC Champion", "tags": [tag_ncaa, tag_sport, tag_bball]}, + {"title": "ACC Champion", "tags": [tag_ncaa, tag_sport, tag_bball]}, + {"title": "Big 12 Champion", "tags": [tag_ncaa, tag_sport, tag_bball]}, + ] + result = polymarket._extract_domain_queries("Arizona Basketball", events) + self.assertIn("NCAA CBB", result) + + def test_skips_generic_tags(self): + tag_sport = self._make_tag("Sports") + events = [ + {"title": "Event 1", "tags": [tag_sport]}, + {"title": "Event 2", "tags": [tag_sport]}, + {"title": "Event 3", "tags": [tag_sport]}, + ] + result = polymarket._extract_domain_queries("test topic", events) + self.assertNotIn("Sports", result) + + def test_skips_topic_word_tags(self): + tag_bball = self._make_tag("Basketball") + events = [ + {"title": "Event 1", "tags": [tag_bball]}, + {"title": "Event 2", "tags": [tag_bball]}, + ] + result = polymarket._extract_domain_queries("Arizona Basketball", events) + self.assertNotIn("Basketball", result) + + def test_requires_minimum_frequency(self): + tag_a = self._make_tag("Unique Tag A") + tag_b = self._make_tag("Unique Tag B") + events = [ + {"title": "Event 1", "tags": [tag_a]}, + {"title": "Event 2", "tags": [tag_b]}, + ] + result = polymarket._extract_domain_queries("test topic", events) + self.assertEqual(result, []) + + def test_caps_at_two(self): + tags = [self._make_tag(f"Tag {i}") for i in range(5)] + events = [{"title": f"Event {i}", "tags": tags} for i in range(3)] + result = polymarket._extract_domain_queries("test topic", events) + self.assertLessEqual(len(result), 2) + + def test_empty_events(self): + result = polymarket._extract_domain_queries("Arizona Basketball", []) + self.assertEqual(result, []) + + def test_events_without_tags(self): + events = [ + {"title": "Event 1"}, + {"title": "Event 2", "tags": None}, + {"title": "Event 3", "tags": []}, + ] + result = polymarket._extract_domain_queries("test topic", events) + self.assertEqual(result, []) class TestFormatPriceMovement(unittest.TestCase): @@ -462,19 +543,19 @@ class TestDepthConfig(unittest.TestCase): self.assertEqual(polymarket.DEPTH_CONFIG["quick"], 1) def test_default_pages(self): - self.assertEqual(polymarket.DEPTH_CONFIG["default"], 2) + self.assertEqual(polymarket.DEPTH_CONFIG["default"], 3) def test_deep_pages(self): - self.assertEqual(polymarket.DEPTH_CONFIG["deep"], 3) + self.assertEqual(polymarket.DEPTH_CONFIG["deep"], 4) def test_result_cap_quick(self): self.assertEqual(polymarket.RESULT_CAP["quick"], 5) def test_result_cap_default(self): - self.assertEqual(polymarket.RESULT_CAP["default"], 10) + self.assertEqual(polymarket.RESULT_CAP["default"], 15) def test_result_cap_deep(self): - self.assertEqual(polymarket.RESULT_CAP["deep"], 20) + self.assertEqual(polymarket.RESULT_CAP["deep"], 25) class TestTextSimilarity(unittest.TestCase):