fix(rerank): ground entity-miss demotion on head token, not full phrase

The entity-grounding demotion required the full multi-word primary
entity as a contiguous substring, so on-entity items missing a trailing
search descriptor were buried: a 323-pt HN thread "Stripe is friendly
to 'friendly fraud'" scored 0 on a "Stripe payments" query. New
_entity_grounded helper keys on the brand head token; items that never
name the brand still miss it and stay demoted. reddit_keyless
_slot_priority, which had re-implemented the old check while claiming
to mirror rerank's signal, now calls the shared helper so the two
paths cannot diverge.
This commit is contained in:
Trevin Chow
2026-06-09 16:24:16 -07:00
parent fd0e47d99f
commit 6a92f63a56
4 changed files with 71 additions and 22 deletions
@@ -170,9 +170,11 @@ def _slot_priority(topic: str, posts: List[Dict[str, Any]]) -> List[Dict[str, An
posts that rerank later demotes as entity misses starves the on-topic
posts the user actually sees (2026-06-06 "OpenClaw vs Hermes" run:
2,000+ upvote Gemma/GPU threads took every slot, then were demoted to
zero). Mirror rerank's demotion signal the topic's stripped primary
entity contained in the post text — so slots go to posts likely to
survive final ranking. Falls back to token-overlap relevance when the
zero). Mirror rerank's demotion signal via the shared `_entity_grounded`
check (head token of the topic's stripped primary entity present in the
post text) so slots go to posts likely to survive final ranking — keying
on the same head token keeps the two paths from diverging. Falls back to
token-overlap relevance when the
topic yields no usable primary entity. Within each tier the incoming
(score-first) order is preserved. Never raises; on any failure the
incoming order is returned unchanged.
@@ -186,7 +188,7 @@ def _slot_priority(topic: str, posts: List[Dict[str, Any]]) -> List[Dict[str, An
entity = rerank._primary_entity(topic).lower()
if entity:
def _matches(post: Dict[str, Any]) -> bool:
return entity in _post_text(post).lower()
return rerank._entity_grounded(_post_text(post).lower(), entity)
else:
prepared = relevance.PreparedQuery(topic)