The entity-grounding demotion required the full multi-word primary
entity as a contiguous substring, so on-entity items missing a trailing
search descriptor were buried: a 323-pt HN thread "Stripe is friendly
to 'friendly fraud'" scored 0 on a "Stripe payments" query. New
_entity_grounded helper keys on the brand head token; items that never
name the brand still miss it and stay demoted. reddit_keyless
_slot_priority, which had re-implemented the old check while claiming
to mirror rerank's signal, now calls the shared helper so the two
paths cannot diverge.
Add a pytest-discovered tests/conftest.py for the last30days scripts path and
remove duplicate per-file sys.path.insert boilerplate from tests.
Normalize affected imports to rely on the shared scripts path and remove the
now-unneeded E402 suppressions.
The Gemini 3.1 Flash Lite preview model is being discontinued on
May 25, 2026. Per Google's GA announcement, the underlying model
architecture is identical and only the model identifier needs to
be updated from `gemini-3.1-flash-lite-preview` to
`gemini-3.1-flash-lite`.
Also relaxes the `_require_gemini_31_preview` guard to accept any
`gemini-3.1-*` identifier (renamed to `_require_gemini_31`), so the
GA name and the still-preview `gemini-3.1-pro-preview` both pass.
PR #285's entity grounding checked only title + snippet. That missed:
- YouTube videos where the entity is mentioned in transcript but not
in title (false demotion of on-topic content)
- Reddit posts where the entity is in top comments but not in title
(false demotion of on-topic discussion)
And it also wasn't strong enough to reliably demote items like the
2026-04-19 Nate Herk "Managed Agents" video - which had no Hermes
anywhere - because the -25 penalty on rerank_score composed to only
-15 on final_score via the 0.60 weight, and engagement bonus partially
offset that.
Two fixes:
1. _candidate_haystack() now joins title + snippet +
metadata[transcript_snippet] + metadata[transcript_highlights] +
metadata[top_comments][*].excerpt/text + metadata[comment_insights].
Catches entity mentions wherever they actually live. Guarded with
isinstance checks so malformed metadata doesn't raise.
2. ENTITY_MISS_FINAL_PENALTY (20.0) applied directly in _final_score
when candidate.explanation contains "entity-miss". This lands the
full penalty weight on the composite signal that cluster-scoring
consumes, instead of being diluted by the rerank_score weight.
Combined effect: entity-miss gap grows from ~15 to ~35 points.
Tests: 8 new scenarios covering transcript match, transcript highlight
match, top-comment match, comment-insight match, empty-text skip,
no-primary-entity no-op, and the dual-penalty composition check.
The 2026-04-19 Hermes Agent Use Cases run had a Nate Herk YouTube video
titled "I Tested Claude's New Managed Agents" score 51 and rank #2
with zero Hermes content. The reranker had intent-specific scoring hints
but no entity-grounding check, so topic-vicinity matches (one offhand
OpenClaw mention) drifted to the top.
Add _primary_entity(topic) that strips intent-modifier suffixes ("use
cases", "workflows", etc.) so "Hermes Agent use cases" yields
primary_entity="Hermes Agent". Pass the entity through to both the LLM
and fallback scoring paths.
Fallback path: if primary_entity is not found (case-insensitive) in
title + snippet, subtract ENTITY_MISS_PENALTY (25 pts). Skip the
demotion for candidates with no text at all (image-only TikToks etc.)
to avoid false negatives on thin-text sources.
LLM path: add a "Primary entity grounding" hint to _build_prompt when
primary_entity is non-empty. Instructs the LLM to score candidates
without the entity at <=30.
Tests: 24 rerank tests pass, including 8 new entity-grounding tests.
Trevin Chow