Add local search quality evaluation harness
Add an optional local evaluator that compares a baseline revision against a candidate checkout, computes deterministic stability metrics, and can call Gemini for judged ranking metrics when configured. The harness isolates child runs with a temporary HOME and a node-free PATH so historical revisions cannot trigger Bird browser-cookie auth during evaluation. Validation: uv run python -m unittest and local smoke/full deterministic eval runs.
This commit is contained in:
@@ -243,10 +243,12 @@ def get_config() -> Dict[str, Any]:
|
||||
|
||||
keys = [
|
||||
('XAI_API_KEY', None),
|
||||
('GEMINI_API_KEY', None),
|
||||
('OPENROUTER_API_KEY', None),
|
||||
('PARALLEL_API_KEY', None),
|
||||
('BRAVE_API_KEY', None),
|
||||
('XIAOHONGSHU_API_BASE', None),
|
||||
('GEMINI_MODEL', None),
|
||||
('OPENAI_MODEL_POLICY', 'auto'),
|
||||
('OPENAI_MODEL_PIN', None),
|
||||
('XAI_MODEL_POLICY', 'latest'),
|
||||
|
||||
Reference in New Issue
Block a user