Optimize model selection for cost-efficiency on structured extraction
The task profile is search tool invocation + JSON extraction — not reasoning or creative work. Mini models handle this equally well at 3-5x lower cost per call. OpenAI changes: - Rename is_mainline_openai_model -> is_search_capable_model - Include mini variants (gpt-5-mini, gpt-4.1-mini) in candidate pool - Exclude gpt-4o-mini (no domain filtering) and nano (no web_search) - select_openai_model() now prefers mini within newest generation - OPENAI_FALLBACK_MODELS: gpt-5-mini first, mainline as last resort - MODEL_FALLBACK_ORDER: same mini-first ordering xAI changes: - Switch alias from grok-4-1-fast (reasoning) to grok-4-1-fast-non-reasoning — same token price, faster response, no wasted reasoning tokens for structured extraction Cost per Reddit search call: ~$0.015 (gpt-5-mini) vs ~$0.044 (gpt-4.1)
This commit is contained in:
@@ -64,13 +64,18 @@ class TestIsModelAccessError(unittest.TestCase):
|
||||
class TestModelFallbackOrder(unittest.TestCase):
|
||||
"""Tests for MODEL_FALLBACK_ORDER constant."""
|
||||
|
||||
def test_contains_gpt4o(self):
|
||||
"""Fallback list should include gpt-4o."""
|
||||
def test_mini_first(self):
|
||||
"""Mini models should come first (cost-efficient for structured extraction)."""
|
||||
self.assertEqual(MODEL_FALLBACK_ORDER[0], "gpt-5-mini")
|
||||
|
||||
def test_contains_mainline_fallbacks(self):
|
||||
"""Fallback list should include mainline models as last resort."""
|
||||
self.assertIn("gpt-4.1", MODEL_FALLBACK_ORDER)
|
||||
self.assertIn("gpt-4o", MODEL_FALLBACK_ORDER)
|
||||
|
||||
def test_gpt41_is_first(self):
|
||||
"""gpt-4.1 should be the first fallback option."""
|
||||
self.assertEqual(MODEL_FALLBACK_ORDER[0], "gpt-4.1")
|
||||
def test_no_gpt4o_mini(self):
|
||||
"""gpt-4o-mini should NOT be in fallback (no domain filtering support)."""
|
||||
self.assertNotIn("gpt-4o-mini", MODEL_FALLBACK_ORDER)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Reference in New Issue
Block a user