I wonder if this could be a good stepping stone to write a local prompt router to optimise what model get what prompt. I.e. if the prompt is just a lookup, send it to haiku, if it's reasoning, send it to opus and if it's implementation send it to sonnet.
I tried it out, but with kev instead of this python script. The issue is that mid session swapping invalidates the cache, which drives cost quite a lot. Ended up loosing money when comparing prompts in most of my transcripts.
If you're not behind a walled garden like i am (vertex), you could probably experiment with routing on effort level instead. Anthropic supports it, but vertex has not added that feature yet.
cupofjoakim · · focus · HN ↗
v18a · · focus · HN ↗
cupofjoakim · · focus · HN ↗
If you're not behind a walled garden like i am (vertex), you could probably experiment with routing on effort level instead. Anthropic supports it, but vertex has not added that feature yet.