Press Space to continue
Caching makes auto-router startups less viable
Press Space to continue
Every time a VC sends me an auto-router company to diligence, I send something like this back. If caching wasn't a thing, auto-routing would make much more sense. But... it is a thing.
Teknium@TekniumAug 25
Just FYI. If you are switching models in a session all the time - you are doing it wrong. Every time you switch, your entire prompt cache is invalidated on the new model you switch to, and you have to repay the full input tokens price for all of it. Stop doing this unless those models are free. This is not a hermes thing - this is a fundamentals of inference thing.
open on x · 1 viewViews and clicks count once per visitor per day. Clicks are opens of this post on X.