Press Space to continue
Context length stagnation despite model improvements
Press Space to continue
It's interesting how, for all of the huge model improvements we've seen over the past two year, the one thing that hasn't improved much at all is context length We've been stuck in the 200,000 up to 1m range for quite a long time now
open on xViews and clicks count once per visitor per day. Clicks are opens of this post on X.