Press Space to continue
This open-source inference engine enables significantly faster
Press Space to continue
UC Berkeley + MIT researchers open sourced a new inference engine and so far IT’S CRAZY: > runs DeepSeek-V4-Flash 284B at 25 tok/s with a RTX 5090 system > 1.46x faster than llama.cpp on the same test. Ollama can’t even serve it > up to 3× faster than Ollama on the same Qwen3.6-35B bench
open on x · 2 viewsViews and clicks count once per visitor per day. Clicks are opens of this post on X.