Looks like it’s being integrated into llama.cpp
https://github.com/ggml-org/llama.cpp/issues/25096
faster speeds for us!
Deepseek v4 is already exceptional value, with alibaba hosted version smarter than original release. Mimo 2.5 pro, and minimax m3, are only models… I can access for free testing that pass my benchmark.
Ai analysis has “cost to run tests” data, which also put mimo 2.5, not pro, on frontier curve. But deepseek 4.1, 4.0 silently improved served from main servers today, is going to blow everything away especially with this speculative decoding tech added. 85% faster means 85% cheaper, even if these techs often degrade performance.
OpenAI made a similar announcement recently too. Curious about the distillation stuff Alibaba is accused of.
Breakthrough hardly. Amazing how the article specifies cloud inference while the OP mentions local
Similar or slightly better than MTP using the same tech
It is much faster than MTP (around 2x). Also if you have read their paper which contains lots of really cool ideas, they clearly documented their limitations and did a good comparison with existing methods. You can run it easily on a strix halo (albeit quantized) and the peformance is fast given how quantized it is.
At least they published their research and went into great length for their novel discoveries, much bettter than any other “frontier” lab did. Them, GLM and Kimi are doing great, but deepseek hands down have the most cool and fun innovative papers in this field.
“Novel” and “groundbreaking” comment. You’ve made your 5 cents
people are saying 60-80% speed up?
Deepseek reports this number
