kimi3

Independent from-scratch implementation of the Kimi K3 architecture (arXiv:2607.24653v1): KDA, NoPE, latent-space MoE, and the systems co-designs. Table 1 reproduced to 0.09%.

9 110 HN points

AGPL-3.0

last commit 2026-08-02

language-model linear-attention machine-learning mixture-of-experts pytorch transformers

Source

About

Independent from-scratch implementation of the Kimi K3 architecture (arXiv:2607.24653v1): KDA, NoPE, latent-space MoE, and the systems co-designs. Table 1 reproduced to 0.09%.