Papers and official implementations
Follow the research and implementation sources that explain the architecture terms used in the Kimi K3 launch materials.
This short video summarizes the page: the source set, the decision boundary and the practical next action.
The Kimi Linear paper introduces KDA and reports up to 75% KV-cache reduction and up to 6x decoding throughput at 1M context in the paper setting. The K3 page points to KDA as part of the model architecture.
The Attention Residuals paper and repository describe cross-layer attention for reducing information dilution in deep PreNorm models, plus Block AttnRes for large-scale training efficiency.
FlashKDA is an official MoonshotAI kernel repository. It is useful for understanding engineering direction, and it becomes more important after official K3 weights and inference notes can be verified.
The map links KDA, Attention Residuals and FlashKDA to the supporting paper or official implementation source.
Cards are generated from the clauxel/kimik3 source ledger and keep source type, priority and status visible.
Kimi Linear paper introducing Kimi Delta Attention, the architecture thread the K3 launch page points back to.
Attention Residuals paper describing cross-layer attention as an alternative to fixed residual accumulation.
Official repository with paper, pseudocode and evaluation summaries for Attention Residuals.
Official FlashKDA kernel implementation that links the KDA research line to inference engineering.
Moonshot Hugging Face collection for Kimi Linear checkpoints and paper references.
Official MoonshotAI GitHub organization, useful for tracking Kimi Code, Kimi CLI, K2 lines, Attention Residuals and FlashKDA.