bw24

From-scratch Rust+CUDA inference engine, bit-exact by construction — NVFP4, MoE, MTP speculative decoding, tuned against measured limits of one RTX 5090 Laptop (sm_120a).

258 30

MIT

last commit 2026-07-09

blackwell cuda gguf gpu-kernels llama-cpp llm-inference moe nvfp4 rust speculative-decoding

Website Source

About

From-scratch Rust+CUDA inference engine, bit-exact by construction — NVFP4, MoE, MTP speculative decoding, tuned against measured limits of one RTX 5090 Laptop (sm_120a).