cuda

The open-source AI voice studio. Clone, dictate, create.

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.