quantization

Accessible large language models via k-bit quantization for PyTorch.

From teacher to tiles — a from-scratch LLM distillation & serving engine: custom Triton/CUDA kernels, FSDP distillation, paged-KV continuous batching, speculative decoding, a Rust gateway, a JAX oracle, and interpretability tooling.

An optimized, high-performance NF4 (NormalFloat 4-bit) dequantization Triton GPU kernel achieving up to 1.41x speedup over bitsandbytes.