llm-inference

Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM