llama-cpp

Search your computer by what's in your files: photos, PDFs, documents, recordings, videos and code. On-device on Mac, Windows and Linux, built on Google DeepMind's EmbeddingGemma 2.

Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable

Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.

Catches silent CPU fallback and mislabeled tok/s in local LLMs. llama.cpp and ollama, one file, no deps.

A unified, unmanaged multimodal runtime. Zero-allocation native AI inference for Java 22+.