Fossy
fossy
Back vllm-project/vllm
vllm

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

90k 21k107 HN points
Apache-2.0
 last commit 2026-08-30
amdblackwellcudadeepseekdeepseek-v3gptgpt-ossinferencekimillama
Website Source
Share:

About

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm website preview

Languages

Python84%Rust6.4%Cuda4.7%C++3.4%Shell1.2%CMake0.3%Dockerfile0.1%C0%HCL0%Jinja0%

Contributors30

DarkLight1337910WoosukKwon840mgoin608hmellor563njhill477youkaichao471Isotr0py436yewentao256417AndreasKaratzas341jeejeelee328LucasWilkinson231NickLucche225ywang96212simon-mo196+16

No features listed.

Comments Theme
Install
pip
Build from source
Keywords
amd
blackwell
cuda
deepseek
deepseek-v3
gpt
gpt-oss
inference
kimi
llama
llm
llm-serving
model-serving
moe
openai
pytorch
qwen
qwen3
tpu
transformer
slug: vllm

Sponsored

Ad space
300 × 250 (IAB)
SponsorSponsorSponsorSponsor
Ad space
300 × 250 (IAB)
Ad space
300 × 250 (IAB)