Fossy
fossy
Back vllm-project/semantic-router
semantic-router

semantic-router

Alternative to Portkey.ai

A programmable Mixture-of-Models router for heterogeneous LLM inference

6.0k 989
Apache-2.0
 last commit 2026-08-30
ai-gatewayguardrailsinferencekubernetesllmllmroutermixture-of-modelspytorchsemantic-routertransformer
Website Source
Share:

About

A programmable Mixture-of-Models router for heterogeneous LLM inference

semantic-router website preview

Languages

Go48.3%Python25.3%TypeScript12.3%Rust6.6%CSS2.6%Shell1.4%ASL1%TeX0.8%Makefile0.7%JavaScript0.3%C++0.2%HIP0.1%SCSS0.1%Svelte0.1%Dockerfile0.1%BibTeX Style0.1%C0%MDX0%GLSL0%HTML0%CMake0%Go Template0%

Contributors30

Xunzhuo595rootfs204theohsiung951fanwang68samzong67wilsonwu58subin958yuluo-yx56FAUST-BENCHOU51WUKUNTAI-021151drivebyer49AayushSaini10147yossiovadia47cryo-zd44+16

Free alternative to Portkey.ai

semantic-router is a free, open-source alternative to Portkey.ai. Here's why:

  • 🤖 Smart LLM Routing
  • 🛡️ Custom AI Guardrails
  • 🔄 Multi-Model Inference
  • ⚙️ Kubernetes Integration
  • ⚡ Optimized LLM Performance
See all open-source alternatives to Portkey.ai
Comments Theme
Platforms
Kubernetes
Install
Build from source
Keywords
ai-gateway
guardrails
inference
kubernetes
llm
llmrouter
mixture-of-models
pytorch
semantic-router
transformer
vllm
slug: semantic-router

Sponsored

Ad space
300 × 250 (IAB)
SponsorSponsorSponsorSponsor
Ad space
300 × 250 (IAB)
Ad space
300 × 250 (IAB)