llmops

🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTelemetry, LangChain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

The platform for LLM evaluations and AI agent testing

See what your coding agents did and what it cost. Breaks each task down into work steps — tools used, files changed, tests run, time and tokens spent. Local-first dashboard for Claude Code, Codex, OpenCode, and more. No login, no telemetry.

Self-hosted LLM router. Cost effective, deterministic, and fast. Secure and private by default.

A token-spend profiler and cost-regression gate for AI agents.

Deterministic snapshot testing for AI agents