spinf-serve

Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0

19 0

Apache-2.0

last commit 2026-10-29

batch-inference classification content-moderation decision-model extraction fp8 gemma inference-engine llm openai-compatible

Website Source

About

Bulk classification and decision inference on one GPU: the content is read once, every question about it is scored in the same pass. OpenAI-compatible, Apache 2.0