Xinference
Open-source inference engine for 300+ open models, plus a hosted API and managed GPUs served from Australia
Xinference is an Apache-2.0 inference engine for serving open models behind an OpenAI-compatible API, developed in the open at xorbitsai/inference (about 9,600 GitHub stars). The Community edition runs on NVIDIA GPUs in a single cluster.
Xinference Holdings, a Singapore company, also sells a hosted Model API and dedicated H100, H200, B200 and A100 instances served from Australia. An Enterprise edition adds AMD, Intel and CPU support plus multi-cluster scaling.
The engine is free; hosted and Enterprise pricing goes through sales.
Pricing: Free (open source); hosted and Enterprise via sales
Choosing an inference engine right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
Choosing an inference engine right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
Xinference Alternatives
Explore 49 products in the Frameworks & Stacks category. View all Xinference alternatives.
vLLM
High-throughput LLM inference engine with PagedAttention for efficient GPU memory usage
llama.cpp
LLM inference in C/C++ with broad hardware support and aggressive quantization
llamafile
Single-file executables that run LLMs locally on Linux, macOS, Windows and BSD with an OpenAI-compatible server
KoboldCpp
Single-binary llama.cpp runtime with an OpenAI-compatible API, web UI, image generation and speech
RamaLama
Run and serve local LLMs in GPU-matched OCI containers with Podman or Docker
Also listed in
Work on Xinference? Feature it at the top of Frameworks & Stacks.
Is your product missing?