Xinference
Open-source inference engine for 300+ open models, plus a hosted API and managed GPUs served from Australia
Xinference is an Apache-2.0 inference engine for serving open models behind an OpenAI-compatible API, developed in the open at xorbitsai/inference (about 9,600 GitHub stars). The Community edition runs on NVIDIA GPUs in a single cluster.
Xinference Holdings, a Singapore company, also sells a hosted Model API and dedicated H100, H200, B200 and A100 instances served from Australia. An Enterprise edition adds AMD, Intel and CPU support plus multi-cluster scaling.
The engine is free; hosted and Enterprise pricing goes through sales.
Pricing: Free (open source); hosted and Enterprise via sales
Choosing an inference engine right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
Choosing an inference engine right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
Xinference Alternatives
Explore 157 products in the Inference APIs category. View all Xinference alternatives.
vLLM
High-throughput LLM inference engine with PagedAttention for efficient GPU memory usage
Project Zero
CPU-only LLM inference engine in C with no runtime dependencies
SGLang
High-performance open-source serving framework for LLMs and multimodal models
OpenAI Decisions API
Typed answers with probabilities for yes/no checks, fixed choices and rubric scores
Microsoft-Decision-1
Microsoft's decision model on Foundry: typed scores and yes/no answers with probabilities instead of text
Also listed in
Work on Xinference? Feature it at the top of Inference APIs.
Is your product missing?