Xinference
Open-source inference engine for 300+ open models, plus a hosted API and managed GPUs served from Australia
Xinference is an Apache-2.0 inference engine for serving open models behind an OpenAI-compatible API, developed in the open at xorbitsai/inference (about 9,600 GitHub stars). The Community edition runs on NVIDIA GPUs in a single cluster.
Xinference Holdings, a Singapore company, also sells a hosted Model API and dedicated H100, H200, B200 and A100 instances served from Australia. An Enterprise edition adds AMD, Intel and CPU support plus multi-cluster scaling.
The engine is free; hosted and Enterprise pricing goes through sales.
Pricing: Free (open source); hosted and Enterprise via sales
Xinference Alternatives
Explore 42 products in the Frameworks & Stacks category. View all Xinference alternatives.
llama.cpp
LLM inference in C/C++ with broad hardware support and aggressive quantization
vLLM
High-throughput LLM inference engine with PagedAttention for efficient GPU memory usage
Dify
Easily build and operate generative AI applications. Create Assistants API and GPTs based on any LLMs.
LangChain
LangChain gives developers a framework to construct LLM‑powered apps easily.
Also listed in
Work on Xinference? Feature it at the top of Frameworks & Stacks.
Is your product missing?