llama.cpp
LLM inference in C/C++ with broad hardware support and aggressive quantization
llama.cpp is a C/C++ inference engine for large language models, designed to run efficiently on CPUs, GPUs, and Apple Silicon. It pioneered the GGUF quantization format and the broader local-LLM tooling space. Supports most popular open-source models including Llama, Mistral, Qwen, Gemma, and Phi. Used as the underlying engine by Ollama, LM Studio, GPT4All, and many other local-LLM tools. Maintained by the ggml-org community.
Pricing: Free
Choosing an inference engine right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
Choosing an inference engine right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
llama.cpp Alternatives
Explore 49 products in the Frameworks & Stacks category. View all llama.cpp alternatives.
vLLM
High-throughput LLM inference engine with PagedAttention for efficient GPU memory usage
Xinference
Open-source inference engine for 300+ open models, plus a hosted API and managed GPUs served from Australia
Jan
Open-source desktop app for running LLMs locally with a clean GUI
GPT4All
Desktop app and Python SDK for running open-source LLMs locally on any device
Work on llama.cpp? Feature it at the top of Frameworks & Stacks.
Is your product missing?