Docker Model Runner
Run and serve local LLMs from Docker Hub or Hugging Face with docker model commands and an OpenAI-compatible API
Docker Model Runner pulls, runs and serves AI models with Docker tooling. Models are packaged as OCI artifacts (GGUF or Safetensors) and come from Docker Hub, any OCI registry or Hugging Face.
The default engine is llama.cpp; vLLM is available on NVIDIA GPUs. Hardware covers Apple Silicon, NVIDIA CUDA, AMD ROCm and Vulkan, and it exposes OpenAI- and Ollama-compatible APIs.
It runs inside Docker Desktop, in Docker Engine on Linux, or as a standalone binary. Apache-2.0. Docker Desktop itself needs a paid plan at larger companies.
Pricing: Free
Choosing a local LLM runtime right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
Choosing a local LLM runtime right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
Docker Model Runner Alternatives
Explore 49 products in the Frameworks & Stacks category. View all Docker Model Runner alternatives.
GPT4All
Desktop app and Python SDK for running open-source LLMs locally on any device
LocalAI
Open-source, self-hosted OpenAI-compatible API for running models on your own hardware
Jan
Open-source desktop app for running LLMs locally with a clean GUI
Foundry Local
Microsoft's on-device AI runtime and SDK that runs ONNX models on the user's NPU, GPU or CPU
Lemonade Server
Local AI server for LLMs, speech and images, tuned for AMD Ryzen AI NPUs and Radeon GPUs
TextGen
Formerly text-generation-webui: open-source desktop app and web UI for local LLMs with an OpenAI-compatible API
Work on Docker Model Runner? Feature it at the top of Frameworks & Stacks.
Is your product missing?