KoboldCpp
Single-binary llama.cpp runtime with an OpenAI-compatible API, web UI, image generation and speech
KoboldCpp is a single-binary local runtime for GGUF models, built on llama.cpp. Prebuilt releases cover Windows, Linux and Apple Silicon macOS, with CUDA and Vulkan GPU backends; ROCm needs a source build.
It serves the KoboldAI Lite web UI and an API on port 5001, with OpenAI-compatible, Anthropic, Ollama and KoboldAI endpoints. The same process can also run image generation, Whisper speech-to-text and several TTS models. AGPL-3.0 licensed.
Pricing: Free
Choosing a local LLM runtime right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
Choosing a local LLM runtime right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
KoboldCpp Alternatives
Explore 49 products in the Frameworks & Stacks category. View all KoboldCpp alternatives.
GPT4All
Desktop app and Python SDK for running open-source LLMs locally on any device
LocalAI
Open-source, self-hosted OpenAI-compatible API for running models on your own hardware
Jan
Open-source desktop app for running LLMs locally with a clean GUI
RamaLama
Run and serve local LLMs in GPU-matched OCI containers with Podman or Docker
Docker Model Runner
Run and serve local LLMs from Docker Hub or Hugging Face with docker model commands and an OpenAI-compatible API
Foundry Local
Microsoft's on-device AI runtime and SDK that runs ONNX models on the user's NPU, GPU or CPU
Work on KoboldCpp? Feature it at the top of Frameworks & Stacks.
Is your product missing?