llamafile
Single-file executables that run LLMs locally on Linux, macOS, Windows and BSD with an OpenAI-compatible server
llamafile, from Mozilla.ai, packages an LLM's weights and runtime into one executable built with Cosmopolitan Libc. The same file runs on Linux, macOS, Windows and the BSDs, on x86-64 and ARM64, with no install step. It can also load any external GGUF model.
Running it opens a terminal chat, a local web UI and an API on port 8080 compatible with the OpenAI and Anthropic Messages APIs. GPU offload works through Metal, CUDA, ROCm and Vulkan, with CPU fallback. Apache-2.0 licensed.
Pricing: Free
Choosing a local LLM runtime right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
Choosing a local LLM runtime right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
llamafile Alternatives
Explore 49 products in the Frameworks & Stacks category. View all llamafile alternatives.
GPT4All
Desktop app and Python SDK for running open-source LLMs locally on any device
LocalAI
Open-source, self-hosted OpenAI-compatible API for running models on your own hardware
Jan
Open-source desktop app for running LLMs locally with a clean GUI
KoboldCpp
Single-binary llama.cpp runtime with an OpenAI-compatible API, web UI, image generation and speech
RamaLama
Run and serve local LLMs in GPU-matched OCI containers with Podman or Docker
Docker Model Runner
Run and serve local LLMs from Docker Hub or Hugging Face with docker model commands and an OpenAI-compatible API
Work on llamafile? Feature it at the top of Frameworks & Stacks.
Is your product missing?