Foundry Local
Microsoft's on-device AI runtime and SDK that runs ONNX models on the user's NPU, GPU or CPU
Foundry Local is Microsoft's on-device inference runtime, built to ship inside an application rather than run as a shared server. It runs models through ONNX Runtime and picks an NPU, GPU or CPU execution provider automatically.
It supports Windows, macOS (Apple silicon) and Linux, with SDKs for C, C++, C#, JavaScript, Python and Rust, an optional OpenAI-compatible local server and a CLI. Models come from a curated catalog (Qwen, Phi, Mistral, GPT OSS, Whisper) or your own ONNX Runtime GenAI models.
The SDK is MIT licensed; the CLI is a preview under Microsoft license terms.
Pricing: Free
Choosing a local LLM runtime right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
Choosing a local LLM runtime right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
Foundry Local Alternatives
Explore 49 products in the Frameworks & Stacks category. View all Foundry Local alternatives.
Jan
Open-source desktop app for running LLMs locally with a clean GUI
LocalAI
Open-source, self-hosted OpenAI-compatible API for running models on your own hardware
Magnitude
Open-source desktop inference engine that tunes its kernels to your hardware and connects local models to your agent.
Lemonade Server
Local AI server for LLMs, speech and images, tuned for AMD Ryzen AI NPUs and Radeon GPUs
TextGen
Formerly text-generation-webui: open-source desktop app and web UI for local LLMs with an OpenAI-compatible API
Dify
Easily build and operate generative AI applications. Create Assistants API and GPTs based on any LLMs.
Work on Foundry Local? Feature it at the top of Frameworks & Stacks.
Is your product missing?