Roofline
Edge AI deployment stack: an MLIR/IREE compiler and device runtime for CPUs, GPUs and NPUs
Roofline builds AI deployment infrastructure for edge devices: a compiler, runtime and benchmarking tools for running models on the chips inside industrial, automotive, robotics and mobile products.
The compiler and runtime stack is built on MLIR and IREE, translating models from PyTorch and TensorFlow into executables tuned for the CPUs, GPUs and NPUs in a heterogeneous system. The runtime is C-based, supports dynamic shapes, and runs on Linux, macOS, Windows and bare metal.
NXP and Arm are named on the site as working with Roofline on edge platform support.
What is Roofline?
Roofline is an edge AI deployment toolchain: an SDK built around an AI compiler, a device-side runtime, and a performance dashboard for tracking real-world results. It targets teams shipping AI inside physical products (industrial equipment, vehicles, robots, phones) rather than teams calling a hosted API.
The compiler
The stack is built on MLIR and IREE. Models from PyTorch and TensorFlow are compiled ahead of time into executables tuned for a specific target, with the aim of using the whole SoC rather than one accelerator. Roofline publishes case studies on the underlying work, including a Vulkan 1.3 HAL for IREE demonstrated on an ARM Mali-G720 GPU, asynchronous heterogeneous execution, dynamic shapes for LLMs, and NXP Neutron LLM enablement.
The runtime
A lightweight C-based engine that executes the compiled model on the device. It handles heterogeneous execution across CPUs, GPUs and NPUs, supports dynamic shapes at execution time, is extendable to new NPUs through HALs, and runs on Linux, macOS and Windows as well as bare-metal systems. This is the piece that makes Roofline comparable to other inference engines, though it targets embedded hardware rather than servers.
Who it is for
Two audiences, and the site addresses them separately: product vendors putting AI into a device, and hardware or IP vendors enabling models on their own silicon. NXP and Arm are named as working with Roofline on edge platform support.
Pricing is not published; the site routes to a contact form. Roofline is RooflineAI GmbH, based in Germany.
Roofline Alternatives
Explore 37 products in the Frameworks & Stacks category. View all Roofline alternatives.
Anycloud
CLI and Python SDK for running AI jobs, services and VMs across your own AWS, Azure, GCP, Lambda and Vast accounts
vLLM
High-throughput LLM inference engine with PagedAttention for efficient GPU memory usage
Jan
Open-source desktop app for running LLMs locally with a clean GUI
Work on Roofline? Feature it at the top of Frameworks & Stacks.
Is your product missing?