ggml.ai (llama.cpp)
MIT-licensed C and C++ inference engine for open-weight language models, on CPU or GPU backends.
llama.cpp is the inference engine behind much of the local model ecosystem. Built on the ggml tensor library, it runs quantised models on CPUs and on CUDA, Metal and Vulkan backends, which makes it a common choice on Arm edge boards and Apple silicon alike.
Edge board, workstation or server, with or without a GPU
Specifications taken from current manufacturer documentation and last checked on 2026-09-08.
Source: manufacturer documentation
Specifications are summarised from publicly published manufacturer material and are provided for orientation only. Always confirm current figures directly with the manufacturer before purchasing or designing in.
MIT-licensed runtime for pulling and serving open-weight models locally through a CLI and local API.
Apache 2.0 licensed 24-billion-parameter model built for low latency and self-hosted deployment.
MIT-licensed 14-billion-parameter small language model built for low-latency reasoning tasks.