What you will be able to do
- Explain the difference between a model file, an engine, and a front-end.
- Choose an engine for your hardware and comfort level.
- Choose a front-end for chatting with documents.
Three layers
- The model fileA GGUF or MLX file of weights, downloaded once.
- The engineLoads the file onto your GPU and does the maths: llama.cpp, Ollama, LM Studio, MLX.
- The front-endThe chat window and document tools: built-in, Open WebUI, AnythingLLM.
- YouEvery layer runs on your machine. Nothing needs the internet after setup.
Most apps bundle two or three layers together. Knowing the layers tells you where a problem lives.
The model file holds the weights. The engine loads those weights onto your hardware and does the arithmetic of prefill and decode from lesson 9.2. The front-end is what you look at: a chat window, document upload, chat history. Some products bundle all three; knowing the layers tells you where to look when something goes wrong, and which piece to swap.
The engines
llama.cpp is the engine under most local apps. It supports NVIDIA (CUDA), AMD (HIP and Vulkan), Apple (Metal), and plain CPUs, and can split a model between card and RAM. It now ships a single llama command with cli and serve subcommands, and a built-in web interface. [source]
Ollama wraps an engine in a background service with one-line install and model pulls, and a local API on your machine. [source] LM Studio is a desktop app with a model browser and a GPU offload control, and a command-line tool, lms, for loading models and starting a local server. [source]
MLX, through the mlx-lm package, is Apple's own framework and runs only on Apple Silicon. [source] vLLM is built to serve many users at once on Linux servers; it is the right tool for a firm-wide server, not for one lawyer's desk. [source]
The front-ends
Open WebUI gives a ChatGPT-style interface in your browser, served from your own machine, and connects to Ollama or any OpenAI-compatible local server. [source] AnythingLLM is a desktop app built around document workspaces; it has its own built-in engine and can also connect to Ollama or LM Studio. [source] llama.cpp's own server and LM Studio both include a chat window, which may be all you need.
| Name | Layer | Runs on | What it is good for | Docs |
|---|---|---|---|---|
| llama.cpp | Engine | Windows, Mac, Linux; NVIDIA (CUDA), AMD (HIP, Vulkan), Apple (Metal), CPU | An engine used by many local apps. Command line and a built-in web UI. Check the supported backend for your hardware. | official |
| Ollama | Engine + simple app | Windows, Mac, Linux; NVIDIA, AMD (ROCm or Vulkan), Apple | One-command install and model pulls; runs in the background with a local API. A scripted setup option. | official |
| LM Studio | Engine + desktop app | Windows, Mac (Apple Silicon), Linux | A point-and-click app with a model browser, GPU offload slider, and an optional local server. A graphical setup option. | official |
| MLX / mlx-lm | Engine | Mac (Apple Silicon) only | Apple’s own machine-learning framework; a native option on a Mac. Python command line. | official |
| vLLM | Engine (server) | Linux with data-centre or high-end GPUs | Built to serve many users at once. Overkill for one lawyer on one machine; relevant for a firm server. | official |
| Open WebUI | Front-end | Any browser; runs on your machine via Python or Docker | A ChatGPT-style interface for a local engine, with users, document upload, and chat history kept locally. | official |
| AnythingLLM | Front-end + workspace | Windows, Mac, Linux desktop app | Chat with documents in workspaces; connects to Ollama, LM Studio, and others, and has a built-in engine. | official |
As of 16 October 2026. Platform support from each project’s official documentation.
Choosing
- First try, any platform: LM Studio. Point and click, shows whether a model fits, and has a GPU slider.
- Scripts and other apps on the same machine: Ollama, or llama.cpp's server.
- Maximum control or unusual hardware: llama.cpp from the command line.
- Mac, best speed: try MLX alongside LM Studio and compare with the benchmark in lesson 9.9.
- Documents and colleagues: Open WebUI or AnythingLLM on top of one of the engines.
Self-check
- What does the engine do that the front-end does not?
- Which engine is Apple-only?
- Which tool would you use for a firm server with many simultaneous users?
- You want a browser-based chat window over Ollama. Which front-end fits?
Check your understanding
- It loads the weights onto the hardware and computes prefill and decode. The front-end only presents input and output.
- MLX (mlx-lm).
- vLLM on Linux.
- Open WebUI.