Local AI, in depth
Optional depth: run a model on hardware you control. Check memory, setup, data flows and results before using client files.
Decide whether to use what you have, buy a device, or build a PC, set it up, and prove it runs offline at the speed the formula predicts.
Why run AI on your own machine, and what you give up
Keep client material on hardware you control, and understand the trade against frontier cloud models.
How a model actually runs on hardware
Weights, bits, the KV cache, reading versus writing, offloading, and mixture-of-experts, with the two formulas that predict fit and speed.
Every part of a computer, through the lens of local AI
What each component does, how much it matters for running models, and how to find out what is already in your machine.
Building a PC for local AI
Choose parts by budget, check they fit together, assemble step by step, and get through the first boot.
Devices other than a custom PC
Apple Silicon Macs, unified-memory mini PCs, dedicated AI boxes, and multi-GPU workstations, compared in one table.
Engines and front-ends: the software layers
What an inference engine does, what a front-end adds, and which combination to choose.
Choosing a model: families, sizes, formats, and licences
Pick an open-weight model for your memory, read its model card, and check its licence.
Setting it up: three tracks, easy and expert routes
Windows with NVIDIA, Windows with AMD, and Mac, each by app and by command line, and how to confirm the model is on the GPU.
Proving it works: offline, measured, and checked against the formula
An offline test anyone can follow, a benchmark, and a comparison between predicted and measured speed.
Keeping it current: engines, models, and hardware
Update the engine safely, swap models on evidence, and know when hardware is the real limit.
A useful way to work
Read in order for a gradual introduction, or use an individual lesson as a reference when a question comes up.