Build with AI
Choose and test a local model, then build document workflows with Python, APIs, retrieval, and bounded agents. Finish with security, deployment, and maintenance. Local processing is not a privacy guarantee.
Why run AI on your own machine, and what you give up
Keep client material on hardware you control, and understand the trade against frontier cloud models.
How a model actually runs on hardware
Weights, bits, the KV cache, reading versus writing, offloading, and mixture-of-experts, with the two formulas that predict fit and speed.
Every part of a computer, through the lens of local AI
What each component does, how much it matters for running models, and how to find out what is already in your machine.
Devices other than a custom PC
Apple Silicon Macs, unified-memory mini PCs, dedicated AI boxes, and multi-GPU workstations, compared in one table.
Building a PC for local AI
Choose parts by budget, check they fit together, assemble step by step, and get through the first boot.
Engines and front-ends: the software layers
What an inference engine does, what a front-end adds, and which combination to choose.
Choosing a model: families, sizes, formats, and licences
Pick an open-weight model for your memory, read its model card, and check its licence.
Setting it up: three tracks, easy and expert routes
Windows with NVIDIA, Windows with AMD, and Mac, each by app and by command line, and how to confirm the model is on the GPU.
Proving it works: offline, measured, and checked against the formula
An offline test anyone can follow, a benchmark, and a comparison between predicted and measured speed.
Your first useful Python script: a document inventory
Read a practice folder and print a CSV without changing its files.
APIs and JSON: connect a model to a small program
Inspect a local request and distinguish syntax from accuracy.
Use a database to preserve facts and sources
Learn a small SQL example before designing a matter system.
Prepare documents before asking AI to analyze them
Keep originals, inspect extraction, and build a coverage manifest.
Keyword search, semantic search, and reranking
Understand what a search result does—and does not—mean.
How document-grounded AI answers work
Locate failures across ingestion, retrieval, and generation.
Prompts, skills, workflows, and agents
Understand the parts before designing an automation.
Build your first agent around a document inventory
Define a small job, limited permissions, and an observable result.
Make recurring work resumable and reviewable
Specify triggers, checkpoints, duplicate handling, and failures.
Maintain project instructions without accumulating clutter
Keep stable rules, changing state, and source material in their proper places.
Choose scripts, models, and human review deliberately
Route work by observable requirements and enforce boundaries outside the prompt.
Recognize prompt injection in documents and tools
Keep source text from becoming authority to act.
Turn a prototype into a service you can maintain
Plan access, storage, monitoring, and recovery before adding users.
Keeping it current: engines, models, and hardware
Update the engine safely, swap models on evidence, and know when hardware is the real limit.