What you will be able to do

  • Run an offline test that shows the answers come from your machine.
  • Run a benchmark and read its two numbers.
  • Compare your measured writing speed with the formula's prediction.

The offline test

The offline test, in four steps
  1. Load the model while onlineDownload and start the model so nothing is mid-download.
  2. Cut the networkUnplug the cable and turn off Wi-Fi, or block the app with the firewall.
  3. Ask and timeAsk a test question; the answer should arrive at normal speed.
  4. Restore and recordReconnect, and record the date, model, and result in your notes.

If the answer still appears with the network gone, the work happened on your machine. The firewall version also proves the app does not quietly try to call out.

Version 1: pull the plug (anyone can do this).

  1. Start your app and load the model while still online, so nothing is mid-download.
  2. Ask a short test question and note how long the answer takes.
  3. Unplug the network cable and turn off Wi-Fi (on Windows, the network icon at the bottom right; on a Mac, the Wi-Fi icon at the top right).
  4. Ask a second, different question. Check whether it can answer with the network disconnected. The speed may vary.
  5. Reconnect. Write down the date, the app, the model, and the result.

Version 2: block the app with the firewall (Windows). This tests whether that executable can complete the task while its outbound access is blocked. Other processes need their own checks. Open PowerShell as administrator. The command below uses the parameters documented for New-NetFirewallRule; replace the path with your app's program file. [source]

text
New-NetFirewallRule -DisplayName "Block local AI app" -Direction Outbound -Program "C:\Path\To\App.exe" -Action Block

Repeat steps 2 and 4 above. Remove the rule afterwards with Remove-NetFirewallRule -DisplayName "Block local AI app".

Two tool settings make the result firmer. llama.cpp's --offline option restricts model downloads to cached files. Do not treat this flag as an operating-system firewall for every tool or process. [source] Ollama can be set to local-only mode with OLLAMA_NO_CLOUD=1, after which its log shows "Ollama cloud disabled: true". [source]

The benchmark

llama-bench ships with llama.cpp. It reports two numbers: pp (prompt processing, the prefill speed) and tg (text generation, the decode speed), both in tokens per second. [source]

text
llama-bench -m path\to\model.gguf

Record a reproducible test

Record the runtime version, model file hash, hardware, command, context length and date. Repeat the task and keep the raw logs. Separate prompt processing from text generation. The earlier claimed measurements are withheld because their raw logs were not supplied. [source]

Record your own benchmark

The earlier example measurements are withheld because the source package did not include raw run logs. Record the model file hash, runtime version, hardware, command, context length, repeated results and date. Keep prompt processing and generated-token speed separate. A bandwidth calculation is a simplified ceiling, not a measured result.

Self-check

  1. In the plug-pull test, what result shows this task can run without a network link?
  2. What do pp and tg measure in llama-bench?
  3. Your card has 448 GB/s and your model file is 5.63 GB. You measure 61 tokens per second. What percentage of the formula is that?
  4. What does llama.cpp's --offline option do?
Check your understanding
  1. The second question is answered with the network disconnected. This tests that task and setup, not every future data flow.
  2. pp is prompt processing (prefill) speed; tg is text generation (decode) speed.
  3. 448 ÷ 5.63 ≈ 79.6 tokens/s predicted; 61 ÷ 79.6 ≈ 77%.
  4. It uses cached models instead of downloading them. Use network controls to test and restrict the full app and its tools.