---
title: "VM GPU passthrough worth-it test: lessons from real setups"
url: https://memory.wiki/HQsz-ZGZ
updated: 2026-11-18T21:55:23.631Z
source: "memory.wiki"
---
# VM GPU passthrough worth-it test: lessons from real setups

## The question behind "VM GPU passthrough worth-it test"

If live migration matters, maybe not; if isolation matters, yes.  Most hardware threads about local LLMs jump straight to a card model. A better starting point is the workload: which model families you actually need, the quantisation you can live with, the context length your users paste, and how many people will hit the box at once. Only then does a shopping list make sense.

## Work out the memory budget first

Model weights are only part of the picture. The KV cache grows with context length and concurrent sessions, the runtime keeps its own buffers, and the desktop or display server may already be holding a slice of VRAM before you load anything. Write down free memory after a cold boot, not the number on the box. If the model plus a realistic context does not fit with headroom, it will spill to system RAM and feel slow in a way that no setting fixes. A [3000 CNY budget tier](https://microaione.com/hardware/budget-3000/) is useful here because it lets you reason in parameter counts and quant levels instead of marketing tiers.

## Platform matters as much as the card

A good GPU on a weak platform is a common regret. Check:

- PSU capacity and quality, with real headroom for transient spikes, not just the nominal TDP.
- Physical clearance: card length, thickness, and the bend radius of the power cable.
- PCIe slot wiring if you plan to add a second card later.
- Airflow: intake and exhaust paths that still work when the side panel is on and the room is warm.
- System RAM: enough to hold the OS, the inference server, and an embedding model without swapping.

## New, used, or unified memory

Used cards can be good value, but inspect them: missing screws, replaced fans, discoloured connectors and odd thermals under a short load test are reasons to walk away. Apple Silicon machines take a different route: unified memory means the model and the system share one pool, so leave clear headroom for macOS and anything else you run. Mini PCs are fine for small models and kiosks if you accept their thermal limits. Whatever you choose, record serial numbers, invoices and photos on install day.

## A sane buying sequence

1. Write the workload in one paragraph: models, context, users, hours.
2. Map it to memory needs with headroom.
3. Choose the platform (PSU, board, case, RAM), then the GPU.
4. Burn in for at least half an hour while watching temperatures and logs.
5. Run your own small eval set before declaring the build done.

## A worked example without fake numbers

Imagine a five-person office that wants a private assistant for drafting and document search. They write down the model family they tested, the quantisation that still answered Chinese and English questions well, and the longest documents they paste. From that they estimate memory with a comfortable margin and decide that one card with enough VRAM is safer than two smaller cards that complicate the setup. They choose a quality PSU with headroom, a case with clear front intake, and enough system RAM to run an embedding model alongside the main one. After assembly they run a burn-in, check temperatures and logs, and only then run their evaluation prompts. Every figure they share afterwards comes from their own measurements, with the method written next to it.

See also [hardware reviews](https://microaione.com/hardware/).

## Quick reminders

- Leave memory headroom for the OS and the embedding model.
- Use one models directory shared by every runtime.
- Pin driver and runtime versions together.
- Publish what the assistant will refuse to do.

*Published by [MicroAI One (本地AI 1号网)](https://microaione.com/), a Shenzhen team working on local LLM hardware, software and deployment.*

---

## Summary
Successful local LLM hardware builds require prioritizing workload requirements and memory headroom over marketing specs. You should determine your specific model and context needs first, then select a platform and GPU that provide sufficient VRAM and thermal capacity to avoid performance bottlenecks.

## Themes
- Local LLM hardware planning
- VRAM memory management
- System platform requirements
- Workload-first deployment

## Key takeaways
- Workload requirements including model family, quantization, and context length should be defined before selecting hardware.
- VRAM capacity must include headroom for the KV cache and concurrent user sessions to prevent system RAM spilling.
- Platform components like the PSU and airflow must be prioritized alongside the GPU to handle transient power spikes and thermal loads.
- A structured buying sequence starting with workload definition and ending with burn-in testing ensures build stability.
- Embedding models require their own dedicated system RAM allocation separate from the main inference model.

## Insights
- The true available VRAM is the total capacity minus the amount consumed by the OS and display server at idle.
- A high-performance GPU is often bottlenecked by inadequate power delivery or poor thermal management in the host system.
- Unified memory architectures like Apple Silicon require a different planning approach than dedicated VRAM because the system and model share the same pool.

## Open questions / gaps
- How should one specifically calculate the exact VRAM overhead for different operating systems versus headless Linux setups?

