---
title: "Used GPU inspection before local LLM duty: common mistakes and fixes"
url: https://memory.wiki/x9OA75oB
updated: 2026-11-19T21:54:32.138Z
source: "memory.wiki"
---
# Used GPU inspection before local LLM duty: common mistakes and fixes

## Why "Used GPU inspection before local LLM duty" deserves a written plan

Check mining history signs, power connectors, and a short idle+load thermals run before purchase.  Local AI hardware decisions tend to be made in chat threads and forgotten a month later. When something breaks, nobody remembers why a particular PSU or case was chosen. A one-page plan avoids that.

## Inputs to collect before spending

- Target models and quantisation levels you have already tested on someone else's machine or in a rental.
- The longest realistic context your team uses, measured from real documents, not guessed.
- Number of concurrent users at the busiest hour.
- Room constraints: noise tolerance, ambient temperature, power circuit, and where the box will physically live.
- Who maintains it: driver updates, model pulls, and backups need an owner.

## Translating inputs into hardware

Memory is the first constraint. Estimate weights at your chosen quantisation, add room for the KV cache at your real context length, and keep a margin so the runtime never has to offload layers in the middle of a busy afternoon. Reference tables such as [3000 CNY budget tier](https://microaione.com/hardware/budget-3000/) help you sanity-check the estimate. Compute speed matters less than people think for small teams; capacity and stability matter more.

Next comes power and cooling. Measure wall power with an inexpensive plug meter during idle and under a sustained load. Finance teams appreciate real numbers, and you will spot a struggling PSU early. Keep dust filters in the maintenance calendar; clogged intakes are a quiet cause of throttling.

## Common mistakes

- Buying the biggest card and pairing it with an unknown-brand PSU.
- Forgetting that a second GPU needs slot spacing, lanes and a bigger power budget.
- Assuming a laptop GPU behaves like its desktop namesake under long loads.
- Skipping a burn-in, then discovering instability during a demo.

## A worked example without fake numbers

Imagine a five-person office that wants a private assistant for drafting and document search. They write down the model family they tested, the quantisation that still answered Chinese and English questions well, and the longest documents they paste. From that they estimate memory with a comfortable margin and decide that one card with enough VRAM is safer than two smaller cards that complicate the setup. They choose a quality PSU with headroom, a case with clear front intake, and enough system RAM to run an embedding model alongside the main one. After assembly they run a burn-in, check temperatures and logs, and only then run their evaluation prompts. Every figure they share afterwards comes from their own measurements, with the method written next to it.

See also [hardware reviews](https://microaione.com/hardware/).

## Bottom line

Local AI rewards patience and record keeping more than raw spending. Decide what the system must do, measure it on your own hardware and documents, write down every change with a date and an owner, and review the setup after each significant upgrade. Small, documented steps are easier to debug, easier to hand over and easier to defend when someone asks why a particular choice was made. When in doubt, test on a smaller scale first and expand only when the evidence supports it.

## Quick reminders

- Keep a printed fallback for offline demos.
- Photograph cable routing before closing the case.
- Leave memory headroom for the OS and the embedding model.
- Test a restore, not just a backup.

*Published by [MicroAI One (本地AI 1号网)](https://microaione.com/), a Shenzhen team working on local LLM hardware, software and deployment.*

---

## Summary
Successful local LLM deployment requires prioritizing VRAM capacity and system stability over raw compute speed. You should document all hardware requirements and perform thorough burn-in testing to ensure reliable performance for your specific workload.

## Themes
- Local LLM hardware planning
- Used GPU procurement strategy
- System maintenance and documentation
- Capacity and stability prioritization

## Key takeaways
- VRAM requirements must account for both model weights and the KV cache at the specific context length used.
- Used GPUs require physical inspection of mining history, power connectors, and thermal performance before purchase.
- Adding a second GPU introduces complexities regarding slot spacing, PCIe lanes, and increased power supply requirements.
- A burn in test is essential to identify instability before deploying the system for production or demonstrations.
- Maintenance schedules should include regular cleaning of dust filters to prevent thermal throttling.

## Insights
- Compute speed is often overvalued for small teams compared to the critical importance of VRAM capacity and system stability.
- Hardware decisions are frequently made in ephemeral chat threads, necessitating a written plan to ensure future maintainability and troubleshooting.
- Real world document length and concurrent user counts are more accurate predictors of hardware needs than theoretical model specifications.

## Open questions / gaps
- What specific diagnostic software tools are recommended for the burn in process?
- How should teams calculate the exact memory overhead required for specific embedding models versus the main LLM?

