Local AI Hardware: What to Buy to Run LLMs on Your PC
The hardware that matters most for running AI locally is the graphics card, and within it one number: video memory (VRAM). VRAM decides which models fit, and memory bandwidth decides how fast they answer. Get a GPU with enough of both, pair it with 32GB or more of system RAM and a power supply that can carry it, and you can run capable language models without sending a single prompt to the cloud. If you would rather not source and test the parts yourself, our AI workstations are assembled and stress-tested in the USA, ship with Windows installed and no bloatware, and use standard parts you can upgrade later.
TL;DR
- VRAM sets the ceiling: an 8GB card tops out around 8B-parameter models, while 16GB and 32GB cards open up much larger ones.
- Speed comes from memory bandwidth. A model that fits entirely in VRAM runs far faster than one that spills into system RAM.
- Quantization shrinks a model to roughly a quarter of its original size, which is why consumer cards can run models that once needed a server.
- Plan 32GB of system RAM, a fast NVMe drive and a power supply sized for your GPU, with airflow to match.
- Buy for the model size you want in a year, not today. A GPU with extra VRAM is the upgrade you will not regret.
Table of Contents
- What running AI locally actually means
- The spec that decides everything: VRAM
- Memory bandwidth: why two cards with the same VRAM feel different
- Quantization: how a big model fits a consumer card
- Picking a tier by how you will use the machine
- The rest of the system: RAM, CPU, storage, power
- Where a prebuilt AI workstation makes sense
- A short buying checklist
- FAQ
- Sources
What running AI locally actually means
Local AI means the model runs on a machine you own, not on a provider’s servers. Your prompts, documents, code and images stay on your hardware. Depending on how an application is written, the work can run on the CPU, on integrated graphics, on a discrete GPU or on a dedicated neural processing unit (NPU), as AMD explains in its overview of AI PCs. The same overview points to two practical gains: lower latency, because the task does not travel to a distant server, and better privacy, because sensitive data is not transmitted.
For language models, image generators and coding assistants of any serious size, the discrete GPU is the part that does the heavy lifting. That is why almost every local AI build guide starts there, and why a gaming PC is a natural starting point for AI work. The card that renders your games is the same card that generates tokens.
The spec that decides everything: VRAM
A language model is a very large file of numbers (its weights) that has to sit in memory while it works. The more parameters a model has, the more memory it needs. Memory capacity therefore limits which models you can run at all, while memory speed limits how quickly they respond, as Engadget’s hardware guide for local AI lays out.
The key point is where that memory lives. VRAM is several times faster than system RAM. A model that fits on the GPU runs quickly. A model that does not fit gets split, with some layers pushed out to system RAM, and generation slows sharply. Nothing crashes, which makes the problem easy to miss: the model simply answers slowly enough that you stop using it.
What fits in 8GB, 16GB, 24GB and 32GB

A 2026 hardware report from LocalAI Computer counted the models that fit at Q4 quantization, reserving 2GB for the runtime. According to that report, 8GB fits 167 tracked models, 16GB fits 203, 24GB fits 258 and 32GB fits 262. Its table lists Qwen3 8B as the largest model that fits in 8GB and Kimi Linear 48B A3B Instruct as the largest in 32GB.
Two things stand out. First, the jump from 8GB to 16GB adds a meaningful group of models and also gives you room for longer context (how much text the model can consider at once). Second, going from 24GB to 32GB adds few models by count but moves the largest usable model from mid-size to a far bigger class.
| VRAM | What it realistically runs | Good for |
|---|---|---|
| 8GB | Small models up to roughly 8B parameters | Trying local AI, simple chat and writing help |
| 12GB to 16GB | 7B to 13B models with comfortable context | Daily assistants, coding help, document Q&A |
| 24GB | A wider range of mid-size models | Heavier coding and reasoning work |
| 32GB | The largest single-consumer-card models | Serious local work, larger context, image and video generation alongside LLMs |
| 96GB class workstation cards | 70B-class models and above on one card | Professional inference, training experiments, private deployments |
The figures in the first four rows follow the report above and the sizing guidance from Engadget, which says a PC with 16GB of RAM can run small models around 7B to 8B parameters, such as Llama 3.1 8B, while something closer to frontier models calls for at least 32GB of memory and a graphics card with 16GB or more of VRAM.
Memory bandwidth: why two cards with the same VRAM feel different
Capacity tells you whether a model fits. Bandwidth tells you how fast it talks. Generating each token requires reading the model’s weights from memory, so generation speed is largely limited by how quickly the card can move data, not by raw compute. Engadget describes local generation as memory-bound for exactly this reason.
In practice, a card with a wide memory bus and fast memory modules will out-generate a card with the same VRAM and a narrower bus. When you compare GPUs for AI, look at memory bandwidth in GB/s on the spec sheet next to the capacity. High-end cards in the current generation carry much faster memory than entry-level ones, and that gap shows up directly in tokens per second.
This is also why spilling into system RAM hurts so much. Once part of the model sits in ordinary RAM, every token waits on the slower memory. The fix is not a faster CPU. It is a card with enough VRAM for the model and its context.
Quantization: how a big model fits a consumer card
Quantization stores each parameter in fewer bits. At about four bits per parameter, a model shrinks to roughly a quarter of its original size, as Engadget notes. That is the reason a 16GB card can run models that would otherwise need a data-center GPU.
For most people, 4-bit quantization is the sensible default. It leaves quality close enough for everyday chat, coding and summarizing, and it makes the hardware budget realistic. Higher-precision versions use more memory per parameter, so if you plan to run them, size your VRAM up accordingly.
Context length is the other memory consumer. Long documents and long conversations store extra data as they grow, on top of the weights. If you plan to feed the model entire PDFs or codebases, leave several gigabytes of headroom beyond the model file. This is why a 16GB card often beats a cheaper 12GB card for the same model size.
Picking a tier by how you will use the machine
Casual gamer who wants to try local AI
If you already game, you probably own a card that can run a small model today. One 2026 beginner build guide suggests the RTX 3060 12GB for 7B to 13B models, or the RTX 4060 Ti 16GB for more context headroom, paired with a mid-range Ryzen CPU. The takeaway holds regardless of exact model: at the entry level, VRAM matters more than CPU class, and 16GB is the point where the experience stops feeling cramped.
Buying new for this tier? Choose the card with the most VRAM your budget allows, pick a power supply with room for a bigger card later, and use a case with good airflow. Our gaming PCs use standard parts, so the GPU can be swapped as your models grow.
Enthusiast who games and runs models on the same machine
This is the sweet spot for a 16GB or 32GB card. A 16GB card handles modern games at high settings and mid-size local models. A 32GB card adds room for larger models, long-context work and image generation in the same session. The Ultra 9 AI Workstation with RTX 5080 16GB and the Ultra 9 AI Workstation with RTX 5090 32GB sit at those two VRAM points. They are designed for people who want one system for play and for AI work.
Professional who needs large models and privacy
If you work with client documents, source code or data that cannot leave your premises, local hardware is the point. Models in the 70B class and above need far more memory than a consumer card offers, and that is where workstation GPUs with 96GB of VRAM come in. The Ultra 9 AI Workstation with RTX PRO 6000 96GB puts that capacity in a single card. For multi-GPU needs, such as large-model inference or training experiments, the Threadripper PRO 9995WX AI Server with dual RTX PRO 6000 GPUs is built for that class of work, and we also take on custom multi-GPU builds.
Tech-savvy user who wants to upgrade over time
Upgradeability is mostly about three choices: a standard-size power supply with spare capacity, a motherboard with a free PCIe slot and room for more RAM, and a case big enough for a longer card. When those are standard parts, you can replace the GPU when a new generation arrives without rebuilding. Our systems are built from standard components for exactly this reason.
The rest of the system: RAM, CPU, storage, power
The GPU gets the attention, but the rest of the machine still counts.
- System RAM: 32GB is a sensible floor for serious local AI, matching Engadget’s guidance for larger models. It also gives you a buffer when models load, and a safety net when a model slightly exceeds VRAM.
- CPU: It does not need to be extreme if the model lives on the GPU. A modern mid-to-high-end desktop processor is plenty. Workstation platforms matter when you add multiple GPUs, because they offer more PCIe lanes.
- Storage: Models are large files, often several gigabytes each, and you will collect many. A fast NVMe drive with generous capacity saves load time and frustration.
- Power and cooling: AI workloads can hold a GPU at high power for long stretches, unlike a game session that varies. One build guide suggests a 1000W power supply is often comfortable for an RTX 4090 and 1200W is a reasonable planning point for an RTX 5090, with good airflow to manage heat. Check your card maker’s recommended wattage and size above it.
If you run models for hours, you can also cap the GPU’s power draw with NVIDIA’s command-line tool, nvidia-smi. That lowers noise and heat for continuous use, in exchange for some speed.
Where a prebuilt AI workstation makes sense
Buying the parts yourself means checking that the card fits the case, the power supply has the right connectors, the cooling can handle sustained load and the drivers are stable. A system that has been assembled and tested as a whole removes that guesswork.
Our AI workstations start at $4,599, and our servers start at $34,999. Every system is assembled and stress-tested in the USA before it ships, comes with Windows installed and no bloatware, and is backed by a 3-year parts-and-labor warranty and lifetime US-based tech support. Shipping is free. After a 48-hour burn-in test, every system ships in 5–10 business days. For a first local AI machine with strong VRAM, the Ultra 7 AI Workstation with RTX 5070 Ti 16GB is a natural starting point, and the full range is on the workstations and servers page.
Because the parts are standard, the machine you buy now is not a dead end. When you outgrow a model size, the card is the component you change.
A short buying checklist
- Name the model size you want to run. Pick the largest model you expect to use in the next year, then work out its Q4 size and add a few gigabytes for context.
- Match VRAM to that size. Aim for the model to fit entirely on the GPU. This is the single most important decision.
- Compare memory bandwidth between cards with the same VRAM. Higher bandwidth means more tokens per second.
- Set system RAM at 32GB or more, and choose a fast NVMe drive with room for a growing model library.
- Size the power supply and cooling for sustained load, not for gaming peaks.
- Leave a path to upgrade: a free slot, a standard power supply and a case with space.
FAQ
How much VRAM do I need to run a local LLM?
For small models around 7B to 8B parameters, 8GB is workable, but 12GB to 16GB is more comfortable because it leaves room for context. For models closer to frontier quality, Engadget suggests at least 32GB of system memory and a graphics card with 16GB or more of VRAM. Larger models in the 70B class need workstation-level VRAM.
Can I use my gaming PC for local AI?
Yes. The GPU that renders games also runs language models, and the same VRAM rules apply. Check how much VRAM your card has, then pick models that fit within it after quantization. If your card has 8GB, expect small models, and consider a larger-VRAM card or a new system when you want more.
Is system RAM or VRAM more important?
VRAM matters more, because it is several times faster than system RAM. A model that fits in VRAM runs quickly, while one that spills into RAM slows down. System RAM still matters for loading models and as a buffer, so plan on 32GB or more.
What does quantization do to a model?
Quantization stores each parameter in fewer bits, commonly about four, which cuts the model to roughly a quarter of its original size. That lets consumer GPUs run models that would otherwise be out of reach. It is the default choice for most local setups.
Do I need a multi-GPU setup?
Not for most people. A single high-VRAM card covers everyday and many professional tasks. Multi-GPU systems suit very large models, training experiments and heavy parallel workloads, and they need a platform with enough PCIe lanes and a strong power and cooling plan.
Sources
- Demystifying AI and AI PCs (AMD)
- What Hardware Does Your PC Need To Run A Local AI Model? (Engadget)
- State of Local AI Hardware 2026 (LocalAI Computer)
- Beginner AI 2026: RTX 4060 Ti 16GB + Ryzen 5 7500F (Selfhostr)



