Run an Open Model on the Machine You Already Own, Integrated Graphics Included

The number that decides which open model runs on your machine is not the name of your graphics card. It is how much memory the model has to fit into. Get that one number right and you can tell, before downloading several gigabytes, what will run, what will crawl, and what will not load at all.

The short answer: a memory rule, not a brand rule

Weights at 4-bit quantisation occupy roughly 0.6 GB per billion parameters. An 8B model lands near 5 GB, a 14B near 9 GB, a 32B near 20 GB. Add another 15 to 25 percent for the KV cache, because every token that enters the context window claims memory of its own and that claim grows as the conversation gets longer.

Now compare that figure with what you actually have. On a discrete card your ceiling is the VRAM on that card and not one byte more. On integrated graphics your ceiling is system RAM minus whatever the operating system and browser already took, which on a 16 GB machine usually leaves 9 to 11 GB.

The result catches people out: a laptop with integrated graphics and 32 GB of RAM will load a model an 8 GB discrete card flatly refuses. The discrete card is faster, but its memory ceiling is fixed, and in this game you have to fit before you can be fast. 💡

Where that trade collapses

Token generation barely depends on compute. It depends on memory bandwidth, because the model has to read all of its active weights once for every single token it produces. You can estimate the ceiling with one division: memory bandwidth divided by model size.

A machine with dual channel DDR4 sits around 50 GB per second. A 5 GB model on that machine has a theoretical ceiling near 10 tokens per second and lands around 6 to 8 in practice, which is perfectly tolerable for rewriting a paragraph or condensing a few notes. Put a 20 GB model on the same machine and the ceiling drops to about 2.5 tokens per second, so a one paragraph answer takes minutes and the setup stops being useful.

Discrete cards carry several times that bandwidth, which is the entire reason the same model answers so much faster on one. It has not become smarter. So the complete rule reads: memory decides what loads, bandwidth decides what is usable.

Step by step

1. Count the memory you actually have. On a discrete card, read the VRAM figure. On integrated graphics, take total RAM and subtract what the system and your open applications already hold. The remainder is your real ceiling.
2. Derive the model size class from that number. Reserve about 70 percent of free memory for weights and keep the rest for context. Divide that by 0.6 and you get the largest parameter count in billions you can run.
3. Choose a quantisation. Q4_K_M is the sensible default and holds the balance between size and quality. If you have room left, move to Q5 or Q6. Below Q4 the quality drop is noticeable: repetition, dropped instructions, confident mistakes on easy questions.
4. Install a runner and pick the right backend. The engine underneath all of them is the llama.cpp project; LM Studio is the desktop interface on top of it and Ollama is the command line version with a local API. Use CUDA for NVIDIA, ROCm for supported AMD cards, Vulkan for nearly everything else including integrated graphics, Metal on a Mac.
5. Set context length on purpose. Long defaults eat memory for no benefit. For rewriting and short questions, 4096 to 8192 tokens is plenty, and that single setting often decides whether a model loads.
6. Confirm every layer went to the GPU. Runners report how many layers were offloaded at load time. If part of the model stayed on the CPU, expect a several-fold slowdown, and drop a size class rather than living with it.
7. Measure the speed, then split the work. If output arrives faster than you read, that task stays local. If it arrives slower, that task belongs to a hosted model.

The errors you will hit, and what they mean

A failed load or an out of memory message means one thing almost every time: weights plus KV cache exceeded free memory. Halve the context length first, drop one quantisation step second, change size class only if neither worked.

If the model loads but the whole machine seizes up and the disk light stays on, the system is swapping. The model did not fit in RAM and spilled onto storage. Nothing reports an error; the machine just becomes unusable.

Repetitive output, sentences that go nowhere, or an endless loop point at two causes: a quantisation pushed too low, or the wrong chat template. The second happens when you add a file manually and the runner fails to recognise which model family formats its conversations that way.

Windows has a quieter failure that is worse for being quiet. Instead of reporting insufficient VRAM, the driver spills the overflow into system RAM. The model loads, everything looks healthy, and speed falls by an order of magnitude with no message anywhere.

Finally, the wrong file. If a model repository offers several safetensors shards instead of a single GGUF, that release was not packaged for these local runners. The GGUF format documentation sets out exactly that distinction.

The honest limit

A model that runs on your laptop is not comparable to a large hosted one, and anyone claiming otherwise is selling something. Rewriting text, condensing notes, explaining a concept and brainstorming all work well at the 8B class. Refactoring across a multi-file project, reading a hundred page document, running long tool chains, or anything that leans on precise factual recall will fail at this class, usually silently and with a confident wrong answer attached.

That gap is also why promotional benchmark numbers mislead here, since they are measured on the full model rather than the quantised copy you actually run. Our guide to reading AI benchmarks is the real prerequisite for this decision, and the rest of the ZAVINO learn section applies the same logic to other tools.

Notes for readers in Iran

Direct access to the main model repositories is blocked from Iranian addresses, and because in-app search in LM Studio and the pull command in Ollama both travel that same path, they fail at the same point. A failed in-app download is a network problem, and swapping runners will not fix it.

The structural workaround is that a GGUF file stands alone. However it reaches your disk, copying it into the runner's model folder is enough for it to be recognised. LM Studio and llama.cpp accept a local path directly, and Ollama builds a model from a Modelfile pointing at that same file. In practice one person downloads once and the file travels on external storage, with no account and no repeated transfer.

The cost side is where local wins outright: no international card, no monthly subscription, no verification number. After the initial download, every query costs nothing and needs no connection.

For hardware, the buying rule in the local market runs opposite to the sales pitch. A second hand card with more VRAM beats a newer card with less, because a model that does not fit runs at no speed at all. On machines with integrated graphics the cheapest upgrade is usually system RAM rather than a card, since RAM raises the model size ceiling directly. Card prices here also track global pressure on the same supply, which is what Nvidia's 500 billion dollar financing plan is built on.

One language point no English guide covers: small models are measurably weaker in Persian than in English, and their ranking in Persian does not match the English tables. Choose by testing three of your own Persian prompts, not by leaderboard position. Open families with serious multilingual training, of the kind seen in the Qwen3.8-Max weights release, tend to be the better starting point, and for coding work DeepSeek's own coding harness showed how fast the pieces of this stack can be swapped.

If you only download one file

Take the largest model that fits in 70 percent of your free memory at Q4_K_M, from a family with real multilingual training. Then give it three prompts of your own rather than sample ones: a rewrite in the language you actually write in, a summary of something you wrote yourself, and a technical question whose answer you already know. If all three come back acceptable, that file will serve you for a long time.