Running Llama models on a NAS

Which Llama model should you run on a NAS?

For a private assistant, Llama 3.1 8B is the practical choice: about 4.9GB at 4-bit, quick on a modern AI NAS processor, and good at everyday writing and questions. The small Llama 3.2 models, 1B and 3B, suit very low-memory systems and simple tasks. Llama 3.3 70B gives much stronger answers but needs a NAS with 64GB to 96GB of memory and is slow without a GPU.

What about Llama 4?

Llama 4 Scout and Maverick use a mixture-of-experts design: only part of the model is active for each word, but the whole model must still sit in memory. At 4-bit, Scout is around 67GB and Maverick around 245GB, so Scout needs a 96GB-class system and Maverick is beyond every desktop NAS we track (Estimated).

Can a NAS run Llama without a GPU?

Yes for the 8B and smaller models, which run at a comfortable pace on the CPU and integrated graphics of current AI NAS systems. 70B models run, but answers come slowly. A graphics card with enough memory changes that, which is why GPU slots matter for anyone planning to use the larger models daily.

Common questions

Is Llama free to use on a NAS?

Llama models are released under Meta's licence, which allows most personal and business use. Check the licence for your use case.

Which quantization should I use?

Q4 is the usual balance of quality and size. Use Q5 or Q8 when memory allows, as the table above shows for each model.