Running local LLMs on a NAS

What does a local LLM need from a NAS?

How fast will it be?

On the processor alone, speed depends mostly on memory bandwidth. A modern laptop-class chip answers comfortably with 8B class models and slowly with 32B class models. A graphics card holding the whole model is many times faster. Measured speeds vary by model, quantization and context length, so treat any single figure with caution until it has been tested on the system in question.

Which models are worth running?

For a private assistant, 8B class models such as Llama 3.1 8B, Qwen3 8B and Gemma 3 12B are a good starting point. For stronger reasoning and writing, 27B to 32B class models such as Gemma 3 27B and Qwen3 32B are worth the memory. 70B class models give the best quality that fits on a NAS, at the cost of speed.

How do you set it up?

Install Ollama, either from the vendor's app centre or as a Docker container, then pull a model and add Open WebUI for a browser chat interface. UGREEN, QNAP and ASUSTOR publish their own guides; on Synology it runs through Container Manager. UGREEN's iDX range and Zettlab's Ultra models also include built-in assistants.

Common questions

Will a local LLM work offline?

Yes. Once a model is downloaded, everything runs on the NAS without an internet connection.

Can several people use it at once?

Yes, but each conversation shares the same hardware. For a team, add memory and ideally a GPU.