Private RAG on a NAS

What does private RAG need from a NAS?

How does it work?

Your documents are split into passages and turned into vectors by the embedding model, which builds a searchable index. When someone asks a question, the closest passages are pulled from the index and sent to the chat model with the question, so the answer is grounded in your files. On a NAS the documents, index and models all stay on the same hardware.

What makes one system better than another for RAG?

Memory first, because two models share it. Then storage speed for the index, especially during the first build. Networking matters when files arrive from other machines or several people query at once. A GPU speeds up the answering model most; embedding runs well on a modern CPU.

Which software should you use?

AnythingLLM is the most complete document workspace and connects to Ollama for local models. Open WebUI includes document collections and works well for a mixed chat and document setup. QNAP's Qsirch indexes the whole NAS and offers a local AI mode on systems with a suitable GPU. UGREEN iDX and Zettlab Ultra systems build semantic search into the operating system.

Common questions

Does RAG need a GPU?

No. A modern processor handles embedding and small chat models. A GPU makes answers quicker and suits larger models.

What file types work?

PDFs, Word and text documents, spreadsheets and web pages are supported by the main tools. Scanned images need OCR first.