Local RAG Knowledge Assistant

An intelligent Q&A assistant built on a local qwen3.5-9b model and vector retrieval, delivering accurate answers while preserving data privacy.

Exploring practical applications of local LLMs over private knowledge.

The premise was simple: could I have an assistant that “knows my own material” in a fully offline environment? Around that goal I built up everything from model selection to the vector store to the retrieval pipeline — and along the way came to understand the essence of RAG: retrieval quality sets the ceiling on answers, and the model is only the last mile.

Highlights

• Locally deployed, data never leaves the intranet
• Vector retrieval plus reranking for higher answer accuracy
• Multi-turn conversation with context memory