https://pichler.dev/tags/ollama/ Recent content in Ollama on Marcus Pichler Hugo en-us marcus@pichler.dev (Marcus Pichler) marcus@pichler.dev (Marcus Pichler) Thu, 20 Aug 2026 00:00:00 +0000 https://pichler.dev/blog/building-rag-chatbot/ Thu, 26 Feb 2026 00:00:00 +0000marcus@pichler.dev (Marcus Pichler) https://pichler.dev/blog/building-rag-chatbot/ <h2 id="tldr">TL;DR</h2> <p>I built a <strong>Retrieval-Augmented Generation (RAG) chatbot</strong> that answers questions about my professional background using my portfolio website as the single source of truth. It runs entirely on my own hardware - a K3s single-node setup with an NVIDIA RTX 3060 - with no cloud AI dependencies. The stack includes <strong>Ollama</strong> serving <strong>Gemma 4 12B</strong> on the GPU (it was vLLM with Gemma 3 4B when this was written), <strong>Qdrant</strong> for vector search, and a <strong>Flask API</strong> with OWASP-aligned security. Everything is deployed via GitLab CI/CD across three repositories, with automated content indexing, monitoring, and zero manual intervention.</p>