RAG Chatbot on Self-Hosted Infrastructure

The assistant on this site answers questions about my background, skills and projects. Inference runs on hardware I operate myself.

Gemma 4 12B is served by Ollama, with LiteLLM in front as an OpenAI-compatible router. Embeddings come from Text Embeddings Inference and are matched against Qdrant. The Flask API orchestrates intent classification, hybrid retrieval, generation and validation against the retrieved content.

A publicly reachable language model needs the controls around it: prompt injection detection, rate limiting, CSRF protection, and answers derived only from indexed site content.