AI Portfolio Assistant — Production RAG + LLMOps
A live RAG chatbot answering questions about my background, built as a complete LLMOps system with eval-gated CI/CD, experiment tracking, and live monitoring.
- RAG
- FAISS
- BM25
- FastAPI
- MLflow
- DagsHub
- Supabase
- GitHub Actions
- AWS EC2
Overview
This chatbot — the one answering questions on this site — is the project. It is not a demo; it is a production system built end-to-end with a full LLMOps pipeline around it.The retrieval layer uses hybrid search: semantic retrieval via FAISS with all-MiniLM-L6-v2 embeddings and keyword retrieval via BM25, merged with Reciprocal Rank Fusion, then reranked with a cross-encoder. The top-5 chunks are passed to Llama 3.1 8B on NVIDIA NIM, which answers in first person as me. The system refuses questions it cannot ground in the source material.What makes this an LLMOps project rather than a chatbot demo is everything around the model: a labeled evaluation set, experiment tracking in MLflow, an eval-gated deployment pipeline that blocks quality regressions, and live per-request monitoring. Every config decision — chunk size, retrieval params, prompt version — was chosen on measured evidence logged in DagsHub.
Architecture
Results
- Hit rate 96.3% — correct project chunk retrieved in top-5 across 28 evaluation questions
- MRR 82.1% — relevant chunk ranked first or second in most queries
- Context precision 69.6% — fraction of retrieved chunks from the correct project
- Eval-gated CI/CD blocks merges when hit_rate or MRR regress beyond 5% threshold
- Every request logged to Supabase — per-stage latency, retrieval confidence, token usage
- Live monitoring dashboard on the homepage refreshes every 30 seconds