DevOps AI Assistant - Local LLM with RAG

Architecture

High-level overview of the RAG architecture.

Project Overview

This project provides a production-ready AI assistant that uses local LLMs (Ollama) and Retrieval-Augmented Generation (RAG) for DevOps documentation. It allows users to query various DevOps tools like Kubernetes, Terraform, Docker, and Ansible using natural language.

Key Features

• **Local LLM Inference**: Utilizes Ollama and supports models such as Llama 3.1, Mistral, and CodeLlama.
• **RAG Pipeline**: Employs Qdrant for vector search to provide accurate and context-aware responses.
• **DevOps Documentation**: Pre-configured to ingest documentation for Kubernetes, Terraform, Docker, Ansible, AWS, and more.
• **Web UI**: Features a clean and responsive chat interface.
• **REST API**: A FastAPI backend enables integration with other tools.
• **GPU Acceleration**: Optimized for NVIDIA GPUs.