Overview
RAGOpt has three core components working together to optimize your RAG pipeline: RAG Pipeline: Handles data loading, parsing, chunking, indexing, and retrieval. Supports multiple providers for embeddings, LLMs, and vector stores. Metric Evaluator: Measures pipeline performance across quality, speed, cost, and safety dimensions. Acts as the objective function for optimization. Bayesian Optimizer: The brain of RAGOpt. Intelligently searches the configuration space by:- Loading your search space definition (which parameters to tune)
- Sampling configurations using Sobol sequences or other strategies
- Encoding parameters as PyTorch tensors for numerical optimization
- Using acquisition functions to select promising configurations
- Iteratively converging on optimal settings
How It Works
1. Setup & Storage
Generate evaluation questions from your data, then store them in dual systems:- Vector Store: Semantic similarity search
- Lexical Store: Keyword-based retrieval
2. Define Your Pipeline
Configure your RAG system via YAML. Use the default LangChain implementation or build custom agentic pipelines.RAG Configuration
Learn how to configure your RAG pipeline
3. Bayesian Optimization
Why Bayesian Optimization?RAG evaluation is expensive. Unlike grid or random search, Bayesian Optimization builds a probabilistic model to intelligently select which configurations to test next, finding optimal settings with far fewer evaluations. Powered by BoTorch: PyTorch-based optimization with flexible sampling strategies (Sobol, random).
Optimization Process
Deep dive into the optimization workflow
4. Comprehensive Metrics
Each iteration evaluates your pipeline across multiple dimensions:- Quality: Context precision/recall, relevancy, faithfulness, correctness (from RAGEval paper & RAGAS)
- Performance: Latency and cost per query
- Safety: Toxicity and bias detection
- Ranking: MRR and NDCG
Metrics System
Explore metrics and create custom evaluations