RAG Triad Evaluation with TrueLens: A Complete Guide

Deploying Retrieval-Augmented Generation (RAG) systems to production requires rigorous evaluation to prevent hallucinations and ensure factual accuracy. The industry standard for evaluating RAG pipelines is the **RAG Triad**, which measures three core metrics: **Context Relevance**, **Groundedness**, and **Answer Relevance**.

This comprehensive guide walks through setting up TrueLens, instrumenting a custom RAG application, configuring feedback functions for the RAG Triad, and analyzing evaluation scores.

1. Environment Setup and Dependencies

Create a virtual environment and install `trulens`, along with your orchestration and LLM provider packages.

# Create and activate virtual environment
python3 -m venv venv
source venv/bin/activate

# Install TrueLens and dependencies
pip install trulens-eval openai langchain lancedb

2. Understanding the RAG Triad Metrics

The RAG Triad evaluates the quality of your application across three key dimensions using LLM-as-a-judge feedback functions:

1. **Context Relevance:** Measures whether the retrieved context chunks are relevant to the user query. 2. **Groundedness:** Evaluates whether the generated answer is fully supported by the retrieved context (checking for hallucinations). 3. **Answer Relevance:** Assesses whether the final response directly answers the user's original query.

3. Configuring TrueLens Feedback Functions

Initialize OpenAI provider feedback functions in TrueLens to automatically compute scores for the RAG Triad.

from trulens_eval import TruChain, Feedback, OpenAI
import numpy as np

# Initialize OpenAI provider for feedback evaluation
openai_feedback = OpenAI()

# 1. Context Relevance: is context relevant to user query?
f_qa_relevance = Feedback(
    openai_feedback.qs_relevance,
    name="Context Relevance"
).on_input().on_retrieved_context()

# 2. Groundedness: is answer supported by retrieved context?
f_groundedness = Feedback(
    openai_feedback.groundedness_measure_with_cot_reasons,
    name="Groundedness"
).on_retrieved_context().on_output()

# 3. Answer Relevance: does answer address user query?
f_ans_relevance = Feedback(
    openai_feedback.relevance,
    name="Answer Relevance"
).on_input().on_output()

4. Instrumenting the RAG Pipeline and Recording

Wrap your LangChain or custom RAG chain with a `TruChain` recorder to evaluate queries during execution.

from trulens_eval import Tru

tru = Tru()

# Wrap your RAG chain with TruLens recorder
tru_chain = TruChain(
    rag_chain,
    app_id="RAG_Application_v1",
    feedbacks=[f_qa_relevance, f_groundedness, f_ans_relevance]
)

# Run queries through the instrumented app
response = tru_chain.query("What are the best practices for fine-tuning Llama 3?")

# Launch dashboard to inspect evaluation metrics
tru.run_dashboard()

Conclusion

By implementing the **RAG Triad** with **TrueLens**, you can continuously monitor, measure, and optimize your LLM application's retrieval precision and factual generation accuracy in production.