ancoleman/ai-design-components/evaluating-llms

LLM Evaluation

Evaluate Large Language Model (LLM) systems using automated metrics, LLM-as-judge patterns, and standardized benchmarks to ensure production quality and safety.

When to Use This Skill

Apply this skill when:

• Testing individual prompts for correctness and formatting
• Validating RAG (Retrieval-Augmented Generation) pipeline quality
• Measuring hallucinations, bias, or toxicity in LLM outputs
• Comparing different models or prompt configurations (A/B testing)
• Running benchmark tests (MMLU, HumanEval) to assess model capabilities
• Setting up production monitoring for LLM applications
• Integrating LLM quality checks into CI/CD pipelines

Common triggers:

• "How do I test if my RAG system is working correctly?"
• "How can I measure hallucinations in LLM outputs?"
• "What metrics should I use to evaluate generation quality?"
• "How do I compare GPT-4 vs Claude for my use case?"
• "How do I detect bias in LLM responses?"

Evaluation Strategy Selection

Decision Framework: Which Evaluation Approach?

By Task Type:

By Volume and Cost:

Layered Approach (Recommended for Production):

1. Layer 1: Automated metrics for all outputs (fast, cheap)
2. Layer 2: LLM-as-judge for 10% sample (nuanced quality)
3. Layer 3: Human review for 1% edge cases (validation)

Core Evaluation Patterns

Unit Evaluation (Individual Prompts)

Test single prompt-response pairs for correctness.

Methods:

• Exact Match: Response exactly matches expected output
• Regex Matching: Response follows expected pattern
• JSON Schema Validation: Structured output validation
• Keyword Presence: Required terms appear in response
• LLM-as-Judge: Binary pass/fail using evaluation prompt

Example Use Cases:

• Email classification (spam/not spam)
• Entity extraction (dates, names, locations)
• JSON output formatting validation
• Sentiment analysis (positive/negative/neutral)

Quick Start (Python):

import pytest
from openai import OpenAI
client = OpenAI()
def classify_sentiment(text: str) -> str:
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "Classify sentiment as positive, negative, or neutral. Return only the label."},
            {"role": "user", "content": text}
        ],
        temperature=0
    )
    return response.choices[0].message.content.strip().lower()
def test_positive_sentiment():
    result = classify_sentiment("I love this product!")
    assert result == "positive"

For complete unit evaluation examples, see examples/python/unit_evaluation.py and examples/typescript/unit-evaluation.ts.

RAG (Retrieval-Augmented Generation) Evaluation

Evaluate RAG systems using RAGAS framework metrics.

Critical Metrics (Priority Order):

1. Faithfulness (Target: > 0.8) - MOST CRITICAL

• Measures: Is the answer grounded in retrieved context?
• Prevents hallucinations
• If failing: Adjust prompt to emphasize grounding, require citations

1. Answer Relevance (Target: > 0.7)

• Measures: How well does the answer address the query?
• If failing: Improve prompt instructions, add few-shot examples

1. Context Relevance (Target: > 0.7)

• Measures: Are retrieved chunks relevant to the query?
• If failing: Improve retrieval (better embeddings, hybrid search)

1. Context Precision (Target: > 0.5)

• Measures: Are relevant chunks ranked higher than irrelevant?
• If failing: Add re-ranking step to retrieval pipeline

1. Context Recall (Target: > 0.8)

• Measures: Are all relevant chunks retrieved?
• If failing: Increase retrieval count, improve chunking strategy

Quick Start (Python with RAGAS):

from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_relevancy
from datasets import Dataset
data = {
    "question": ["What is the capital of France?"],
    "answer": ["The capital of France is Paris."],
    "contexts": [["Paris is the capital of France."]],
    "ground_truth": ["Paris"]
}
dataset = Dataset.from_dict(data)
results = evaluate(dataset, metrics=[faithfulness, answer_relevancy, context_relevancy])
print(f"Faithfulness: {results['faithfulness']:.2f}")

For comprehensive RAG evaluation patterns, see references/rag-evaluation.md and examples/python/ragas_example.py.

LLM-as-Judge Evaluation

Use powerful LLMs (GPT-4, Claude Opus) to evaluate other LLM outputs.

When to Use:

• Generation quality assessment (summaries, creative writing)
• Nuanced evaluation criteria (tone, clarity, helpfulness)
• Custom rubrics for domain-specific tasks
• Medium-volume evaluation (100-1,000 samples)

Correlation with Human Judgment: 0.75-0.85 for well-designed rubrics

Best Practices:

• Use clear, specific rubrics (1-5 scale with detailed criteria)
• Include few-shot examples in evaluation prompt
• Average multiple evaluations to reduce variance
• Be aware of biases (position bias, verbosity bias, self-preference)

Quick Start (Python):

from openai import OpenAI
client = OpenAI()
def evaluate_quality(prompt: str, response: str) -> tuple[int, str]:
    """Returns (score 1-5, reasoning)"""
    eval_prompt = f"""
Rate the following LLM response on relevance and helpfulness.
USER PROMPT: {prompt}
LLM RESPONSE: {response}
Provide:
Score: [1-5, where 5 is best]
Reasoning: [1-2 sentences]
"""
    result = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": eval_prompt}],
        temperature=0.3
    )
    content = result.choices[0].message.content
    lines = content.strip().split('\n')
    score = int(lines[0].split(':')[1].strip())
    reasoning = lines[1].split(':', 1)[1].strip()
    return score, reasoning

For detailed LLM-as-judge patterns and prompt templates, see references/llm-as-judge.md and examples/python/llm_as_judge.py.

Safety and Alignment Evaluation

Measure hallucinations, bias, and toxicity in LLM outputs.

Hallucination Detection

Methods:

1. Faithfulness to Context (RAG):

• Use RAGAS faithfulness metric
• LLM checks if claims are supported by context
• Score: Supported claims / Total claims

1. Factual Accuracy (Closed-Book):

• LLM-as-judge with access to reliable sources
• Fact-checking APIs (Google Fact Check)
• Entity-level verification (dates, names, statistics)

1. Self-Consistency:

• Generate multiple responses to same question
• Measure agreement between responses
• Low consistency suggests hallucination

Bias Evaluation

Types of Bias:

• Gender bias (stereotypical associations)
• Racial/ethnic bias (discriminatory outputs)
• Cultural bias (Western-centric assumptions)
• Age/disability bias (ableist or ageist language)

Evaluation Methods:

1. Stereotype Tests:

• BBQ (Bias Benchmark for QA): 58,000 question-answer pairs
• BOLD (Bias in Open-Ended Language Generation)

1. Counterfactual Evaluation:

• Generate responses with demographic swaps
• Example: "Dr. Smith (he/she) recommended..." → compare outputs
• Measure consistency across variations

Toxicity Detection

Tools:

• Perspective API (Google): Toxicity, threat, insult scores
• Detoxify (HuggingFace): Open-source toxicity classifier
• OpenAI Moderation API: Hate, harassment, violence detection

For comprehensive safety evaluation patterns, see references/safety-evaluation.md.

Benchmark Testing

Assess model capabilities using standardized benchmarks.

Standard Benchmarks:

Domain-Specific Benchmarks:

• Medical: MedQA (USMLE), PubMedQA
• Legal: LegalBench
• Finance: FinQA, ConvFinQA

When to Use Benchmarks:

• Comparing multiple models (GPT-4 vs Claude vs Llama)
• Model selection for specific domains
• Baseline capability assessment
• Academic research and publication

Quick Start (lm-evaluation-harness):

pip install lm-eval
 
# Evaluate GPT-4 on MMLU
lm_eval --model openai-chat --model_args model=gpt-4 --tasks mmlu --num_fewshot 5

For detailed benchmark testing patterns, see references/benchmarks.md and scripts/benchmark_runner.py.

Production Evaluation

Monitor and optimize LLM quality in production environments.

A/B Testing

Compare two LLM configurations:

• Variant A: GPT-4 (expensive, high quality)
• Variant B: Claude Sonnet (cheaper, fast)

Metrics:

• User satisfaction scores (thumbs up/down)
• Task completion rates
• Response time and latency
• Cost per successful interaction

Online Evaluation

Real-time quality monitoring:

• Response Quality: LLM-as-judge scoring every Nth response
• User Feedback: Explicit ratings, thumbs up/down
• Business Metrics: Conversion rates, support ticket resolution
• Cost Tracking: Tokens used, inference costs

Human-in-the-Loop

Sample-based human evaluation:

• Random Sampling: Evaluate 10% of responses
• Confidence-Based: Evaluate low-confidence outputs
• Error-Triggered: Flag suspicious responses for review

For production evaluation patterns and monitoring strategies, see references/production-evaluation.md.

Classification Task Evaluation

For tasks with discrete outputs (sentiment, intent, category).

Metrics:

• Accuracy: Correct predictions / Total predictions
• Precision: True positives / (True positives + False positives)
• Recall: True positives / (True positives + False negatives)
• F1 Score: Harmonic mean of precision and recall
• Confusion Matrix: Detailed breakdown of prediction errors

Quick Start (Python):

from sklearn.metrics import accuracy_score, precision_recall_fscore_support
y_true = ["positive", "negative", "neutral", "positive", "negative"]
y_pred = ["positive", "negative", "neutral", "neutral", "negative"]
accuracy = accuracy_score(y_true, y_pred)
precision, recall, f1, _ = precision_recall_fscore_support(y_true, y_pred, average='weighted')
print(f"Accuracy: {accuracy:.2f}")
print(f"Precision: {precision:.2f}")
print(f"Recall: {recall:.2f}")
print(f"F1 Score: {f1:.2f}")