LLM Evaluation & Testing
Comprehensive guide to evaluating and testing LLM applications including prompt testing, output validation, hallucination detection, benchmark creation, A/B testing, and quality metrics.
Quick Reference
When to use this skill:
- Testing LLM application outputs
- Validating prompt quality and consistency
- Detecting hallucinations and factual errors
- Creating evaluation benchmarks
- A/B testing prompts or models
- Implementing continuous evaluation (CI/CD)
- Measuring retrieval quality (for RAG)
- Debugging unexpected LLM behavior
Metrics covered:
- Traditional: BLEU, ROUGE, BERTScore, Perplexity
- LLM-as-Judge: GPT-4 evaluation, rubric-based scoring
- Task-specific: Exact match, F1, accuracy, recall
- Quality: Toxicity, bias, coherence, relevance
Part 1: Evaluation Fundamentals
The LLM Evaluation Challenge
Why LLM evaluation is hard:
- Subjective quality - "Good" output varies by use case
- No single ground truth - Multiple valid answers
- Context-dependent - Same output good/bad in different scenarios
- Expensive to label - Human evaluation doesn't scale
- Adversarial brittleness - Small prompt changes = large output changes
Solution: Multi-layered evaluation
Layer 1: Automated Metrics (fast, scalable)
↓
Layer 2: LLM-as-Judge (flexible, nuanced)
↓
Layer 3: Human Review (gold standard, expensive)
Evaluation Dataset Structure
python
from dataclasses import dataclass
from typing import List, Optional
@dataclass
class EvalExample:
"""Single evaluation example."""
input: str # User input / prompt
expected_output: Optional[str] # Gold standard (if exists)
context: Optional[str] # Additional context (for RAG)
metadata: dict # Category, difficulty, etc.
@dataclass
class EvalResult:
"""Evaluation result for one example."""
example_id: str
actual_output: str
scores: dict # {'metric_name': score}
passed: bool
failure_reason: Optional[str]
# Example dataset
eval_dataset = [
EvalExample(
input="What is the capital of France?",
expected_output="Paris",
context=None,
metadata={'category': 'factual', 'difficulty': 'easy'}
),
EvalExample(
input="Explain quantum entanglement",
expected_output=None, # No single answer
context=None,
metadata={'category': 'explanation', 'difficulty': 'hard'}
)
]
Part 2: Traditional Metrics
Metric 1: Exact Match (Simplest)
python
def exact_match(predicted: str, expected: str, case_sensitive: bool = False) -> float:
"""
Binary metric: 1.0 if match, 0.0 otherwise.
Use for: Classification, short answers, structured output
Limitations: Too strict for generation tasks
"""
if not case_sensitive:
predicted = predicted.lower().strip()
expected = expected.lower().strip()
return 1.0 if predicted == expected else 0.0
# Example
score = exact_match("Paris", "paris") # 1.0
score = exact_match("The capital is Paris", "Paris") # 0.0
Metric 2: ROUGE (Recall-Oriented)
python
from rouge_score import rouge_scorer
def compute_rouge(predicted: str, expected: str) -> dict:
"""
ROUGE metrics for text overlap.
ROUGE-1: Unigram overlap
ROUGE-2: Bigram overlap
ROUGE-L: Longest common subsequence
Use for: Summarization, translation
Limitations: Doesn't capture semantics
"""
scorer = rouge_scorer.RougeScorer(['rouge1', 'rouge2', 'rougeL'], use_stemmer=True)
scores = scorer.score(expected, predicted)
return {
'rouge1': scores['rouge1'].fmeasure,
'rouge2': scores['rouge2'].fmeasure,
'rougeL': scores['rougeL'].fmeasure
}
# Example
scores = compute_rouge(
predicted="Paris is the capital of France",
expected="The capital of France is Paris"
)
# {'rouge1': 0.82, 'rouge2': 0.67, 'rougeL': 0.82}
Metric 3: BERTScore (Semantic Similarity)
python
from bert_score import score as bert_score
def compute_bertscore(predicted: List[str], expected: List[str]) -> dict:
"""
Semantic similarity using BERT embeddings.
Better than ROUGE for:
- Paraphrases
- Semantic equivalence
- Generation quality
Returns: Precision, Recall, F1
"""
P, R, F1 = bert_score(predicted, expected, lang="en", verbose=False)
return {
'precision': P.mean().item(),
'recall': R.mean().item(),
'f1': F1.mean().item()
}
# Example
scores = compute_bertscore(
predicted=["The capital of France is Paris"],
expected=["Paris is France's capital city"]
)
# {'precision': 0.94, 'recall': 0.91, 'f1': 0.92}
Metric 4: Perplexity (Model Confidence)
python
import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer
def compute_perplexity(text: str, model_name: str = "gpt2") -> float:
"""
Perplexity: How "surprised" is the model by this text?
Lower = More likely/fluent
Use for: Fluency, naturalness
Limitations: Doesn't measure correctness
"""
model = GPT2LMHeadModel.from_pretrained(model_name)
tokenizer = GPT2Tokenizer.from_pretrained(model_name)
inputs = tokenizer(text, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs, labels=inputs["input_ids"])
loss = outputs.loss
perplexity = torch.exp(loss).item()
return perplexity
# Example
ppl = compute_perplexity("Paris is the capital of France") # Low (fluent)
ppl2 = compute_perplexity("Capital France the is Paris of") # High (awkward)
Part 3: LLM-as-Judge Evaluation
Pattern 1: Rubric-Based Scoring
python
from openai import OpenAI
client = OpenAI()
EVALUATION_PROMPT = """
You are an expert evaluator. Score the assistant's response on a scale of 1-5 for each criterion:
**Criteria:**
1. **Accuracy**: Is the information factually correct?
2. **Completeness**: Does it fully answer the question?
3. **Clarity**: Is it easy to understand?
4. **Conciseness**: Is it appropriately brief?
**Response to evaluate:**
{response}
**Expected answer (reference):**
{expected}
Provide scores in JSON format:
{{
"accuracy": <1-5>,
"completeness": <1-5>,
"clarity": <1-5>,
"conciseness": <1-5>,
"reasoning": "Brief explanation"
}}
"""
def llm_judge_score(response: str, expected: str) -> dict:
"""
Use GPT-4 as judge with rubric scoring.
Pros: Flexible, nuanced, scales well
Cons: Costs $, potential bias, slower
"""
prompt = EVALUATION_PROMPT.format(response=response, expected=expected)
completion = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": prompt}],
response_format={"type": "json_object"}
)
import json
scores = json.loads(completion.choices[0].message.content)
return scores
# Example
scores = llm_judge_score(
response="Paris is the capital of France, located in the north-central part of the country.",
expected="Paris"
)
# {'accuracy': 5, 'completeness': 5, 'clarity': 5, 'conciseness': 3, 'reasoning': '...'}
Pattern 2: Binary Pass/Fail Evaluation
python
PASS_FAIL_PROMPT = """
Evaluate if the assistant's response is acceptable.
**Question:** {question}
**Response:** {response}
**Criteria:** {criteria}
Return ONLY "PASS" or "FAIL" followed by a one-sentence reason.
"""
def binary_eval(question: str, response: str, criteria: str) -> tuple[bool, str]:
"""
Simple pass/fail evaluation.
Use for: Unit tests, regression tests, CI/CD
"""
prompt = PASS_FAIL_PROMPT.format(
question=question,
response=response,
criteria=criteria
)
completion = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": prompt}],
temperature=0.0 # Deterministic
)
result = completion.choices[0].message.content
passed = result.startswith("PASS")
reason = result.split(":", 1)[1].strip() if ":" in result else result
return passed, reason
# Example
passed, reason = binary_eval(
question="What is the capital of France?",
response="The capital is Paris",
criteria="Response must mention Paris"
)
# (True, "Response correctly identifies Paris as the capital")
Pattern 3: Pairwise Comparison (A/B Testing)
python
PAIRWISE_PROMPT = """
Compare two responses to the same question. Which is better?
**Question:** {question}
**Response A:**
{response_a}
**Response B:**
{response_b}
**Criteria:** {criteria}
Return ONLY: "A", "B", or "TIE", followed by a one-sentence explanation.
"""
def pairwise_comparison(
question: str,
response_a: str,
response_b: str,
criteria: str = "Overall quality, accuracy, and helpfulness"
) -> tuple[str, str]:
"""
A/B test two responses.
Use for: Prompt engineering, model comparison
"""
prompt = PAIRWISE_PROMPT.format(
question=question,
response_a=response_a,
response_b=response_b,
criteria=criteria
)
completion = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": prompt}],
temperature=0.0
)
result = completion.choices[0].message.content
winner = result.split()[0] # "A", "B", or "TIE"
reason = result.split(":", 1)[1].strip() if ":" in result else result
return winner, reason
# Example
winner, reason = pairwise_comparison(
question="Explain quantum computing",
response_a="Quantum computers use qubits instead of bits...",
response_b="Quantum computing is complex. It uses quantum mechanics."
)
# ("A", "Response A provides more detail and explanation")
Part 4: Hallucination Detection
Method 1: Grounding Check
python
def check_grounding(response: str, context: str) -> dict:
"""
Verify response is grounded in provided context.
Critical for RAG systems.
"""
GROUNDING_PROMPT = """
Context: {context}
Response: {response}
Is the response fully supported by the context? Answer with:
- "GROUNDED": All claims supported
- "PARTIALLY_GROUNDED": Some claims unsupported
- "NOT_GROUNDED": Contains unsupported claims
List any unsupported claims.
"""
prompt = GROUNDING_PROMPT.format(context=context, response=response)
completion = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": prompt}]
)
result = completion.choices[0].message.content
status = result.split("\n")[0]
unsupported = [line for line in result.split("\n")[1:] if line.strip()]
return {
'grounding_status': status,
'unsupported_claims': unsupported,
'is_hallucination': status != "GROUNDED"
}
Method 2: Factuality Check (External Verification)
python
def check_factuality(claim: str, use_search: bool = True) -> dict:
"""
Verify factual claims using external sources.
Options:
1. Web search + verification
2. Knowledge base lookup
3. Cross-reference with trusted source
"""
if use_search:
# Use web search to verify
from tavily import TavilyClient
tavily = TavilyClient(api_key="your-key")
# Search for evidence
results = tavily.search(claim, max_results=3)
# Ask LLM to verify based on search results
VERIFY_PROMPT = """
Claim: {claim}
Search results:
{results}
Is the claim supported by these sources? Answer: TRUE, FALSE, or UNCERTAIN.
Explanation:
"""
prompt = VERIFY_PROMPT.format(
claim=claim,
results="\n\n".join([r['content'] for r in results])
)
completion = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": prompt}]
)