raveeshbhalla/dspy-gepa-logger/migrate-gepa

Migrate DSPy GEPA to gepa-observable

Migrate existing DSPy GEPA code to gepa-observable for integrated observability: real-time dashboard, custom observer callbacks, and LM call capture working together.

Quick Migration

Step 1: Install gepa-observable

pip install dspy-gepa-logger

Step 2: Change Import

# Before
from dspy.teleprompt import GEPA
# After
from gepa_observable import GEPA

Step 3: Add Observability (Full System)

from gepa_observable import GEPA
optimizer = GEPA(
    metric=my_metric,
    auto="medium",
    # Observability system:
    server_url="http://your-server:3000",  # Dashboard integration
    project_name="My Project",
    capture_lm_calls=True,                  # LM call logging
    capture_stdout=True,                    # Console capture
    verbose=True,                           # LoggingObserver
    observers=[MyCustomObserver()],         # Custom callbacks
)

Step 4: Everything Else Stays the Same

• Metric functions (5-arg signature already required by DSPy GEPA)
• Program definitions (ChainOfThought, ReAct, etc.)
• Training/validation data format
• Budget params (auto, max_full_evals, max_metric_calls)
• compile() call: optimizer.compile(student=program, trainset=train, valset=val)

The Observability System

gepa-observable provides an integrated observability system where all components work together:

All observers receive the same 8 lifecycle events:

1. SeedValidationEvent - Initial validation scores
2. IterationStartEvent - Each optimization iteration
3. MiniBatchEvalEvent - Minibatch evaluations
4. ReflectionEvent - Proposed prompt changes
5. AcceptanceDecisionEvent - Accept/reject decisions
6. ValsetEvalEvent - Full validation evaluations
7. MergeEvent - Candidate merge operations
8. OptimizationCompleteEvent - Final results

Custom Observer Example

class MyObserver:
    def on_seed_validation(self, event):
        avg = sum(event.valset_scores.values()) / len(event.valset_scores)
        print(f"Seed score: {avg:.2%}")
    def on_iteration_start(self, event):
        print(f"Iteration {event.iteration}, parent score: {event.parent_score:.2%}")
    def on_reflection(self, event):
        for comp, text in event.proposed_texts.items():
            print(f"Proposing for {comp}: {text[:100]}...")
    def on_valset_eval(self, event):
        if event.is_new_best:
            print(f"NEW BEST: {event.valset_score:.2%}")
    def on_optimization_complete(self, event):
        print(f"Done! Best: {event.best_score:.2%} in {event.total_iterations} iters")
# Use with other observers - they all work together
optimizer = GEPA(
    metric=my_metric,
    auto="medium",
    server_url="http://localhost:3000",
    observers=[MyObserver()],
    verbose=True,  # Also keep LoggingObserver
)

Migration Checklist

For Notebooks

1. [ ] Install: pip install dspy-gepa-logger
2. [ ] Change import: from gepa_observable import GEPA
3. [ ] Add server_url for dashboard
4. [ ] Add custom observers if needed
5. [ ] Run cells in order - API is 100% compatible

For Scripts

1. [ ] Install: pip install dspy-gepa-logger
2. [ ] Find all from dspy.teleprompt import GEPA
3. [ ] Replace import
4. [ ] Add observability parameters to GEPA constructor
5. [ ] No other changes needed

References

• references/api-reference.md - Complete parameter docs, observer protocol, all event types
• references/examples.md - Full before/after code examples