ai4scientist/nano-scientist/dse-loop

DSE Loop: Autonomous Design Space Exploration

Autonomously explore a design space: run → analyze → pick next parameters → repeat, until the objective is met or timeout is reached. Designed for computer architecture and EDA problems.

Context: $ARGUMENTS

Safety Rules — READ FIRST

NEVER do any of the following:

• sudo anything
• rm -rf, rm -r, or any recursive deletion
• rm any file you did not create in this session
• Overwrite existing source files without reading them first
• git push, git reset --hard, or any destructive git operation
• Kill processes you did not start

If a step requires any of the above, STOP and report to the user.

Constants (override via $ARGUMENTS)

Override inline: /dse-loop "task desc — timeout: 4h, max_iterations: 100, patience: 15"

Typical Use Cases

Workflow

Phase 0: Parse Task & Setup

1. Parse $ARGUMENTS to extract:

• Program: what to run (command, script, or Makefile target)
• Parameter space: which knobs to tune and their ranges/options (may be incomplete — see step 2)
• Objective metric: what to optimize (and how to extract it from output)
• Constraints: hard limits that must not be violated (e.g., timing must close)
• Timeout: wall-clock budget
• Success criteria: when is the result "good enough" to stop early?

1. Infer missing parameter ranges — If the user provides parameter names but NOT ranges/options, you MUST infer them before exploring:

a. Read the source code — search for the parameter names in the codebase:

• Look for argparse/click definitions, config files, Makefile variables, module parameters, #define, parameter (SystemVerilog), localparam, etc.
• Extract defaults, types, and any comments hinting at valid values

b. Apply domain knowledge to set reasonable ranges:

c. Start conservative — begin with 3-5 values per parameter. Expand range later if the best result is at a boundary.

d. Log inferred ranges — write the inferred parameter space to dse_results/inferred_params.md so the user can review: ```markdown # Inferred Parameter Space

```

e. Boundary expansion — during the search, if the best result is at the min or max of a range, automatically extend that range by one step in that direction (but log the extension).

1. Read the project to understand:

• How to run the program
• Where results are produced (stdout, log files, reports)
• How to parse the objective metric from output
• Current/baseline configuration (if any)

1. Create working directory: dse_results/ in project root

• dse_results/dse_log.csv — one row per design point
• dse_results/DSE_REPORT.md — final report
• dse_results/DSE_STATE.json — state for recovery
• dse_results/inferred_params.md — inferred parameter space (if ranges were not provided)
• dse_results/configs/ — config files for each run
• dse_results/outputs/ — raw output for each run

1. Write a parameter extraction script (dse_results/parse_result.py or similar) that takes a run's output and returns the objective metric as a number. Test it on a baseline run first.

1. Run baseline (iteration 0): run the program with default/current parameters. Record the baseline metric. This is the point to beat.

Phase 1: Initial Exploration

Goal: Quickly survey the space to understand which parameters matter most.

Strategy: Latin Hypercube Sampling or structured sweep of key parameters.

1. Pick 5-10 diverse design points that span the parameter ranges
2. Run them (in parallel if independent, via background processes or sequential)
3. Record all results in dse_log.csv:

`` iteration,param1,param2,...,metric,constraint_met,timestamp,notes 0,default,default,...,baseline_val,yes,2026-03-13T10:00:00,baseline 1,val1a,val2a,...,result1,yes,2026-03-13T10:05:00,initial sweep ... ``

1. Analyze: which parameters have the most impact on the objective?
2. Narrow the search to the most sensitive parameters

Phase 2: Directed Search

Goal: Converge toward the optimum by making informed choices.

Strategy: Adaptive — pick the approach that fits the problem:

• Few parameters (≤3): Fine-grained grid search around the best region from Phase 1
• Many parameters (>3): Coordinate descent — optimize one parameter at a time, holding others at current best
• Binary/categorical params: Enumerate promising combinations
• Continuous params: Binary search or golden section between best neighbors
• Multi-objective: Track Pareto frontier, explore along the front

For each iteration:

1. Select next design point based on results so far:

• Look at the trend: which direction improves the metric?
• Avoid re-running configurations already evaluated
• Balance exploration (untested regions) vs exploitation (near current best)

1. Modify parameters: edit config file, command-line args, or source constants

1. Run the program: execute and capture output

1. Parse results: extract the objective metric and check constraints

1. Log to `dse_log.csv`: append the new row

1. Check stopping conditions:

• Timeout reached? → stop
• Max iterations reached? → stop
• Patience exhausted (no improvement in N iterations)? → stop
• Success criteria met (metric is "good enough")? → stop
• Constraint violation pattern detected? → adjust search bounds

1. Update `DSE_STATE.json`:

``json { "iteration": 15, "status": "in_progress", "best_metric": 1.23, "best_params": {"cache_size": 32768, "assoc": 4, "pipeline_width": 2}, "total_iterations": 15, "start_time": "2026-03-13T10:00:00", "timeout": "2h", "patience_counter": 3 } ``

1. Decide next step → back to step 1

Phase 3: Refinement (if time allows)

If the search converged and there's still time budget:

1. Local perturbation: try ±1 step on each parameter from the best point
2. Sensitivity analysis: which parameters can be relaxed without hurting the metric?
3. Constraint boundary: if a constraint is nearly binding, explore near-feasible points

Phase 4: Report

Write dse_results/DSE_REPORT.md:

# Design Space Exploration Report
 
**Task**: [description]
**Date**: [start] → [end]
**Total iterations**: N
**Wall-clock time**: X hours Y minutes
 
## Objective
-**Metric**: [what was optimized]
-**Direction**: minimize / maximize
-**Baseline**: [value]
-**Best found**: [value] ([improvement]% better than baseline)
 
## Best Configuration
| Parameter | Baseline | Best |
|-----------|----------|------|
| param1    | default  | best_val |
| param2    | default  | best_val |
| ...       | ...      | ... |
 
## Search Trajectory
| Iteration | param1 | param2 | ... | Metric | Notes |
|-----------|--------|--------|-----|--------|-------|
| 0 (baseline) | ... | ... | ... | ... | baseline |
| 1 | ... | ... | ... | ... | initial sweep |
| ... | ... | ... | ... | ... | ... |
| N (best) | ... | ... | ... | ... | ★ best |
 
## Parameter Sensitivity
-**param1**: [high/medium/low impact] — [brief explanation]
-**param2**: [high/medium/low impact] — [brief explanation]
 
## Pareto Frontier (if multi-objective)
[Table or description of non-dominated points]
 
## Stopping Reason
[timeout / max_iterations / patience / success_criteria_met]
 
## Recommendations
-[actionable insights from the exploration]
-[which parameters matter most]
-[suggested follow-up explorations]

Also generate a summary plot if matplotlib is available:

• Convergence curve (metric vs iteration)
• Parameter sensitivity bar chart
• Pareto frontier scatter (if multi-objective)

State Recovery

If the context window compacts mid-run, the loop recovers from DSE_STATE.json + dse_log.csv:

1. Read DSE_STATE.json for current iteration, best params, patience counter
2. Read dse_log.csv for full history
3. Resume from next iteration

Key Rules

• Work AUTONOMOUSLY — do not ask the user for permission at each iteration
• Every run must be logged — even failed runs, constraint violations, errors. The log is the ground truth.
• Never re-run an identical configuration — check dse_log.csv before each run
• Respect the timeout — check elapsed time before starting a new iteration. If the next run is likely to exceed the timeout, stop and report.
• Parse metrics programmatically — write a parsing script, don't eyeball logs
• Keep raw outputs — save each run's full output in dse_results/outputs/iter_N/
• Constraint violations are not improvements — a design point that violates constraints is never "best", regardless of the metric
• If a run crashes, log the error, skip that point, and continue with the next
• If the same crash repeats 3 times with different configs, stop and report the issue

Example Invocations

# Minimal — just name the parameters, let the agent figure out ranges
/dse-loop "Run gem5 mcf benchmark. Tune: L1D_SIZE, L2_SIZE, ROB_ENTRIES. Objective: maximize IPC. Timeout: 3h"
 
# Partial — some ranges given, some not
/dse-loop "Run make synth. Tune: CLOCK_PERIOD [5ns, 4ns, 3ns, 2ns], FLATTEN, ABC_SCRIPT. Objective: minimize area at timing closure. Timeout: 1h"