combined-ai/isplay/isplay-analysis

Investigate Agent Decisions With isplay

Goal

Explain an agent decision from captured evidence and controlled replay. Do not infer intent from the final answer alone and do not claim causal proof.

Core Loop

1. Establish ISPLAY_API_URL, ISPLAY_PROJECT_ID, and runId. If runId is absent, use isplay runs list --project "$ISPLAY_PROJECT_ID" and select the run that matches the user's task; do not silently guess between plausible runs.
2. Run isplay investigate <runId>. Treat its state as authoritative:

• needs_capture: stop the analysis and report the fields in missingEvidence.
• needs_checkpoint: stop counterfactual work and ask for a newly captured run with a checkpoint before the decision.
• ready_to_test: write the observed baseline, then test hypotheses.
• replay_in_progress: wait or inspect the job; do not interpret partial output.
• needs_fixture: inspect the open requirement. Add a fixture only with explicit provenance and authority.
• replay_failed: report the failure; do not turn it into a decision explanation.
• ready_to_report: write the evidence-bounded report.

1. Form at most three hypotheses. Each must change one prompt/context item, tool argument/result, or checkpoint state field and predict one observable effect.
2. Run one hypothesis batch with recorded-only model policy, pause-for-fixture tool policy, and three repetitions. Never enable live models or tools unless the user explicitly asks and the environment authorizes it.
3. Resolve only the fixture requirements necessary for the selected hypotheses. Never describe an analyst, AI, or simulator fixture as observed or live output.
4. Run isplay investigate <runId> --replay <replayId> for each candidate replay used in the conclusion.
5. Report observed baseline facts separately from replay-backed effects. Include IDs, first divergence, comparability, fixture provenance, validity labels, residual uncertainty, and the smallest useful next test.

Evidence Rules

• Cite IDs and hashes from the bundle; do not reconstruct missing prompt or tool content.
• Prefer the earliest checkpoint before the suspected decision.
• Use expectedBaseHash when changing captured context.
• Reject broad prompt rewrites and multi-variable interventions as first tests.
• Treat a single trial, fixture-dependent result, nondeterministic result, or non-comparable trace as inconclusive.
• Phrase replay findings as “under this intervention” or “under this fixture,” never as causal certainty.

References

• Read references/recipes.md before creating experiments or fixtures; it contains exact JSON and commands.
• Read references/trace-reading.md when interpreting context, first divergence, comparability, or fixture provenance.
• Read references/hypothesis-playbook.md when hypotheses are not obvious from the baseline.
• Read references/report-template.md immediately before writing the final report.
• Read references/platform-map.md only when an API/CLI field or object is unclear.
• Read references/tutorial.md for a complete worked sequence.