openscientist-io/openscientist/prioritization

Prioritization

When to Use This Skill

• When you have multiple hypotheses and need to choose which to test first
• When you're deciding between exploring new areas vs deepening current investigation
• When iteration budget is running low and you need to focus

The Core Question

"What will provide the most scientific insight right now?"

Prioritization Framework

Option 1: Test a Hypothesis

Pick the hypothesis that bears most directly on the research question, can be tested with the data at hand, and would be informative whether the result is positive or negative. Test it first.

Option 2: Explore Data

When to prioritize exploration:

• ✅ Early iterations (< 10% of budget)
• ✅ You're stuck and need fresh ideas
• ✅ Current hypotheses are all low-priority
• ✅ You haven't looked at key variable types yet

When NOT to explore:

• ❌ Late iterations (> 80% of budget)
• ❌ You have high-priority hypotheses ready to test
• ❌ You're avoiding testing a difficult hypothesis

Option 3: Search Literature

When to prioritize literature:

• ✅ Before testing a hypothesis (to inform design)
• ✅ After unexpected result (to understand mechanism)
• ✅ When hypothesis was rejected (to generate alternatives)
• ✅ Early iterations (build knowledge foundation)

When NOT to search:

• ❌ As procrastination (avoiding analysis)
• ❌ Late iterations when you should be synthesizing
• ❌ You already searched this topic

Option 4: Record/Synthesize Findings

When to prioritize synthesis:

• ✅ You've confirmed a finding but haven't recorded it
• ✅ Late iterations (> 70% of budget)
• ✅ You have 3+ findings that might be related
• ✅ You're approaching iteration limit

When NOT to synthesize:

• ❌ Early iterations (too early to connect dots)
• ❌ You only have 1-2 isolated findings
• ❌ High-priority hypotheses remain untested

Decision Tree

START: What should I do next?
│
├─> Do I have unrecorded findings?
│   YES → Record them now
│   NO → Continue
│
├─> Do I have a promising, testable hypothesis?
│   YES → Test highest-scoring hypothesis
│   NO → Continue
│
├─> Is this early in investigation (<20% iterations)?
│   YES → Explore data OR search literature
│   NO → Continue
│
├─> Am I stuck or confused?
│   YES → Search literature for mechanisms
│   NO → Continue
│
├─> Is this late in investigation (>70% iterations)?
│   YES → Synthesize findings, test remaining high-priority hypotheses
│   NO → Continue
│
└─> Default: Generate new hypotheses OR explore unexplored data

Iteration-Specific Guidance

Early Phase (Iterations 1-10)

Priority:

1. Explore data structure
2. Search literature for domain knowledge
3. Identify major patterns
4. Test broad hypotheses

Example actions:

• Check data distributions and correlations
• Search "hypothermia metabolomics neuroprotection"
• Test for major group differences
• Generate initial hypothesis set

Middle Phase (Iterations 11-40)

Priority:

1. Test mechanistic hypotheses systematically
2. Follow up on positive findings
3. Refine understanding based on negative results
4. Build coherent story

Example actions:

• Test specific pathway hypotheses
• Calculate flux ratios and indices
• Search literature for unexpected results
• Connect findings across analyses

Late Phase (Iterations 41-50)

Priority:

1. Test remaining high-priority hypotheses
2. Record all confirmed findings
3. Synthesize into mechanistic model
4. Identify knowledge gaps

Example actions:

• Finish testing top-scoring hypotheses
• Ensure all findings are documented
• Prepare for final report
• Note what couldn't be answered

Common Prioritization Mistakes

❌ Analysis paralysis

• Don't overthink - pick something and test it
• You can always revise based on results

❌ Chasing rabbits

• Don't get distracted by low-priority tangents
• Stay focused on research question

❌ Perfectionism

• Don't wait for "perfect" hypothesis
• Test and learn from results

❌ Ignoring constraints

• Don't test untestable hypotheses
• Work with the data you have

❌ Linear thinking

• Don't feel obligated to finish one thread before starting another
• Sometimes pivoting is the right move

Example: Prioritization in Action

Situation: Iteration 15 of 50. Current state:

• 2 confirmed findings recorded
• 1 hypothesis rejected (salvage flux)
• 3 pending hypotheses generated
• Unexplored: lipid data

Options:

1. Test H2: "CMP→CDP bottleneck"
2. Test H3: "PUFA enrichment"
3. Test H4: "Sex difference"
4. Explore lipid data
5. Search literature on rejected hypothesis

Decision: Test H2 (highest priority hypothesis)

Rationale:

• High score (100)
• Directly addresses finding (CMP elevation)
• Feasible with current data
• Middle phase → focus on mechanistic hypotheses

Quick Reference

Test hypothesis → When you have a promising hypothesis ready

Explore data → When early (<20%) OR stuck

Search literature → Before testing hypothesis OR after unexpected result

Record findings → When you've confirmed something significant

Synthesize → When late (>70%) OR have 3+ related findings

Key Principle

Don't optimize for perfection. Optimize for scientific insight.

Each iteration should either:

1. Rule something out (negative result)
2. Confirm something (positive result)
3. Generate new ideas (exploration/literature)

If your action doesn't achieve one of these, reconsider.