wingedguardian/genesis-agi/forecasting

Forecasting

Purpose

Make specific, falsifiable predictions with calibrated confidence levels. Track accuracy over time using Brier scores. Apply superforecasting methodology (Tetlock/Good Judgment Project) to any domain — technology trends, project outcomes, market shifts, competitive moves, risk assessment.

When to Use

• User asks for a prediction or forecast on any topic.
• Strategic reflection identifies a decision that depends on uncertain futures.
• Surplus compute is available and a prediction review is due.
• A previously made prediction is approaching its resolution date.
• Deep reflection surfaces a trend worth formally tracking.

Superforecasting Principles

1. Triage — Focus on questions where effort improves accuracy. Ignore

questions that are either trivially knowable or fundamentally unknowable.

1. Fermi decomposition — Break big questions into smaller, estimable

components. "Will X happen?" → "What's the base rate? What's different this time? What signals would I expect to see?"

1. Balance inside and outside views — Start with the reference class

(base rate from historical analogues), then adjust with specific evidence. Never skip the outside view.

1. Update incrementally — Bayesian updating. New evidence shifts

confidence by small amounts, not dramatic swings. Avoid overreaction.

1. Calibration over precision — A well-calibrated 60% is better than

an overconfident 90%. Your 70% predictions should come true ~70% of the time.

1. Distinguish noise from signal — Most new information is noise.

Ask: does this actually change the probability, or does it just feel important because it's recent?

1. Consider contrarian views — Actively seek evidence against your

current position. What must be true for the opposite outcome?

1. Post-mortem every resolution — When a prediction resolves, analyze

WHY you were right or wrong, not just whether. Update process, not just beliefs.

1. Express uncertainty numerically — "Likely" is ambiguous. 70% is not.

Use the probability scale below.

1. Separate confidence from conviction — High confidence (90%) means

high probability. Strong conviction means you've thought deeply. You can have low confidence with strong conviction (you've analyzed it thoroughly and it's genuinely uncertain).

Signal Taxonomy

Signal strength:

• Strong — Multiple independent sources, quantitative, leading, from sources with track record
• Moderate — Single authoritative source, specialist opinion, qualitative
• Weak — Social buzz, anecdote, rumor, single unverified claim

Confidence Scale

Adjustment rules: +/-5-15% per strong signal, +/-2-5% per moderate signal. If gut says 80% but analysis says 55%, trust the analysis.

Cognitive Bias Checklist

Before finalizing ANY prediction, check against these 8 biases:

Reasoning Chain Template

For each prediction, construct:

1. Reference Class (Outside View)

• What is the base rate for this type of event?
• 3-5 historical analogues with outcomes
• Starting probability from base rate alone

2. Specific Evidence (Inside View)

• List each signal with type, strength, and direction
• For each signal: percentage adjustment from base rate
• Net adjustment

3. Synthesis

• Start at base rate
• Apply net adjustment
• State final probability with explicit reasoning

4. Key Assumptions

• What must remain true for this prediction to hold?
• For each assumption: conditional probability shift if violated

5. Resolution Criteria

• Exact date or trigger for resolution
• Specific, observable criteria (not subjective)
• Data source for verification

Brier Score

Brier = (predicted_probability - actual_outcome)^2

Where actual_outcome is 0 (didn't happen) or 1 (happened).

Track cumulative Brier score across all resolved predictions. Review monthly. If cumulative Brier > 0.25, recalibrate methodology.

Contrarian Mode

When explicitly requested or when consensus confidence exceeds 85%:

1. Identify the consensus view and its evidence
2. Search specifically for counter-consensus evidence
3. Ask: "What must be true for the opposite to happen?"
4. If contrarian case is credible (>15% probability), include it
5. Always label contrarian predictions as such alongside consensus

Domain Source Guides

Output Format

prediction_id: <PRED-YYYY-MM-DD-NNN>
created: <YYYY-MM-DD>
domain: <technology | finance | geopolitics | climate | ai_ml | general>
time_horizon: <1_week | 1_month | 3_months | 1_year>
prediction: <specific, falsifiable statement>
confidence: <probability 0.05-0.95>
reasoning_chain:
  reference_class:
    base_rate: <probability>
    analogues:
      - <historical analogue and outcome>
  specific_evidence:
    - signal: <description>
      type: <leading | lagging | base_rate | expert | data | anomaly | structural | sentiment>
      strength: <strong | moderate | weak>
      adjustment: <+/- percentage>
  synthesis: <narrative combining outside and inside views>
  key_assumptions:
    - assumption: <what must hold>
      if_violated: <probability shift>
resolution:
  date: <YYYY-MM-DD>
  criteria: <exact observable condition>
  data_source: <where to verify>
bias_check: <which biases were checked and adjustments made>
status: active | resolved | expired
updates:
  - date: <YYYY-MM-DD>
    old_confidence: <previous>
    new_confidence: <updated>
    reason: <what changed>
resolution_result:
  date: <YYYY-MM-DD>
  outcome: true | false
  evidence: <what happened>
  brier_score: <calculated score>
  lesson: <what to learn from this>

Prediction Review Schedule

• Weekly: Review all active predictions. Update confidence if new evidence.
• Monthly: Calculate cumulative Brier score. Identify calibration drift.
• On resolution: Score immediately. Post-mortem. Update procedures.

References

• Tetlock, P. (2015). Superforecasting: The Art and Science of Prediction
• src/genesis/learning/ — Outcome tracking for Brier score integration
• src/genesis/identity/REFLECTION_STRATEGIC.md — Strategic reflection context