Markov Chains

How the model works

A Markov chain has states and a transition matrix P, where Pᵢⱼ is the probability of moving from state i to state j in one step. Each row sums to 1. The future depends only on the present state, not the history (the Markov property).

The distribution after n steps is π₀Pⁿ. For a chain where every state can reach every other and there is no fixed cycle, this converges to a unique stationary distribution π with πP = π, whatever the starting state. The simulator finds π by repeatedly applying P.

The bars compare the fraction of days spent in each state so far with π. They agree better and better as the walk gets longer (the ergodic theorem). Mean return time to a state is 1/πᵢ.

Illustrative weather model with editable probabilities, not real climate data. Rows are normalised automatically if they do not sum to 1.

Guided experiments

1. Default weather. Run 1000 days. Prediction: the visit fractions converge to the stationary distribution shown by the outline bars, whatever the starting day. Run the chain
2. Sticky rain. Make rain persist (rain → rain = 0.9). Prediction: long rainy spells, and the stationary share of rain rises substantially. Sticky rain
3. Absorbing state. Rain → rain = 1. Prediction: once it rains it rains forever, so π puts 100% on rain. Absorbing rain

Challenge

For a two-state chain with P(sun → rain) = 0.2 and P(rain → sun) = 0.4, what fraction of days are rainy in the long run?

References

• Grinstead & Snell, Introduction to Probability, ch. 11 “Markov Chains” (open textbook).