Lab notes · Free tool
Markov chain calculator
Enter a transition matrix. Get the exact stationary distribution, the full mean first passage time matrix, mean recurrence times, and the Kemeny constant. Solved with the fundamental matrix of Kemeny and Snell, right in your browser.
Transition matrix P
Row i, column j = probability of jumping from state i to state j. Each row must sum to 1.
| → 1 | → 2 | Σ | |
|---|---|---|---|
| 1 | 1.00 | ||
| 2 | 1.00 |
Examples
Kemeny constant
K = 3.50
Expected steps to reach a π-random target, identical from every start.
Stationary distribution
Long-run fraction of time in each state. Solves πP = π.
Mean first passage times m(i → j)
Expected steps to reach column state j for the first time, starting from row state i. Diagonal shows recurrence times.
| → 1 | → 2 | |
|---|---|---|
| 1 | 4.00 | 3.33 |
| 2 | 10.00 | 1.33 |
Expected steps to return to a state. Always exactly 1/π.
01 · The equations
Mean first passage time
The expected number of steps to reach state j for the first time from state i satisfies a one-step recursion: you always spend one step, and if it lands on any state k other than the target, you still owe the expected time from k:
Fixing the target j turns this into a linear system with one equation per starting state. The calculator solves it via the fundamental matrix Z = (I − P + Π)⁻¹, where every row of Π is the stationary vector, using the identity m(i, j) = (z(j, j) − z(i, j)) / π(j).
02 · The reciprocal law
Recurrence times and π
On the diagonal, m(j, j) is the mean recurrence time: how long the chain takes to return to j. Kemeny and Snell prove it is exactly the reciprocal of the stationary probability:
In the calculator the diagonal cells are highlighted: check that each one equals 1 divided by the π shown above it.
03 · Grouping states
The lumpability condition
Sometimes you want to merge states into blocks (all "sunny" variants into one weather state, all "waiting" substates into one queue state) and still have a Markov chain. Kemeny and Snell showed exactly when that works: for every pair of blocks A and B, the probability of moving from a state of A into block B, summed over all states of B, must be the same for every state in A. If any state of A feels block B differently than its blockmates, the lumped process loses the Markov property.
You can test it in the calculator: load "Two communities" and check that states 1 and 2 send the same total probability (0.05) into the block {3, 4}. That chain lumps cleanly into a 2-state chain between communities.
04 · The book
Finite Markov Chains (1960)
All of the above comes from John G. Kemeny and J. Laurie Snell, Finite Markov Chains, first published by Van Nostrand in 1960 and reprinted by Springer in 1976 in the Undergraduate Texts in Mathematics series. It remains the cleanest treatment of fundamental matrices, first passage times, and lumpability. The constant K that this calculator reports also comes from that book, and it has a property strange enough that we built it a page of its own.
Kemeny's constant, visualized in 3DFAQ
Common questions
What is the mean first passage time equation for a Markov chain?+
The mean first passage times satisfy m_ij = 1 + Σ over k ≠ j of p_ik · m_kj: one step is always taken, and if that step lands on k instead of the target j, the expected remaining time is m_kj. Fixing j gives a linear system whose solution is column j of the passage-time matrix. This calculator solves it through the fundamental matrix Z = (I − P + Π)⁻¹, using m_ij = (z_jj − z_ij) / π_j.
How do I compute the stationary distribution of a Markov chain?+
Solve the linear system πP = π together with the normalization Σ π_j = 1. For an irreducible finite chain the solution is unique and strictly positive. Enter any transition matrix above and the calculator solves the system exactly.
Why does the mean recurrence time equal 1/π (Kemeny and Snell)?+
Because visits to state j happen, in the long run, a fraction π_j of all steps. If a fraction π_j of your steps are arrivals at j, the average gap between arrivals must be 1/π_j steps. Formally, π_j = 1/m_jj is proven in Kemeny and Snell, Finite Markov Chains (1960; Springer reprint 1976).
What is the Kemeny and Snell lumpability condition?+
A chain is lumpable with respect to a partition of its states when, for every pair of blocks A and B, the total probability of jumping from a state in A into block B (the sum of transition probabilities into all states of B) is the same for every state in A. When this holds, the process on blocks is itself a Markov chain. Kemeny and Snell give this necessary and sufficient condition in Chapter VI of Finite Markov Chains.
What does this Markov chain calculator compute?+
From any transition matrix up to 6 states it computes the exact stationary distribution π, the full matrix of mean first passage times m_ij, the mean recurrence times m_jj = 1/π_j, and the Kemeny constant K. It validates that rows sum to 1 and that the chain is irreducible. Everything runs in your browser; no data is sent anywhere.
Kemeny Studio builds AI agents for enterprise operations. Pages like this one exist because we share a name with the mathematician and an obsession with systems that converge.