# Markov Chains

Model for systems that change over time in a random manner

A Markov chain is a special type of **stochastic process**, defined in terms of the

conditional distributions of future states given the present and past states. If the current state only depends on the previous state.

A sequence of random variables 𝑋1, 𝑋2,… is called a **stochastic process** or random process with discrete time parameter.

## Homogeneous Markov Chain

The initial probability vector **v** gives the distribution of the state at time 1. **P^n** gives the probabilities of transitions over n time periods. **vP^n** gives the distribution of the state at time n + 1.

A stationary distribution is a probability vector v such that **vP =v**.