Model for systems that change over time in a random manner

A Markov chain is a special type of stochastic process, defined in terms of the
conditional distributions of future states given the present and past states. If the current state only depends on the previous state.

A sequence of random variables 𝑋1, 𝑋2,… is called a stochastic process or random process with discrete time parameter.

Homogeneous Markov Chain



Probability mass function -> Discrete (finite number of different values)

Probability density function -> Continuous (every value in an interval)

Both have cumulative distribution function, where f(x) = P(X<x)

The inverse of the CDF is called quantile function, and it is useful for indicating where the probability is located in a distribution.

Discrete Distributions

f(x) are all probability mass function(pmf)

Discrete Uniform Distribution



How to know if the model is working?

We’ve reviewed some of the most representative modern machine learning algorithms in the previous series. These can get us a good start to build supervised predictive models. Only learning about the models is not enough, since you can’t manage something you can’t measure. This post will focus on the performance measure, how good is the model and how to understand it.


Mean Squared Error (MSE)