Model for systems that change over time in a random manner

A Markov chain is a special type of **stochastic process**, defined in terms of the

conditional distributions of future states given the present and past states. If the current state only depends on the previous state.

A sequence of random variables 𝑋1, 𝑋2,… is called a **stochastic process** or random process with discrete time parameter.

## Homogeneous Markov Chain

Probability mass function -> Discrete (finite number of different values)

Probability density function -> Continuous (every value in an interval)

Both have cumulative distribution function, where f(x) = P(X<x)

The inverse of the CDF is called quantile function, and it is useful for indicating where the probability is located in a distribution.

# Discrete Distributions

f(x) are all probability mass function(pmf)

## Discrete Uniform Distribution

How to know if the model is working?

We’ve reviewed some of the most representative modern machine learning algorithms in the previous series. These can get us a good start to build supervised predictive models. Only learning about the models is not enough, since **you can’t manage something you can’t measure**. This post will focus on the performance measure, how good is the model and how to understand it.