Mixture Models¶
Definition¶
A mixture model assumes the data was sampled from a mixture of several probability distributions, weighted by some \(w_{k}\), for the \(k\)'th distribution.
Mathematically, mixture models are defined as:
\[
p(x) = \sum_{k=1}^K w_{k} \cdot p_{k}(x | \theta_{k})
\]
where: \(w_{k}\) is the weighting on the \(k\)'th distribution \(p_{k}\) is the \(k\)'th distribution \(\theta_{k}\) are the parameters corresponding to the \(k\)'th distribution \(x\) is the thing we sampled.
Note that summation of the weights must add up to 1 since they form a Simplex:
\[
\sum_{k=1}^K w_{k}= 1
\]
Since the weights are positive and sum to one, this means that: All mixture distributions correspond to a convex combination of the mixture components.