Sparse Coding¶
Definition¶
The input data is expanded into a sparse representation. Here we have \(p \gg d\), which results in a large dictionary \(\mathbf{D} \in \mathbb{R}^{p \times d}\). This dictionary is huge, with many rows.
However, we only use a couple rows of the dictionary when producing the input \(\mathbf{x}\). This means that our \(\mathbf{h}\) is sparse with only a few non-zero entries such that:
\[
\mathbf{x} \approx \mathbf{hD}
\]
