Latent Factors and Factor Analysis

Definition

Given an input feature vector \(\mathbf{x} \in \mathbb{R}^{1 \times d}\), you try to find additional hidden information that proves useful to your current situation.

These are called latent factors: \(\mathbf{h} \in \mathbb{R}^{1\times p}\), and we want to find these factors such that we can create a linear weighting with a coefficient matrix \(\mathbf{D} \in \mathbb{R} ^{p\times d}\) such that:

\[ \mathbf{x} = \sum_{k=0}^p h_{k} \mathbf{D}_{k} \]

Sometimes \(\mathbf{D}\) is called a dictionary.

---

How do we find the dictionary?

Finding the dictionary to get latent factors

Example

Let \(\mathbf{h} = [0, 0.5, 0.3, 0.2, \dots , 0]\), then the approximation is 50% dictionary item 2, and 30% dictionary item 3, and 20% dictionary item 4.