Sparse Solution

Definition

Refers to a solution vector \(\mathbf{w}\), where many elements are exactly zero. This implies that many features are redundant.

Sparsity leads to simpler, more interpretable models, since you focus on the most important features.

Sparsity also leads to less computation and acts as a regularizer to prevent overfitting (ℓ1 Regularization).

References

Proximal Update Gradient Descent with Lasso ℓ1 Regularization