Covariate Shift

Definition

When the distribution of the input features \(\mathbf{X}\) changes between the training and test sets, while the conditional distribution \(p(\mathbf{y}| \mathbf{X})\) stays the same.

Example

Gathering images throughout the year. During winter season, the images will usually be darker, and during summer season, the images will usually be lighter.

You train a model on images that are lighter, but test it on darker images.

This results in the model implicitly putting more weight on lighter images, implying that it is more innacurate when given a darker image.

In other terms, p(y = has person | x = room snapshot) is the same throughout the year, but p(x = room snapshot) changes.