Cross Validation¶
Definition¶
An approach to obtain a biased yet reasonable estimation on the performance of the model trained on the whole dataset.
We do NOT split the data into train and test splits! This is due to us wanting to have the model be trained on as much quality data as possible. We use the same dataset to measure \(GE(f_{\cal D})\)
We are able to estimate \(GE(f_{\cal D})\) by measuring many \(f_{k}\)'s trained on subsets of the dataset \(\cal D\). Note that the trained \(f_{k}\)'s have independent parameters to the actual \(f\) trained on the whole dataset
k-fold cross validation and repeated random subsampling only differ on how the subsets are created.
Variants of Cross Validation¶
k-fold Cross Validation Repeated Random Subsampling