In today’s class professor explained about the sampling methods, the sampling methods are two types cross-validation and the bootstrap. These methods refit a model of interest to sample formed from the training set, in order to obtain additional information about the fitted model.
Test error: The test error is the average error that results from using a statistical learning method to predict the response on anew observation, on that was not used in training the method.
Training error: The training error can be easily calculated by applying the statistical learning method to the observations used in its training, But the training error rate often is quite different from the test error rate, and in particular the former can dramatically underestimate the latter.
The Validation Set Approach is a valuable method for estimating test error, but it comes with certain limitations stemming from variability and the risk of potential model underfitting. Caution should be exercised when interpreting its findings, particularly when deploying the model on the entire dataset