3

Apologies for the basic question, but I've been reading about cross-validation methods/metrics and I would like to confirm if my understanding is correct:

  1. Some can only be used for regression models (e.g., Root Mean Squared Error, Mean Absolute Error, R2 Error, etc.);
  2. Some can only be used for classification models (e.g., ROC/AUC);
  3. Some can be used for both regression and classification models (e.g., Leave One Out Cross-Validation, K-fold Cross-Validation, etc.)

Thank you in advance.

TWest
  • 347
  • 1
  • 3
  • 9

1 Answers1

5

There are two separate aspects here:

  1. The evaluation metric. Some like the MSE, MAE etc. are more applicable to regression tasks (but may also be usable in classification, e.g., the MSE is the Brier score), others like accuracy, sensitivity etc. (don't use any of these, use proper scoring rules instead) can only be used for classification. There are many pitfalls in choosing evaluation metrics.
  2. The cross validation setup: LOOCV vs k-fold cross validation, or other variants.

You will need to decide which evaluation metric you want to use and how you set up your cross validation.

Stephan Kolassa
  • 123,354