How to compare predictive models after cross validation if you have different response units?

Question

Say you have two possible different responses, $Y$ and $Z$, and they are in different units and have different ranges. Your covariates are $X_1$, $X_2$, and $X_3$. Model 1 uses $Y$ as the response, and $X_1$, $X_2$, and $X_3$ as the predictors. Model 2 uses $Z$ as the response, and $X_1$, $X_2$, and $X_3$ as the predictors.

How do you decide which predictive model is better after cross validation if you just need a good model and don't have a preference of $Y$ or $Z$ as the response? $MSE$ doesn't seem to be a good measure of fit after cross validation because it's unit dependent. Any other suggestions besides using cross validation?

Answer 1

In each case you have a measure of fit, the root mean square error. But as you point out, the units are different so there is no way to compare the two models. You could if you have a unit less measure. But I do not see a common form of normalization.