2

I'm testing two different models that differ only in terms of how the dependent variable has been transformed (e.g., Model 1 DV = Y, Model 2, DV = √Y).

I've read that AIC is not appropriate here -- are there any other scoring systems or statistical tests I can use to evaluate the benefits or drawbacks of this transformation?

Nick Cox
  • 56,404
  • 8
  • 127
  • 185
Nick Cain
  • 138

2 Answers2

2

If your goal is predictive accuracy, then you could use cross validation. Just square your predictions for your square root dependent variable model and compare the results to the original untransformed y variable.

This will allow you to compare your models on a wide range of common error metrics such as MSE. In the case of leave one out cross validation, the results are consistent with what you would get if you were able to calculate the AIC of both models.

Jason Sanchez
  • 710
  • 1
    Why not use a MANOVA approach? Stack up the two models and then evaluate the multivariate stats that come with the module(s). Here's a link to a paper about the method: A Primer on Multivariate Analysis of Variance (MANOVA) for Behavioral Scientists by Russell Warne – user78229 Jun 14 '15 at 18:05
  • I suppose I don't quite know how one would apply it in this case. If you have a new way to think about it, I would like learn the technique.

    My (limited) understanding is that MANOVA is simply to test for differences in group means when there is more than one dependent variable. If Nick has many continuous variables that he doesn't want to bin, then I don't know how he would even start using MANOVA.

    Also, I have read that MANOVA shouldn't be applied when the two dependent variables are highly correlated, which would be the case if one is just the square root of the other.

     – Jason Sanchez Jun 15 '15 at 01:09
  • 1
    Your comment raises the question in my mind of the logic behind the two models. Why is a simple transformation of the DV relevant, important or even helpful? What do you hope to learn with this approach? – user78229 Jun 15 '15 at 11:17
2

Try a Box-Cox transformation as maybe some other power is much better than 1 and 0.5. Complete instructions are found in all of the Draper and Smith regression books. It is the use of the geometric mean that allows an appropriate comparison among all such models. (But that means you also have to deal with dependent values of zero.)

There are several good discussions about this and one is Box-Cox transformations in R.

Community
  • 1
JimB
  • 3,734
  • 11
  • 20