Square-root transform dependent variable to gaussian for ML problem

Question

I have a ML problem with dependent y-variables that are skew-symmetric to the right. I perform the transformation $\sqrt y$ on my dependent variable before running the ML algorithm, and then make a set of predictions on my test data. These are transformed predictions, and I would like to transform them back from $\sqrt y$ to $y$. However, from my statistics class I know that:

$$E[y^\frac 12]^2 \neq E[y] $$

what we do know, based on the assumption from the transformation, is that:

$$y^\frac 12 \sim N(\mu,\sigma^2) $$

However I'm not sure how to use this to get back to $E[y]$. Also, I think it is important to note that each prediction $y_i$ corresponds to the predictions for how well individual person i performs in a game, and that each person i doesn't necessarily have the same mean and variance, since some individuals are better than others / more variable at the game.

Any thoughts on this will help!

Thanks,

Answer 1

First, though you say

what we do know, based on the assumption from the transformation, is that: $E[y^\frac 12] \sim N(\mu,\sigma^2)$

note that if "the ML algorithm" is something like linear least squares, then there should be no expectation, i.e. $\sqrt{y}$ is normally distributed, rather than its mean.

However, the distributional assumption is not strictly needed to answer your question.

If $z=\sqrt{y}$ has mean $\mu$ and variance $\sigma^2$, then by definition $\mathbb{E}[z]=\mu$ and $\sigma^2=\mathbb{E}\left[(z-\mu)^2\right]=\mathbb{E}[z^2]-\mu^2$. Then since $y=z^2$ its average is given by $$\mathbb{E}[y]=\mu^2+\sigma^2$$

Note that to compute error bars, you would have to use some distributional assumption. Technically if $z$ is Normal, then $y/\sigma^2$ will have a non-central Chi-squared distribution. However in practice, as implied by Carl's answer, you can just compute "normal" confidence intervals on $z$ (i.e. in terms of $\pm\sigma$) and then square their endpoints (e.g. $[y_{10},y_{90}]=[z_{10}^2,z_{90}^2]$, since order statistics are "preserved" under monotonic transforms).

Answer 2

True enough that $E[y^\frac 12]^2 \neq E[y]$. The question here is what is more important $E[y^\frac 12]$ or $E[y]$? As it turns out, the $E[y^\frac 12]$ measurement is more predictive of what one can expect from measurements because it has the 'nice' properties of a normal distribution where the unmodified data itself would not. Suppose one wants to know what value to expect in the original population units. Then one calculates $E[y^\frac 12]^2$ full well knowing that that value will be less than $E[y]$, where the latter is augmented and biased for right skew by the not so useful properties of the original population.

The same thing is done for standard deviation and other moments. For example, one calculates the $\hat{\mu}\pm\hat{\sigma}$ for the transformed distribution and then $(\hat{\mu}\pm\hat{\sigma})^2$ become the (now asymmetric) locations of $\pm 1 \text{ SD}$ in the original scaling. Now, if we calculate standard deviation from the original population it will be larger than the standard deviation of transformed values, and not useful for predicting probabilities. If we want to predict anything, or calculate a probability, we need to either think in terms of the transformed population, or alternatively, we can think in terms of the normal squared distribution (which is also Chi-squared one).