How could Lilliefors use Monte Carlo if the estimand is not distribution-free?

Question

Lilliefors test is a well-known statistical test for normality. Its idea is based on the Kolmogorov-Smirnov test, except the CDF is replaced by the CDF of the normal distribution with $\mu, \sigma^2$ chosen as the estimated ones from the data.

In other words given data $\{X_1, \ldots, X_n\}$ we form its empirical CDF $F_n$, and consider the estimand $$E := \sup_x \left|F_n(x) - \text{cdf}_{N(\hat{\mu}, \hat{\sigma^2})}(x)\right|.$$ The idea is that if the data does come from a normal distribution, then $E$ must be small. In [1], Lilliefors claimed to use Monto Carlo method to compute numbers that defines the smallness.

Question

Since $E$ is not distribution-free, shouldn't have Lilliefors done a Monto Carlo numerical estimate for all kinds of $\mu$ and $\sigma^2$? Why would a single table be enough?

Reference

[1] On the Kolmogorov-Smirnov Test for Normality with Mean and Variance Unknown-[Hubert W. Lilliefors]
[2] The Probability Integral Transformation When Parameters are Estimated from the Sample-[F. N. David, N. L. Johnson]

What are the pros and cons of the notations $$\hat\sigma^2,\quad \hat{\sigma^2}, \quad \widehat\sigma^2, \quad \widehat\sigma^{,2}, \quad \widehat{\sigma^2} $$ coded respectively as \hat\sigma^2, \hat{\sigma^2}, \widehat\sigma^2, \widehat\sigma^{\,2}, \widehat{\sigma^2}$\text{?} \qquad$ — Michael Hardy, Sep 25 '23 at 15:41

Glen_b · Answer 1 · 2023-09-24T08:32:13.413

7

It might not be distribution-free but for a given sample size it is location and scale free.

The internally standardized joint distribution of the observations does depend on the form of the distribution, but at the normal it does not depend on the specific values of $\mu$ and $\sigma$.

Specifically, for i.i.d. observations from a normal distribution, the vector of internally standardized values is a pivotal quantity; its joint distribution doesn't depend on any unknown parameters.

Even if you didn't wish to/know how to show it algebraically, it's easy enough to look at simulation and see that it doesn't matter what $\mu$ and $\sigma$ are.

As such, the distribution of the test statistic, which depends on those standardized values in turn isn't affected by what values $\mu$ and $\sigma$ take.

Consequently he could just simulate standard normals. I don't know that he did - it's a long time since I read the paper - but it would make sense. Nothing would change if he used any other normal, though, he could please himself.

Similar comments apply to the other two cases Lilliefors looked at in that paper (i.e. just $\mu$ or just $\sigma$ unknown) and to the exponential cases that were covered in his other paper.

edited Sep 24 '23 at 08:32

answered Sep 24 '23 at 00:08

Glen_b

282,281

What do you mean by internally standardized values? / Do you mean that for a fixed sample size $n$ while $E$ is not distribution free (i.e. the same for all continuous distributions), but the distribution of $E$ is the same for all normal distributions? If that's the case, I'm really interested in knowing a proof. Lilliefors didn't fully justify this in [1], except that Lilliefors mentioned [2]. However, I don't see how [2] relates to this problem. Maybe it's written too long ago.. – Student Sep 24 '23 at 14:36
Standardizing by the sample's own mean and s.d. rather than by some externally-derived value, $Z_i=\frac{Y_i-\bar{Y}}{s}$. 2. Yes, that's what I mean. 3. Note that if you take a sample from a standard normal and multiply by $\sigma$ and add $\mu$ you have a sample from $N(\mu,\sigma^2)$, but the sample mean is larger (shifted right) by $\mu$ and the sample standard deviation has been multiplied by $\sigma$. Now when you subtract $\bar{Y}$ you're subtracting that $\mu$ you added and when you divide by $s$ you're dividing by that $\sigma$ you multiplied by, taking you back to an internally...

Glen_b

Sep 24 '23 at 16:32

... standardized version of $Z$. We don't need to derive it to see that it's the same. That is, internal standardized samples have the same distribution no matter what location and scale you start with. The standardization removes the effect of $\mu$ and $\sigma$. It induces some structure to the multivariate distribution, but it's the same structure no matter what $\sigma>0$ you multiply and what $\mu$ you add at the start. – Glen_b Sep 24 '23 at 16:33

Thank you, @Glen_b, and I apologize for selecting the other answer. Your answer is also great, but the full answers are given in the comments.. By the way, we know that Lillefors' trick works for the normal and the exponential distributions. Is there a classification of family of distributions such that the trick works too? – Student Sep 25 '23 at 13:41

No need to apologize; select whatever you think is the best answer. The crucial thing with the approach is simply that you can make a pivotal quantity to use in the test (see the link). If you start with a location-scale family like the normal or the shifted exponential that will go pretty much exactly as Lilliefors did those two, but it could be some other family as long as you end up with a pivotal quantity. e.g. in a Weibull you'd rescale by the estimated scale parameter and take a power related to the estimated shape parameter. – Glen_b Sep 26 '23 at 00:46

1

... ctd I believe you get a pivotal quantity that way (indeed, its log is location-scale family so I am pretty sure it works), though I haven't gone through and proved it. It doesn't work for everything though; not every distribution family has a suitable pivotal quantity like that. – Glen_b Sep 26 '23 at 02:07

Sextus Empiricus · Accepted Answer · 2023-09-24T15:33:25.987

shouldn't have Lilliefors done a Monto Carlo numerical estimate for all kinds of $\mu$ and $\sigma^2$?

If the $X_i$ are iid Gaussian variables, then $E$ is independent from distribution parameters $\sigma$ and $\mu$, but depends only on the sample size $n$.

Consider generating samples from a distribution $N(\mu,\sigma^2)$ by first generating samples from $N(0,1)$ then multiplying by $\sigma$ and adding $\mu$. That multiplication and addition will make the estimates $\hat{\mu}$ and $\hat\sigma$ shifted and scaled with similar factors. The two distributions $F_n(x)$ and $\text{cdf}_{N(\hat{\mu}, \hat{\sigma^2})}(x)$ will be shifted and stretched in the same way and the parameters $\mu$ and $\sigma$ have no effect.

An example is in the following image:

The image above is a sample of size $n = 20$ generated with $\mu = 0$ and $\sigma = 1$, the sample below is a sample of size $n = 20$ generated with $\mu = 1$ and $\sigma = 0.5$ (effectively the same samples with just a different position and scale).

The only difference is that the curves are shifted and squeezed in the horizontal direction, but because this happens in the same way for the empirical distribution function and the estimated cumulative distribution function, the distance in the vertical direction remains the same.

How could Lilliefors use Monte Carlo if the estimand is not distribution-free?

Question

Reference

Related

2 Answers2

Linked