Interpreting normality tests results

Question

I have $35$ data points with me. These are the mean values calculated from a human population. Using a statistical software, I performed the normality tests - Shapiro Wilk, Jarque-Bera, Anderson-Darling. The data set passed all these three tests (p(normal) values were greater than $\alpha$ level = $0.05$) hence, the null hypothesis that these mean values belongs to a population which is normally distributed was accepted. Then I plotted the corresponding histogram. But their I saw no such normal distribution. The histogram was positively skewed. And when I increased the bin size from 4 to 6 I found the distribution to be bimodal. Now, I do not understand how to interpret the normality tests results?

The 35 data values are:

3.092756417, 2.42970367, 2.672162617, 2.724095487, 3.241397572, 2.889870095, 
3.06717103, 3.367232596, 3.50428447, 3.502092638, 2.756195965, 3.195877988, 
3.947919563, 3.039449873, 3.271063048, 2.467537862, 3.809382378, 2.613748277, 
2.40550882, 3.299434613, 2.735915946, 3.187394643, 3.223552688, 2.614370296, 
2.674806202, 2.638087031, 2.589459676, 2.460064912, 2.555203245, 2.153496203, 
3.591850829, 2.274449589, 2.25529134,  2.202203377, 2.535145726

The question shouldn't really be about interpreting the normality results: they are perfectly clear. It should be about why you don't think these histograms appear to come from some underlying Normal distribution. Your basis for that seems to be that they visually deviate from a fitted bell curve. What you are overlooking is the fact that any truly random sample ought to deviate somewhat from its parent distribution. That exposes the real issue: how much a deviation can we expect and what sizes and patterns of deviations ought to constitute evidence against the hypothesized distribution? — whuber, Aug 28 '16 at 22:12
@Glen_b These data points are the mean parity of mothers in 45-49 years age group (for all states) in a country's population. And I got an assignment in class to test normality. — Dark_Knight, Aug 28 '16 at 23:44
Note that if you're doing an assignment, you're expected to make that explicit in your question (even when you're not asking us to answer the question). See the [help/on-topic] under homework and please add the self-study tag and read its tag-wiki. Speaking more generally (outside assignment work), it would be odd to test normality without a specific reason to do so — Glen_b, Aug 28 '16 at 23:54
I just want to add that your interpretation of a $p\text{-value}>0.05$ is problematic and you should be a little bit more careful: accepting the null is a different shoe as the statement that the null can not be rejected at a given signficance level. — BloXX, Apr 10 '18 at 10:37

score 5 · Accepted Answer · edited Apr 13 '17 at 12:44

Histograms - especially those with few bins - can sometimes be quite misleading (See here for some eye-opening examples).

Indeed, this is part of what is going on here -- changing from 4 to 6 bins, or even where the first bin starts seems to have some substantial effects on the impression generated by the histogram. One thing to keep in mind is that the numbers of values in each bin here are quite small, so quite a bit of uncertainty attaches to the impressions we get.

For example with your data, and using 4 bins like your first plot, a simple shift of bin-origin produces quite different impressions (instead of moving the bins, I moved the data, but the appearance of the plot is the same as moving the bins the other direction):

On the other hand, there aren't enough data points to use a lot of bins, so there's not a great deal to do about this. I would suggest using about 8 bins and trying more than one bin-origin.

So what do we have here?

Firstly, the sample is almost certainly not actually drawn from a normal population. Normality is a model, which may often be useful but is unlikely to actually describe the population.

However, the sample is pretty consistent with being drawn from a normal distribution - you can't discern non-normality using the tests you applied. Even if you could, this would not of itself indicate that you should not use normality as a model. That decision is more like an effect-size issue than a significance issue.

(I would advise against applying a collection of normality tests -- or indeed usually any. If you need to test normality for some reason, choose one test that best measures whatever deviation(s) from normality that it's most important for you to be able to pick up.)

The sample is at best very mildly skewed to the right (both the most common measures of it suggest very weak right skewness), and has a hint of bimodality, but neither would be strong enough to suggest the sample is not consistent with having come from a normal distribution.

For example, the third moment skewness is only about 1 standard error from 0.

We can see the hint of bimodality in the Q-Q plot:

The left and right parts are nearly linear, but there's a section in the middle that looks like the line was "pushed up" at that point -- leaving a gap with only one data value in it, where normally we'd expect several points. This is the main source of the suggestion of bimodality that's in the data -- but you do see this kind of thing with small samples.

So while you're observing those things (possible skewness and bimodality in the sample) those impressions are very weak and don't indicate that the sample could not have come from a normal distribution.

Even if you could tell that the sample didn't come from a normal distribution (and it almost certainly doesn't, simply because such simple models are almost never going to describe real data), that doesn't mean you shouldn't assume normality.

Thanks. It was helpful. How did you get skewness value about $1$. I found it to be $0.4390918$ ? — Dark_Knight, Aug 28 '16 at 16:09
My text does not say that the skewness value is 1. It says something slightly different to that. I agree that the skewness is about 0.4, but its standard error is also about 0.4 ... so it's only one standard error away from 0 -- so it doesn't indicate that the population the sample came would necessarily be skew at all -- indeed it would also be consistent with a population that's skewed the other direction from the sample. You don't have a large enough sample to say much other than normality seems to be a reasonable approximation. — Glen_b, Aug 28 '16 at 21:32

Interpreting normality tests results

1 Answers1