Z-test and sampling distribution

Question

The basic trick behind the z-test is the "sampling distribution". The "sampling distribution" is the distribution of all means of all samples.

So what we are interested is: if the population means of the two groups really differs.

H0 - the two groups (control and treated) have the same mean

1. So we first calculate the std of the treated group (std of the control group over n of the treated squared).
2. Then we use:
   • this std of the treated (for normalization)
   • actual mean of the treated (because of the assumption that H0 is true) which equals the actual mean of the control group
   • the treated group mean also called the sample mean (available because we usually have the data of the treated group)

to calculate a Z score.

3. We produce Z in such a way that it can be compared with the sampling distribution which is represented by a Z distribution z(0,1).
4. As a final step we just calculate how likely/unlikely is that Z value. This is equivalent to how likely/unlikely is the suggested mean (in H0) given the sampling distribution.

Is all of the above correct, generally speaking?

Answer 1

Once again, I find myself in a position where I would like to leave a comment but do not have the reputation, and so must leave an incomplete answer.

I am assuming you are a newcomer to statistics, and have created my comment with this in mind, forgoing completeness for simplicity.

Anyways, as Lauren said, your description of a sampling distribution indicates some lack of understanding, which I will seek to rectify. The understanding of sampling distributions is very important, an understanding which my fellow stats undergrads are still struggling with 4 years into study!

Something which helped further my understanding of sampling distributions is the fact that any function of a random variable is itself a random variable.

(Disclaimer: this is all going to be from a classical approach.)

I'm going to start from the basics: a statistic is a function of the data. A statistic which we are very familiar with is the mean, which is the sum of all the data divided by the number of data we have. Let's not start there. Let's instead create a new statistic, which I will call 'John Madden's statistic'. Let's also say we have some sample (x1, x2,...xn) from a normal distribution with mean 2 and variance 1 (and that they are independent and identically distributed). See this distribution on wolfram|alpha: http://www.wolframalpha.com/input/?i=normal%282%2C1%29

John Madden's Statistic = f(x1,x2,...xn) = 2*x2, that is, that my statistic is just two times the second data point that we have. The fact that I selected the second data point is arbitrary; the point of this statistic is that it doesn't have much meaning. Well, this statistic is a function of a random variable, and so must be a random variable itself! You don't have to understand why, but the distribution will be normal with mean 4 and variance 4. That is, this statistic has a distribution, just like our original data did! Except that its a little different, see it here: http://www.wolframalpha.com/input/?i=normal%284%2C2%29

Now that we've seen the concept of a sampling distribution with a meaningless statistic, we will instead look at a more important one, one which you asked about: the mean. The mean is again, a function of a random variable, and so a random variable itself. Up above, I hid how I calculated the distribution of my statistic, but for the mean, I will spell it out a little more. The statistic that is the mean of a normal distribution has as its mean (confused yet? the statistic we are looking at right now which happens to be the mean also has a mean, because it is a random variable. That is, the mean has a mean; there is a certain value the mean is most likely to take.) the same mean as the original population (I will not prove this here unless you really want me to), but a different variance. This variance happens to be the standard deviation of the original population divided by the number of observations we have (I can also prove this if you need me to). So, like above, the mean is normally distributed with a certain mean and variance.

How do we know that the mean is normally distributed? Because of the Central Limit Theorem, which I'm sure is something you have learned/are learning/will soon learn.

Answer 2

Close but there are some issues:

The basic trick behind the z-test is the "sampling distribution". The "sampling distribution" is the distribution of all means of all samples.(There is a very general definition of a sampling distribution-- I would expand this)

H0 - the two groups have the same mean

So we first calculate the std of the sample (std of the population over n of the sample squared). Then we use:

• this std of the sample (for normalization)
• actual mean of the sample (because of the assumption that H0 is true) which equals the first group mean (for example the mean of the control group).(This is not phrased quite right-- Should be the population mean under the null hypothesis which is 0.) -the sample mean (available because we usually have the data of the treated group) to calculate the Z score.

We produce Z in such a way that it can be compared with the Null Distribution which is represented by a Z distribution z(0,1) i.e. assume the sampling distribution is normal and then we need to standardize our observed result to compare Z to a N(0,1). As a final step we just calculate how likely/unlikely is that Z value.

This is equivalent to how likely/unlikely is the suggested mean (in H0) given the sampling distribution. I don't think this is right