Determining sample size

Question

If I have a population and want to draw samples from there, what are the steps to determine the number of samples to draw from the population? In other words, if I have population N and want to draw k samples from them so that the samples are representative of the entire population, how can I determine k?

I am having some insights from here but in the end of this document I find-

You can still use this formula if you don’t know your population standard deviation

which is my case (i.e., I do not know the population variance). How can I use that formula if I don't know the population standard deviation/ variance?

n.b. I posted this question to that forum but still no luck.

Answer 1

There are two questions here and although I don't disagree with the comments I think the tone could be friendlier and a more helpful response could be given. (1) The first is a basic question from a beginner who wants to understand the puzzle of sample size determination. We should be honest and admit that it is a perplexing problem and the answer seems almost like circular reasoning.

(2) The second is a simple question with a simple answer.

Let me address (2) first since it is simple. In practice you never know the true variance. What you do is guess at it. This can be done by looking at how the answer changes as the population variance would vary over a range of plausible values. You might find the plausible values through literature on similar studies or through your ownpilot study or you can do a two stage adaptive design where the initial stage is used primarily to determine if you need additional data and if so how much. The first stage of an adaptive design is similar to a pilot study but the statistics behind it and the method of final determination of sample size is more complex.

Now about (1), determining sample size (or a formula for it) when comparing the difference between a hypothesized value of a parameter and the true value involves knowing something called the effect size. That is a normalized difference between the hypothesized value and the true value. But to know that requires knowing the true value. But that was your original problem! You want to find out or get a good estimate of the parameter. If you knew the true value you wouldn't have a problem and hence would not need to take a sample to estimate it. It is natural as a first reaction to see this circular argument that tells you that determining the sample size is impossible.

But that is not really so. What you really want to do is specify how big a difference between the hypothesized value and the true value (in terms of effect size) would be large enough for you to want to declare it significant. So you pretend that you have this effect size and see what the probability would be that you would reject the null hypothesis if you have this effect size. This probability will be a function of the sample size and the critical value for your test statistic. The critical value is determined by setting a significance level (aka type I error) for the test. This you can calculate. But it does depend on some unknowns that you have to guess at such as the population standard deviation that we spoke about in the answer to question (2).

This then describes sample size determination for hypothesis testing. a similar thing can be done for confidence intervals where effect size and type I error is replaced by a specified width and a confidence level respectively.

Answer 2

Some more thoughts to add to what Timo said. Usually you might follow some steps like

1. Decide how tight a precision you're trying to get. Say, you want to estimate the population mean to within 0.30.

2. Take a small pre-sample and estimate the standard deviation using

   $s = \sqrt{ \frac{1}{n-1} \sum (x_i - \bar{x})^2 }$

3. Go use that formula you linked to, calculate n (and if n is small you might consider using a t table instead but since you're programming you probably have heaps of data).

4. Go and take that larger sample. Do what you wanted with it.

5. At this point, you might discover it wasn't enough data. What's the best course of action at this point really depends on what you're doing -- how important is it to get a precise result? How hard is getting more data? The statistical tools surrounding these questions are way more than I understand, and my advice to you would be to not worry too hard: if you feel like taking more data to get better precision, take more data, and if you don't, don't. A statistician would say it's technically incorrect to go back to step #2 here and just plug in your new, improved s, but... well, that's what I would do.

Answer 3

I'm not an expert on the subject, but you can estimate the standard deviation either by calculating it from entire population (if it is feasible) or calculate it from a sample as large as possible. Size sqrt(population size) should be usually enough, but the larger, the better. Just use the following formula:

(taken from Wikipedia) to estimate unbiased standard deviation.