Why does the variance of a sample change if the observations are duplicated?

Question

The variance is said to be a measure of spread. So, I had thought that the variance of 3,5 is equal to the variance of 3,3,5,5 since the numbers are equally spread. But this is not the case, the variance of 3,5 is 2 while the variance of 3,3,5,5 is 1 1/3.

This puzzles me, given the explanation that variance is supposed to be a measure of spread.

So, in that context, what does measure of spread mean?

Answer 1

If you define variance as $s^2_{n}=$$\,\text{MSE}\,$$=\frac1n \sum_{i=1}^n (x_i-\bar{x})^2$ -- similar to population variance but with sample mean for $\mu$, then both your samples would have the same variance.

So the difference is purely because of Bessel's correction in the usual formula for the sample variance ($s^2_{n-1}=\frac{n}{n-1}\cdot \text{MSE}=\frac{n}{n-1}\cdot \frac1n \sum_{i=1}^n (x_i-\bar{x})^2=\frac{1}{n-1}\sum_{i=1}^n (x_i-\bar{x})^2$, which adjusts for the fact that the sample mean is closer to the data than the population mean is, in order to make it unbiased (taking the right value "on average").

The effect gradually goes away with increasing sample size, as $\frac{n-1}{n}$ goes to 1 as $n\to\infty$.

There's no particular reason you have to use the unbiased estimator for variance, by the way -- $s^2_n$ is a perfectly valid estimator, and in some cases may arguably have advantages over the more common form (unbiasedness isn't necessarily that big a deal).

Variance itself isn't directly a measure of spread. If I double all the values in my data set, I contend they're twice as "spread". But variance increases by a factor of 4. So more usually, it is said that standard deviation, rather than variance is a measure of spread.

Of course, the same issue occurs with standard deviation (the usual $s_{n-1}$ version) as with variance -- when you double up the points the standard deviation changes, for the same reason as happens with the variance.

In small samples the Bessel correction makes standard deviation somewhat less intuitive as a measure of spread because of that effect (that duplicating the sample changes the value). But many measures of spread do retain the the same value when duplicating the sample; I'll mention a few --

• $s_n$ (of course)

• the mean (absolute) deviation from the mean

• the median (absolute) deviation from the median

• the interquartile range (at least for some definitions of sample quartiles)

Answer 2

As some sort of mnemonic, $V\,X = E\,V\,X + V\,E\,X$. So the expected value of a sample's variance is too low, with the difference being the variance of the sample's mean.

The usual sample variance formula compensates for that, and the variance of the sample's mean scales inversely with sample size.

As an extreme example, taking a single sample will always show a sample variance of 0, obviously not indicating a variance of 0 for the underlying distribution.

Now for 2 and 4 evenly weighted samples, the corrective factors are $2/1$ and $4/3$, respectively. So your calculated expected variances differ by a factor of $2/3$. The variance of the sample itself is $1$ in either case. But the first case presents a weaker case for $4$ being the mean of the base distribution, and every other value would mean a larger variance.