7

When analyzing a dataset based on percentages, I have on occasions worked with data as "full" value (i.e. "50") or "reduced" value (i.e. ".50").

However, it just occurred to me that this could have a serious impact on how standard deviation and variance.

If my standard deviation and variance are above 1, the standard deviation will be smaller than the variance. But if they are below 1, the standard deviation will be bigger than the variance.

Does it mean it is good practice to never have a dataset resulting in a variance below 1? Or is there a way to account for this?

Cristol.GdM
  • 173

2 Answers2

10

If my standard deviation and variance are above 1, the standard deviation will be smaller than the variance. But if they are below 1, the standard deviation will be bigger than the variance.

Actually, this is not the case, because you're ignoring the units. Standard deviation of a percentage is measured in percent, while the variance is not.

It's like having a standard deviation of 20 cm (variance 400 cm$^2$) and then worrying about if you measure it in meters, that the variance (0.04 m$^2$) is smaller than the standard deviation (0.2 m). While 0.04 is less than 0.2, you're comparing apples and oranges - they're in different units!

So you can't say that the variance is bigger than or smaller than the standard deviation. They're not comparable at all.

Nothing is amiss: you can happily work with values above 1 or below 1; everything remains consistent. There's nothing to account for.

The only source of a problem I perceive there is if some of your values are in whole percent (50) and some are written as a fraction (0.50), without keeping track of the corresponding units. That you would worry about.

Glen_b
  • 282,281
-3

Variance (which ST is derived from) measures differences from the mean. Given differences from the average d1 and d2, whether D > 0 or D < 0 this always applies: d1 <=> d2 === d1^2 <=> d2^2.

Examples:

d1 = 2
d2 = 3
2 < 3 and 4 < 9

d1 = 0.1
d2 = 0.5
0.1 < 0.5 and 0.01 < 0.25

d1 = 0.5
d2 = 2
0.5 < 2 and 0.25 < 4

I don't know exactly how standard deviation works in practice but it holds true that larger differences give a larger value and small differences give a smaller value even if squaring switches from increasing to decreasing going down passed one.

I think this is valid as once you reach as low as 1, the only way from there toward 0 is down. On a plotter it should produce a continuous curve.

Standard deviation will be compared to standard deviation and variance with variance. As long as you stick to that whether using the standard deviation (square root) or variance (averaged squared differences) they'll remain proportionately comparable.

I believe that SD will effectively be converting it from a geometric (squared curve) to a linear curve.

  • Please excuse that I am from a programming background. The notation I use is works are <=> means compare (-1 less than, 0 same, 1 more than) and === means exactly equal. Numbers in variables are part of the variable name, multiplication only happens with an explicit *.
jgmjgm
  • 95
  • 1
    "I don't know exactly how standard deviation works in practice": that is not a crime but it does rule out this being a correct and useful answer. @Glen_b already answered this correctly. You're ignoring his main point which is that different units of measurement are key to the misunderstanding. Sorry, but this misses the point already made and does not help. – Nick Cox Jan 30 '20 at 16:03
  • That's someone's answer on reddit. I had the almost the same question and the units of measurement doesn't answer the question, it's not related to units but instead real values, specially with no whole numbers. That is, when it goes from growing to shrinking, will that cause a problem. The default answer would be no because the result is still proportionately consistent (you could probably do a proof for this) which I believe is what's important. The explanation you're putting forward might not be invalid, but people like me who have to same question don't get it. – jgmjgm Jan 30 '20 at 16:54
  • Though my question's a bit different because I was wondering if two standard deviations would still be comparable or not rather than SD versus variance. – jgmjgm Jan 30 '20 at 16:54
  • Sorry, but I don't get the reddit reference. But I gather that you have a question, in which case you should please ask it. I don't think your answer to this question is correct or helpful and have commented and voted accordingly. In fact I can't even work out what you are saying as you nowhere explain what d1 and d2 are and how they relate to SD and variance. – Nick Cox Jan 30 '20 at 16:57
  • Maybe the answer here was copied to reddit. My question is more general, do values less than one break anything or need special treatment. It seems basic but I don't know exactly what the SD is being used for. My presumptive answer is that it relies on proportionate difference in values rather than discrete values so proper uses are fine (comparing oranges with oranges or apples with apples) with values below zero. – jgmjgm Jan 30 '20 at 17:15
  • I doubt anyone can make sense of your reddit allusion without a URL, unless they posted it. But the rules of arithmetic are what they are. The facts that, forgetting units, $x^2 > x$ if $x > 1$ and $x^2 < x$ if $0 < x < 1$ don't have any untoward statistical consequences. – Nick Cox Jan 30 '20 at 17:19
  • It was just the top answer in search. That's also the conclusion I came to. I wasn't specifically thinking of necessarily comparing variance with SD but instead coming to the conclusion any case like that, would be invalid and that it relies on the values being proportionate whether being stretched or compressed and whether a value increases or decreases. – jgmjgm Jan 30 '20 at 17:28