0

I have a set of scores with a maximum range from -12 to +12 for a set of categories of a variable. There are two populations that each have a set of scores for the same categories of the same variable. The sets of scores look something like this:

Category Population One Population Two
Red +7 -3
Orange +1 -2
Yellow 0 +5
Purple +2 -1
Green -10 +1

The scores between the two populations are very different for each category (i.e. Green has a strong negative association for Pop One, and a positive association for Pop Two). Additionally, the categories also have clearly different effects on the population association (i.e., Green scores very negatively compared to Red in Population One).

Is there a way to statistically “prove” whether the differences are significant?

Is there then a way to "prove" which pairs are the most significantly different (aka, Green is the most different between Pop One and Two)?

Note that the scores for a given population will always sum to zero or near zero, so comparing the averages will be insignificant.

Is it justifiable to report significance based on the standard deviation of each population? i.e. "The score for Green in Population One is two standard deviations into the negative, so it is the most significant score." OR "Any score below one standard deviation away from 0 was not considered significant"

If it is justifiable to use this, should I calculate the standard deviation using the median rather than the mean, since the mean will be zero or near zero?

It is feasible to get the score for each individual in each population, and then have the "population score" be the mean of the individuals. Would this make it easier to run statistical tests on significance?

Thank you in advance for any advice!

Greylan Schmittou
  • 3
  • Given that the averages are essentially the same, please tell us the sense in which the scores are "very different." And could you explain what might be special about the presence of negative scores? That seems irrelevant. – whuber Jun 08 '23 at 21:02
  • The scores are very different in that what is important is the relationship between the category and the population. The two populations clearly have very different relationships to the different categories. The problem (I have been told) is that negative numbers do not function well within some statistical tests, maybe that is not true? – Greylan Schmittou Jun 10 '23 at 16:19
  • Could you be specific about what "clearly ... different relationships" might be? I cannot see any clear differences based on this tiny sample. What statistical tests do you have in mind that cannot accommodate negative numbers? – whuber Jun 10 '23 at 17:36
  • Statististical test relate to whether or not samples are different relative to the probabilistic variations that may occur in sampling. Your example does not give details about how you obtained the data and what it represents. This makes it difficult to match it with a statistical test... – Sextus Empiricus Jun 16 '23 at 13:10
  • ... example: Say I have five coloured coins. One coin is red with +7 on one side and -3 on the other side, one coin is orange with +1 on one side and -2 on the other side, one coin is yellow with +0 on one side and +5 on the other side, one coin is purple with +2 on one side and -1 on the other side, one coin is green with -10 on one side and +1 on the other side. The table shows the results of flipping the five coins each once in population A and another time in population B. There are $2^5 = 32$ equally probably outcomes. Are the results different? Yes. Is it statistically significant? No... – Sextus Empiricus Jun 16 '23 at 13:14
  • ... that might sound like a silly example, but it is just one of many ways that these data can be generated. without background, it has little meaning and we can not suggest a statistical test that might be suitable to analyze it (and also the purpose of the testing needs to be clear, aside from the meaning of the data). – Sextus Empiricus Jun 16 '23 at 13:16

1 Answers1

0

Is there a way to statistically “prove” whether the differences are significant?

Statistical significance can only be computed* when you have an idea about the statistical variations that occur in your samples.

You haven't given any information about your samples that allow to make such estimates.

Therefore the answer is 'No'.

*it is a bit strong to say 'prove'.

Sextus Empiricus
  • 77,915
  • Thank you! This answers my question. So if I understand correctly; if I were to score each individual in the population so that the "population" score for each category was the average of all individual scores, then it might be possible to run a statistical test to compare the results of the two populations? – Greylan Schmittou Aug 01 '23 at 01:18
  • @GreylanSchmittou A statistical test relates to statistical variations. For example measurement errors or variations due to sampling from a larger population. In your question it is not clear to me what statistical variations you might have. Possibly the populations are fully sampled and without measurement error. Then there is no statistical variations. See also: Statistical inference when the sample "is" the population – Sextus Empiricus Aug 01 '23 at 18:42