2

I have thousands of evaluation scores, over several sites, which vary in number of evaluations for each. I am tasked with comparing the evaluation means for each site to the all-site mean. The approach previous has been to use a student t-test on each site, with the all-site mean as the test value. The all-site mean is a simple unweighted mean.

Is this a reasonable method, given that each site is included in the all-sites mean, and that each site contributes differing numbers of evaluations to that mean?

If this is not a useful approach, what would a better one look like? Would the application of weighting on the all-site mean be an improvement?

D_Look
  • 21

1 Answers1

1

Is this a reasonable method, given that each site is included in the all-sites mean, and that each site contributes differing numbers of evaluations to that mean?

Note that there are two issues here:

1) treating the "whole data" as a population rather than a sample

2) comparing a subgroup to "everything else + that subgroup"

In respect of (2), the approach itself (properly conducted) will yield the same inference as comparing the subgroup to everything else (but with more effort).

See the comments here about the same issue in a different context.

It's easier to do the standard thing.

In respect of (1), if you don't have the population about which you wish to make inference, you shouldn't treat it as such.

Community
  • 1
Glen_b
  • 282,281