1

In its Procedures Handbook 4.1 (January 2020) , the What Works Clearinghouse (WWC) announces calculating the weighted average of the effect sizes in its meta-analyzes using the fixed effects model (plural) and not the fixed effect model (singular). They present the following arguments :”unlike the fixed effect (singular) model, the fixed effects (plural) model does not assume that the studies are estimating a common effect. Instead, the fixed-effects model assumes that the observed variation among the effect sizes in the meta-analyses reflects the true variation in population effects.” The random effects model is not mentioned.

I already have a first question here: what is the difference between the random effects model and the fixed effects model (plural) and where can we find information on this subject?

My second question concerns the calculations implemented by WWC. In my opinion, these calculations are the same as those that would be used when using the fixed effect (singular) model. Does this mean that the model of fixed effects (plural) and the model of fixed effect (singular) materialize by the same calculations? But then, what is their difference?

Here are the main steps (appendix H of Procedures Handbook 4.1) :

Each effect-size = Hedges’g=g

Each variance = $\text{var}(g) = w^2 \cdot ((n_i + n_c)/(n_i \times n_c)+g^2/(2 \times (n_i + n_c)))$ (no between-studies variance)

with

• $n_i$ = intervention student sample size;

• $n_c$= control student sample size;

• $w = 1- 3/ (4(n_i + n_c) - 9)$

The weight associated with each effect size = $W=1/\text{var}(g)$

The fixed-effects meta-analytic average = $M = (W_1 \cdot g_1 + W_2 \cdot g_2 +…)/(W_1 + W_2 + …)$

Igor F.
  • 9,089
Nathalie_RR
  • 71
  • Your link seems to be dead. Do you have an alternate link? – Eli Mar 05 '20 at 14:18
  • 1
    I can't answer definitively without seeing more information. I would guess their fixed effects (plural) model has a coefficient for each study and uses the weighted least squares to fit thde model. – Eli Mar 05 '20 at 14:22
  • 1
    The article by Rice and colleagues in Royal Stat Soc Series A 2018 181 205-227 is the primary source here I think – mdewey Mar 05 '20 at 16:49
  • 1
    See also: https://stats.stackexchange.com/a/156625/1934 – Wolfgang Mar 05 '20 at 19:19
  • The answer from the WWC :

    The “fixed-effect” and “fixed-effects” models are mathematically equivalent. However, we adopt the “fixed-effects” terminology to acknowledge (philosophically) that there very likely is more than one true effect in the population of effects. The assumption is that the observed variability pattern in effects generalizes to the population of possible effects. We are currently limited (statistically) in our ability to estimate the full distribution under a random effects modeling approach, but this is an ongoing area of research and development for the WWC.

     – Nathalie_RR May 01 '20 at 08:34

0 Answers0