5

I would like to test the difference of two odds ratios given the following R-output:

f=with(data=imp, glm(Y~X1+X2, family=binomial(link="logit")))
s01=summary(pool(f1))
s01

                    est        se         t       df   Pr(>|t|)

(Intercept) -1.7805826 0.1857663 -9.585070 391.0135 0.00000000 
  X1           0.2662796 0.1308970  2.034268 390.4602 0.04259997

  X2           0.6757952 0.3869652  1.746398 395.6098 0.08151794 

cbind(exp(s01[, c("est", "lo 95", "hi 95")]), pval=s01[, "Pr(>|t|)"])

                            est     lo 95     hi 95       pval
             (Intercept) 0.1685399 0.1169734 0.2428389 0.00000000
             X1          1.3051000 1.0089684 1.6881459 0.04259997
             X2          1.9655955 0.9185398 4.2062035 0.08151794

To do so, I would need to take the difference of the log odds and obtain the standard error (outlined here: Statistical test for difference between two odds ratios?).

One of the predictor variables is continuous and I am not sure how I could compute the values required for $SE(logOR)$.

Could someone please explain whether the output I have is conducive to this method?

Igor F.
  • 9,089
ksroogl
  • 413
  • X1 and X2 above aren't odds ratios. They are coefficients that represent the change in log-odds of an outcome variable per unit change of a predictor variable. It's not at all clear what comparing X1 and X2 gains you, as even if both variables were continuous the values of the coefficients would depend on the units of measurement. E.g., miles versus millimeters for a predictor would give wildly different numeric coefficient values for the same underlying data. Perhaps you are interested in some estimate of predictor importance, but this is not the right way to do that. – EdM Jun 15 '18 at 19:34
  • This design was based on a previous research that took an identical approach (simply unclear how the analysis was executed); what do you propose as the right way to estimate predictor (relative) importance? what can I glean from the above analysis? – ksroogl Jun 18 '18 at 00:36
  • This approach might work if all predictors were on the same scale or were all normalized (e.g., to 0 mean and unit standard deviation). You could try that approach but please be aware that such normalization of a categorical variable isn't always very reliable as it can depend heavily on the particular sample in question. There is an extensive set of threads on this site about feature selection. A more specific description of what you are trying to accomplish (not just the particular approach you happened to come up with to try to reach your goal) might help get you a more helpful answer. – EdM Jun 18 '18 at 01:14

1 Answers1

1

If you want to test whether exp(X1) is statistically different from exp(X2) then you do not need to perform any statistical tests in this case.

Just observe the confidence intervals. In general, we cannot infer that point estimates are statistically different when the confidence intervals overlap, but this is a special case.....

The confidence interval for exp(X1) is (1.01, 1.68) while for exp(X2) it is (0.92, 4.21)

The confidence interval for exp(X1) is completely contained within the confidence interval for exp(X2)

Therefore they are not statistically different.

Robert Long
  • 60,630
  • 2
    Overlap of 95% confidence intervals does not mean that there is no significant difference between the point estimates; that's much more stringent than the usual p < 0.05 requirement. The relation of point-estimate significance to confidence-interval overlap is not completely straightforward; see this page for thorough discussion in the context of t-tests. – EdM Jun 15 '18 at 19:22
  • @EdM Thanks, I am aware that point estimates can be significantly different in the presence of some overlap. But here one CI is completely contained within the other. Can you provide a counter-example where this is the case, and the point estimates are significantly different ? – Robert Long Jun 15 '18 at 19:25
  • 1
    I agree in this case with complete containment; just wanted to clarify for future readers of this page. Bigger problem is that the comparison of the OP's X1 and X2 coefficients doesn't seem to make sense at all, as I note in a comment above. Even if the values are numerically indistinguishable on their present scales of measurement, they really are incommensurate and that result would seem to have no useful interpretation. Changing the scale of the continuous variable with this approach could lead to an apparent "significant" difference without meaning. – EdM Jun 15 '18 at 19:41
  • @EdM Good point - I will edit the answer to make it moreclear that this is a special case. – Robert Long Jun 15 '18 at 19:51