6

Is it common practice (and adequate) to regroup two binary dependant variables into a single 4-level dependent variable to take advantage of the multinomial regression? For instance, say we have information on two related conditions (outcomes) A and B. A new 4-category variable would be defined such that:

category 1 = Neither conditions A nor B
category 2 = Condition A (only)
category 3 = Condition B (only)
category 4 = Both conditions A and B

This allows running a single multinomial regression instead of using two binary logistic models that include the same predictors.

Dominic Comtois
  • 2,129
  • Could you tell us what you're doing?

    Also, wouldn't such a model have 4 categories (what about neither A nor B)?

     – Peter Flom Apr 10 '12 at 15:43
  • @Peter this is strictly theoretical. I'm following a class right now and the prof uses MLR this way and I'm thinking that might be stretching a little the notion of a single outcome having "mutually exclusive categories". You're right for the 4th category, I'll do an edit for that. – Dominic Comtois Apr 10 '12 at 16:28

3 Answers3

4

As @Riaz Rizvi suggests, this may not be a good idea.

Your scheme enforces a particular (and rather unlikely) covariance structure on the problem by flattening to a multinomial this way. Since you suspect, or at least wish to allow the possibility that the presence of A is informative of B, then you should be working with a bivariate probit. Working with two separate logistic models is not going to be able to represent this. The model is a regression with an explicit correlated bivariate latent variable generating the choice probabilities, as discussed briefly in the link and at greater length in good econometrics texts.

Affine
  • 2,377
conjugateprior
  • 20,883
  • 1
    I'm confused that in your link the bivariate probit is a multinomial model yet you say to use that is not a good idea. Did you mean it is a bad idea to use two separate logistic or probit models rather than a multinomial one? Or am I missing something? – Momo Apr 10 '12 at 22:25
  • The example on that page is a bivariate probit, but here's a more targeted exposition – conjugateprior Apr 10 '12 at 22:40
  • But well caught with the multinomial reference, I'll change that in the answer. – conjugateprior Apr 10 '12 at 22:44
  • It still sounds like you agree with his professor since you are advocating a joint model rather than separate models for each variable. – Neil G Apr 10 '12 at 22:49
  • Thanks. However, it's more the connection between the multinomial and a multivariate binomial model I was curious about. You say the multinomial probit is wrong, one should use multivariate probit eg bivariate. But isn't that equivalent, as the Wikipedia entry suggests? Multivariate Probity=Multinomial Probity? – Momo Apr 10 '12 at 22:50
  • @Neil G I am agreeing with his professor that a joint model is wanted. I'm disagreeing on which joint model would be sensible. – conjugateprior Apr 10 '12 at 22:53
  • Sorry it seems I'm a bit sleepy and sloppy at the moment Let's try this again: The bivariate probit is for when you think that the separate error terms share a 'shock' that couples them. The 'targeted exposition' above is more explicit about that. – conjugateprior Apr 10 '12 at 23:01
  • Yes, don't you think that's going beyond the question to hypothesize this shock term? – Neil G Apr 10 '12 at 23:07
  • @NeilG That was just the way I operationalized the questioner's: "say we have information on two related conditions (outcomes)" (emphasis mine). The shock is just a reification of the unobserved cause of the relatedness/dependency to develop the model. Just thinking of them as correlated (which is an implication) will also work. – conjugateprior Apr 11 '12 at 00:25
  • Sure, that's a fine way to see things. But, the simplest assumption is no assumption, which means to just use multinomial logistic regression. Anyway, since you agree with the professor, you might want to change your introductory sentence… – Neil G Apr 11 '12 at 00:33
  • 2
    Thanks for the response. Could you comment briefly on the consequences of using the multinomial method instead of a bivariate method in such a situation? (Would the estimates be biased in any way, or would their precision be under/overestimated?) – Dominic Comtois Apr 11 '12 at 15:51
1

A multinomial is perfectly fine in this situation, but it comes at two costs:

  • An explosion in the number of parameters. (If you were to combine $n$ binary variables like this, you would have $2^n$ parameters instead of the original $n$.)
  • The solution is harder to interpret if the original variables are actually independent. (If you would have had a simple relationship such that input variable $x$ implies dependent variable $y=1$, you would now have $x$ implies that the combined dependent variable takes on one of the many outcomes corresponding to $y=1$.)

The major advantage is that your model can use the additional parameters to encode distributions not possible in the original model.

Neil G
  • 15,219
0

I don't think so. The multinomial distribution is derived from n independent variables, but your situation has two dependent variables. A multinomial regression is not applicable here.

Rian Rizvi
  • 111
  • 1
    Could you explain this a little bit further? I don't understand why you say that a multinomial distribution is derived from $n$ independent variables. Do you mean outcomes? The outcomes are definitely dependent since they're mutually exclusive. – Neil G Apr 10 '12 at 21:49