2

I am modelling a logistic binomial response with around 10 (continuous and categorical) explanatory variables. I would like to model it as a bayesian glm and had a look at the bayesglm function on ARM (package).

The package says: modeling with independent normal, t, or Cauchy prior distribution for the coefficients.

So since I have both categorical and continuous independent variables followed by a binary response variable, would a cauchy or normal distribution be best suitable (I had previously thought a beta would be best since my response variable was binomial)?

A bit lost on what prior scale and prior df to use from the package. Can I please get some help and advice on what distribution (and values) I should use.

Can I also ask, are there any other ways I can compute a bayesian model to make predictions? thank you

Silver
  • 45

1 Answers1

2

As described in Bayesian logit model - intuitive explanation? thread, logistic regression can be described as a linear combination

$$ \eta = \beta_0 + \beta_1 X_1 + ... + \beta_k X_k $$

that is passed through the link function $g$:

$$ g(E(Y)) = \eta $$

where the link function is a logit function

$$ E(Y|X,\beta) = p = \text{logit}^{-1}( \eta ) $$

where $X_1,\dots,X_k$ are the predictor variables and $Y$ is a target variable. What we want to estimate is $\beta_0,\beta_1,\dots,\beta_k$ parameters. When using Bayesian approach, for estimating some parameter of interest we start with defining a prior that describes out out-of-data knowledge about the parameter, and then use the prior combined with the likelihood function (that tells us what the data says about the parameter) to obtain a posterior (i.e. estimate of distribution of the parameter). So we choose priors for parameters and normal, $t$, Cauchy etc. are perfectly fine priors for parameters of regression model or generalized linear model. Beta distribution is inappropriate choice for usual regression model since it would imply that estimated parameters must be bounded in $[0,1]$ interval (you are probably confusing it with beta-binomial model).

The parameters for priors are different story. You need to choose them based on your out-of-data knowledge about the parameters, so this is a problem-specific choice. If you have no a priori knowledge, you can use a weekly informative prior, e.g. normal distribution centered at zero with large variance.

Community
  • 1
Tim
  • 138,066
  • thank you. i understand the prior option now.

    but how could I assign a prior to categorical variables? I also have gender as a variable, is it still ok to use a normal prior for gender?

     – Silver Dec 03 '16 at 16:38
  • @Silver as I wrote, you do not assign priors to variables in here, but to regression parameters. You do not have different kind of regression parameters for different variables. – Tim Dec 03 '16 at 17:32
  • sorry, I'm not that clever and my mind just can't seem to get this right. I think I understand what you mean now: the prior is the guess for the coefficients but can you help me with an example: I know the coefficient for men will be positive/higher (compared to women) and lower for children (compared to adults), how will I set the parameter and prior? or would it be better to sue a non-informative prior since I can't give exact numerical guesses for coefficients?

    thank you for the help

     – Silver Dec 04 '16 at 14:53
  • Hi @Tim, do we then, interpret the results the same way as in the classical approach? Example if genderM is -0.30, then do we say male patients are 0.74 times less likely to be ... as compared to female patients. or male patients have an odds of 0.74 times that of female patients in .... Which term is more appropriate? – HNSKD Jun 28 '21 at 02:07
  • @HNSKD the interpretation in Bayesian and non-Bayesian logistic regression is exactly the same. The difference is that in the Bayesian case you have extra information about the uncertainty of the parameters. – Tim Jun 28 '21 at 07:06