2

I am setting up a Cox model, to model the probability that new customers leave the bank .

I had a nice set up, with plenty of significant explanatory variables, until the result of one categorical variable troubled me. The categorical variable I have called Depth with categories Only Bank, Bank+Pension and Bank+Pension+Insurance. So if the new customer only made a new account, he falls under that first category. If he also took with him his pension, he falls under the second, and if he added to that bought some insurance at the bank, he falls under the third category.

The common belief in the bank is that the "deeper" the customers are, the more likely are they to stay. This is also intuitive. If a customer has both insurance and pension added to the normal bank account, he would not bother with the hassle of changing company.

Looking at all the customers, here is an overview over how the Depth customers "dies" (Failed means that they left, and Censored means that they have not left after a year): Overview

We see from the data that slightly more Only Bank customers are "censored" (meaning they are staying) than then Bank+Pension customers. This is not what I expected. We expected more Bank+Pension customers would be staying. Anyway, I suspect that this difference is not significant, and that it actually is no difference in the hazard rate between the two first categories.

My question/problem is that if I do a Cox regression with only one explanatory variable, being the Depth variable, there is actually no significant difference between the two first cateogories (p-value 25%) . Regression results. NOTE: <code>Only Bank</code> is reference group

But when I add new explanatory variables (even just one) suddenly the difference becomes significant (really small p-values) and my regression results tell me that Bank+Pension customers have a higher hazard rate than Only Bank customers. Here is an example, where I added a binary variable X (not correlated to Depth): With added binary variable

Now my result tells me that Bank+Pension customers have a higher hazard rate! This is bad! I have no idea what is going on here. What is going wrong in the estimation process here? Any explanation is recieved with great gratitude. I suspect it has something to do with the great difference in observations between the categories, but I am a newbie here...

EDIT: from link #4 @Scortchi provided below, I copy paste the following:

Misspecified models: The underlying theory for t-statistics/p-values requires that you estimate a correctly specified model. Now, if you only regress on one predictor, chances are quite high that that univariate model suffers from omitted variable bias. Hence, all bets are off as to how p-values behave. Basically, you must be careful to trust them when your model is not correct. -- I see that I obviously have sinned, testing my predictor of annoyance with regressing on only that predictor.

Erosennin
  • 1,734
  • Is -110.824 a typo for -1.10824? – Scortchi - Reinstate Monica Apr 15 '15 at 10:02
  • Yes, sorry, that is a typo! – Erosennin Apr 15 '15 at 13:08
  • 1
    This is a popular topic in corporate finance literature within "financial intermediation" area. It's often related to something called "lending relationships" or "relationship banking". Make sure you at least browsed the literature to see what the researchers are doing. – Aksakal Apr 15 '15 at 13:35
  • 1
    Also, why do you combine the relationships into one "depth" variable? Why not split them into B, P and I? This way you can leverage entire dataset. I would also look at separate models for these three types of customers. – Aksakal Apr 15 '15 at 14:10
  • What do you mean by leverage entire dataset? Yes, it could be done, but I do not understand where this would make things better? Not because I oppose of what you are saying, but because I do not understand :) – Erosennin Apr 16 '15 at 06:15
  • I think @Aksakal's wondering what happened to customers with a bank a/c & insurance but no pension - whether you excluded them from the analysis. Of course if there are no such customers it doesn't matter. On the separate models idea: you could treat these types as strata - allow a different baseline hazard for each type but estimate common hazard ratios for the other predictors. (But I'd tend to do that only when the hazard functions for different types don't look proportional.) – Scortchi - Reinstate Monica Apr 23 '15 at 10:16

1 Answers1

2

Any coefficient in a multiple regression model represents the relationship between a predictor & the response holding constant all the other predictors in the model. So not only the point estimate, p-value, &c. of a coefficient change when you add in other terms, but also its interpretation. Including more predictors also explains more of the variability in the response, tending to reduce standard errors (when they're not too correlated with existing predictors). You shouldn't be surprised: see here, here, here, here, here, here, & here.

So your intuition needs to be qualified to "the 'deeper' the customers are, the more likely they are to stay, other things being equal", & moreover you need to have a clear notion of what those other things are. Possible explanations include:—

  • Confounding effects aren't accounted for in your model—perhaps you only recently started providing pension savings & attracted a lot of customers of the type that have a high propensity to switch about.
  • Your intuition's wrong—perhaps people with pensions have more incentive to switch to the best provider for them at any given time because they have more at stake.
  • The true hazard ratio's slightly under one but the estimate is slightly over, owing to sampling error (a p-value of 0.046 is only modest evidence against that).
  • The model's not well specified— did you check the proportional-hazards assumption, linearity of any continuous predictors, &c?
Community
  • 1
Scortchi - Reinstate Monica
  • 29,907
  • is it possible to account for confounding effects?
  • Yes, maybe my intution is wrong...
  • Would I check the PH assumption by plotting a Kaplan-Meyer plot? I have not done this for all the explanatory variables - no. Would this be necessary?Linearity for any continous variables I have not checked for either.

    • I am afraid I am just a beginner here, trying to learn on the way.

    – Erosennin Apr 15 '15 at 12:29
  • I guess I am surprised for a lot of reasons, one being that the difference between the amount leaving in the two groups is only 1%. Does it not surprise you as well @Scortchi?

    Also, when you explain that a coefficient in multiple regression model represents the relationship between a predictor and the response, holding all the other predictors constant. This interpretation of each coefficient, I get, but I fail to see how this relates to my problem. Yes, the p-value might change, but it surely cannot be random how p-changes?

     – Erosennin Apr 15 '15 at 13:31
  • (1) Only ones you've measured. (3) It's easier to check the PH assumption using complementary log-log plots. For continuous predictors plot Schoenfeld residuals against survival time. Check linearity by plotting martingale residuals against each predictor. I couldn't follow your 2nd comment. – Scortchi - Reinstate Monica Apr 15 '15 at 13:50
  • I am afraid I do not understand how I can account for confounding effects. But I can read up on that. I will look into if I can plot complementary log-log plots in SAS. Also, I do not understand why I would check for linearity, but I will also read up on that.

    What I tried to explain in my second comment is that the difference of 73.31% and 72.34% (censored parts) is so small, so it surprises me that things turn out the way they do, even though p-values in general change when you add a predictor.

    Thank you for the help I've gotten so far. I understand I must read up on my own.

     – Erosennin Apr 16 '15 at 06:24
  • On confounders: if you can measure them you can include them in them model; if not you can only speculate about their existence & effects. On linearity: you're assuming that the relationship between log-hazard & any continuous predictor is linear, which could be a good approximation or a poor one. (If you don't have any continuous predictors it's irrelevant of course.) – Scortchi - Reinstate Monica Apr 23 '15 at 10:30