20

In the credit risk industry (and finance industry as a whole, at least here in the UK), there is a very common and accepted 'proper' way to build scorecards.

The general framework seems to be:

  1. Binning your predictors, merging neighboring bins with similar Weight of Evidence (WOE) values, generally aiming for a monotonic relationship between the target and predictor
  2. Filter methods variable selection (calculating the Information Value [IV] of each predictor, removing those with a low IV)
  3. WOE-transform these predictors (target encoding), fitting a logistic regression model to the transformed data

I was wondering if anyone knows why this practice is followed? It seems like a dated and unusually specific approach, and I never see a blanket approach of binning/WOE encoding/IV filtering used in any other industry. I think an equivalent approach for regression tasks would be to bin and mean-encode all predictors before using them in linear regression, but I have never seen/heard of that used anywhere, including in credit risk.

I expand on my confusion below:

  • Why binning? It seems to me that binning numeric predictors discards information and adds so much arbitrariness and manual work to the process, and while it can help deal with outliers and missing values without having to think as much, so can creating flags to capture missing data, median imputation and winsorization, all of which can be easily automated and still leaves the option of adding spline terms, interaction terms, etc.
  • Why is information value used? If it's necessary to apply filter methods for variable selection (e.g. you have thousands of predictors), why not evaluate the performance (e.g. the out-of-sample AUC) of many 1-variable logistic regression models and filter based on that, which would have the benefit of aligning with the target metric
  • For a bin, the WOE is just the log-odds of that bin (plus a constant), so I suppose it would be a form of target encoding that is suited for logistic regression? This does make some sense to me once you've already binned your data (if target encoding actually improves performance), but I'm still left wondering why we binned the predictors in the first place? Most sources say to 'establish a monotonic relationship' between the predictors and the target, but logistic regression without nonlinear transformations and interaction terms can do that anyway, no? I feel like this just complicates model interpretation over fitting logistic regression models on the raw predictors, and using any form of target encoding always means you have to be that much more careful of target leakage

There must be a reason that this approach seems to be used in all the big banks and financial institutions but is rarely used elsewhere when preprocessing data for logistic regression. Are there any obvious strengths or reasons that I'm missing, or are there historical reasons for how this approach became so universal?

kjetil b halvorsen
  • 77,844
Liam Morgan
  • 469
  • 3
  • 12
  • 5
    I think your impressions are correct. Methodologies get built out with the tools available at the time, compromises and shortcuts are made, and the result becomes "the way we've always done it". Even if theoretical and technological developments mean the old ways are ripe to be reconsidered. Especially true of finance and other highly regulated industries. – Paul Mar 11 '22 at 18:10
  • 3
    This is an interesting question and gets an upvote from me! Be careful with what you propose in your second bullet point, however. Univariate variable screening is not ideal. – Dave Mar 11 '22 at 18:10
  • See also https://stats.stackexchange.com/questions/462052/intuition-behind-weight-of-evidence-and-information-value-formula/462445#462445, https://stats.stackexchange.com/questions/87182/what-is-the-role-of-the-logarithm-in-shannons-entropy/463828#463828 – kjetil b halvorsen Oct 22 '22 at 14:12

2 Answers2

2

At least for the US, it is due to regulatory reasons. The customer-facing risk models must be explainable and actionable. Some FIs, including mine, are already in favor of using spline-based models.

Also, you have to look into how it makes sense for the Business itself. For example, the inputs of the model. If you have loan or application-level features in your custom credit score (disastrous, but some FIs still do this), binning would allow the model to be more tolerant of those changes.

Let's say it's an automobile loan through a dealership where the loan is practically never the same as how it is submitted. I am promising my customer risk-based pricing of 4% for their loan, they are requesting X dollars more for tax, title, and fees which will affect the numeric features in my model.

What is $X (tied to payment, debt, LTV ratios) more in risk if it wasn't binned? Does it make sense for me to adjust my pricing and inconvenience my customers over <1 bps in additional risk?

Katsu
  • 21
1

As a person working on this industry, I believe that logistics regression is the easiest to explain your score card. There are so many machine learning advanced techniques for classification problem, but let's think the main problem of ML is unexplainable. When you devise a scorecard system, the salesperson in the future will ask why you score a customer like that (the salesperson might believe that's a good customer based on what he knows, you have to explain to him that he has signs of bad customers such as bad credit history somewhere else and that's what Logistics Regression could serve you well whilst ML cannot).

  1. Binning => It should be a manual work because you have to understand why the data look like that. It could be done automatically but the data might offer no meaning because the sample is biased.
  2. We do use a lot of filtering criteria (WOE, IV, Marginal IV). I think they are easy to calculate and explain to non-risk persons as well, which explained its popularity.
  3. I just think bining is necessary and transforming predictors improve the performance as you already wrote.
Kim Anh Vu
  • 11