1

Suppose I have built a complicated non-linear regression model where the dependent variable is y and the independent variables are x1, x2 and x3. In that case, how can I answer the question

if I change the x3 variable by x%, what will be the change in the dependent variable y?

Here the independent variable x3 has a non linear relationship with y. This is kind of sensitivity analysis what I got to know.

utobi
  • 11,726
Mash
  • 11
  • I think "sensitivity analysis" involves a lot more than just asking the question you ask in your middle paragraph. But the term gets used differently in different fields. In my experience, it's used more for what if your model itself is wrong, or what if we have bias of some sort in our sample. But Googling showed that Investopedia has one definition; Wikipedia another; and neither match with what I've seen. So, more detail might be helpful – Peter Flom Sep 21 '23 at 14:50
  • 1
    @PeterFlom I could see it being a form of sensitivity analysis. For instance, if you think you're adding 20mL of X into a solution of Y, you expect result Z. However, what happens if you make a small measuring error and add 20.1mL? Do you get Z + 0.1 or Z + 1000? – Dave Sep 21 '23 at 20:08

1 Answers1

0

NONLINEARITY MAKES THIS COMPLICATED BUT NOT IMPOSSIBLE.

Let’s look at a fairly simple model that features a nonlinear relationship between the features and the outcome.

$$ \hat y =x_1^2x_2^2x_3^2 $$

If you want to know how $\hat y$ changes as a feature changes, the mathematical operations is to take a (partial) derivative.

$$ \dfrac{ \partial \hat y }{ \partial x_1 }=2x_1x_2^2x_3^2 $$

Therefore, how $\hat y$ changes as $x_1$ changes depends on the values of all three features. There is no simple description that $\hat y$ increases by $\hat\beta_1$ when $x_1$ increases by one unit like there is for a strictly linear relationship.

Therefore, the sensitivity analysis is going to be complicated, as the sensitivity is not constant. You might want to assess a number of changes from a number of starting conditions, depending on what matters to your work.

As a starting point, one idea I have seen done is to look at the difference between $\hat y$ when the features are at key values (say their means) vs the value of $\hat y$ when one feature is moved by a certain amount (say one standard deviation) and the other features are kept constant. One paper I remember doing something like this was Hoberg and Phillips (2010), section 4.2.

REFERENCE

Hoberg, Gerard, and Gordon Phillips. "Product market synergies and competition in mergers and acquisitions: A text-based analysis." The Review of Financial Studies 23.10 (2010): 3773-3811.

Dave
  • 62,186