0

Say we have the following multivariate regression model:

$ y = \beta_1 x_1 + \beta_2 x_2 + \varepsilon $

The OLS formula for the first coefficient looks like this

$ \hat{\beta}_1 = \frac{Cov(\tilde{y}, \tilde{x_1})}{Var(\tilde{x_1})} \tag{1} $

where

$\tilde{y} = y - \frac{Cov(y, x_2)}{Var(x_2)}x_2$

$\tilde{x_1} = x_1 - \frac{Cov(x_1, x_2)}{Var(x_2)}x_2$

I would like to express $\hat{\beta}_1$ as a function of the correlation between the regressors $x_1$ and $x_2$ as well as their volatility (I assume volatilites and correlation are random variables themselves and not static parameters). I managed to get a quite nice expression re-arranging terms and using a Taylor approximation around correlation(x_1,x_2)=0:

$ \hat{\beta}_1 = \frac{\rho_{y,x_1}\sigma_y}{\sigma_{x_1}} - \frac{\rho_{y,x_2}\sigma_{y}}{\sigma_{x_1}}\rho_{x_1,x_2}$

where $\rho$ is the correlation among two variables Now If I extend the model to a tri-variate case

$ y = \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \varepsilon $

yields

$ \hat{\beta}_1 = \frac{Cov(\tilde{y}, \tilde{x_1})}{Var(\tilde{x_1})} \tag{8} $

where

$$\tilde{y} = y - \frac{Cov(\tilde{y_2}, \tilde{x_2})}{Var(\tilde{x_2})}x_2 - \frac{Cov(\tilde{y_3}, \tilde{x_3})}{Var(\tilde{x_3})}x_3$$

$\tilde{x_1} = x_1 - \frac{Cov(\tilde{x_{12}}, \tilde{x_2})}{Var(\tilde{x_2})}x_2 - \frac{Cov(\tilde{x_{13}}, \tilde{x_3})}{Var(\tilde{x_3})}x_3$

with

$\tilde{x_2} = x_2 - \frac{Cov(x_2, x_3)}{Var(x_3)}x_3$

and

$\tilde{x_3} = x_3 - \frac{Cov(x_2, x_3)}{Var(x_2)}x_2$

$\tilde{y_3} = y - \frac{Cov(y, x_3)}{Var(x_3)}x_3$

$\tilde{y_2} = y - \frac{Cov(y, x_2)}{Var(x_2)}x_2$

I would like to use the same approach also for the trivariate case, but computations get very messy and they do not seem to lead to a nice expression. Is there a reference about someone trying to do this in the past? What other approach could I use?

user9875321__
  • 190
  • 2
    Use the results at https://stats.stackexchange.com/questions/107597 and a little bit of algebra to relate covariances to correlation coefficients. – whuber Feb 21 '24 at 18:22
  • good starting point, surely helps a lot. Are you aware specifically of someone trying to simplify those ugly expressions (for 3 or more regressors) e.g. via Taylor approximations? – user9875321__ Feb 22 '24 at 13:41
  • 1
    A Taylor approximation would seem to have no point, given there are such good analytical and numerical procedures available. – whuber Feb 22 '24 at 14:48
  • if you look at an expression when you have say 3 or 4 regressors and you want to get the coefficient of only one of those, you seem that even if you have a closed form solution, they are clearly heavy expressions. But I see your point, we don't get much by linearizing an ugly formula into something less ugly but still ugly xD – user9875321__ Feb 22 '24 at 17:35
  • 2
    There is a simple iterative procedure to obtain a single coefficient: see https://stats.stackexchange.com/a/166718/919. – whuber Feb 22 '24 at 17:44
  • 1
    Great answer that one, helps a lot! – user9875321__ Feb 23 '24 at 09:02

0 Answers0