0

I have an equation in the form of,

$$ ax + by = cz + dr + es $$

The variables are $x,y,z,r,s$ and the rest are the coefficients. In a book I see the author does a fit and finds values for all these coefficients. I am not sure however how to accomplish this using linear regression. In OLS I am used to expressions such as y ~ x + z, how can I fit the expression above in a form so that OLS will acccept it? I tried to leave only one variable on one side,

$$ ax + by - xz - dr = es $$

for example, so I could do s ~ x + y + z + r (I negated the data values of z and r), but that still does not give me the coefficient of s.

kjetil b halvorsen
  • 77,844
BBSysDyn
  • 1,012
  • 1
  • 9
  • 19
  • 1
    You can only get the coefficients in the units of $b$, i.e. $y \sim -\beta_xx+\beta_zz+\beta_rr+\beta_ss$, where $\beta_x=a/b$ etc. – Aksakal Jul 05 '16 at 14:34
  • 1
    I don't see how to do this within OLS. Far easier to use a general optimizing routine (like optim() in R) to optimize $a, b, c, d, e$ in order to minimize (say) the squared difference between the left and the right hand side. Add a number of constraints, or your optimizer will happily give you a perfect fit with $a=b=c=d=e=0$, which is probably not what you want. – Stephan Kolassa Jul 05 '16 at 14:35
  • Yeah - I had the same feeling, glad to hear a confirmation. Thanks, – BBSysDyn Jul 05 '16 at 14:37
  • 1
    You do this with PCA, not OLS. The model is of the form $(a,b,c,d,e)\cdot(x,y,-z,-r,-s)=0$. To minimize the sum of squares of residuals, find a hyperplane in $\mathbb{R}^5$ that best approximates the data $(x_i,y_i,-z_i,-r_i,-s_i)$. That hyperplane is generated by all but the last principal components. Another way to frame this is that you are attempting to identify near-collinearity among the five variables. That question is answered at http://stats.stackexchange.com/questions/16327. My answer there uses exactly your question to illustrate the technique. – whuber Jul 05 '16 at 15:18
  • 1
    Does this answer your question? Testing for linear dependence among the columns of a matrix – kjetil b halvorsen Aug 12 '20 at 01:39

0 Answers0