5

in my statistics course, it was mentioned that for the Ordinary least squares method (OLS) taking variables $X_i$ and $Y_i$ as the response variable, the random variable is solely $Y_i$.

However for the Orthogonal regression method (ORM) both the $X_i$ and $Y_i$ are considered to be random variables.

From my understanding in the OLS we optimize the regression based on individually picked values, so for each $x_i$. However, in the ORM the optimization is not made for some fixed $x_i$ value. Is this the correct reason, or is there something else?

rannoudanames
  • 770
  • 1
  • 6
  • 23

1 Answers1

4

It's not true that, with OLS, only the dependent variable, $Y$, is random. In fact, both can be random variables, however OLS is centered on minimizing the mean squared deviation between $Y$ and $X\beta$, conditioned on the variable $X$. That is, if your equation is $$Y = X\beta + e$$ where $e$ is a vector of IID components, then the least squares problem is to find the vector $\beta$ such that $$\mathbb{E}[(Y- X\beta)^2\mid X]$$ is minimized.

In this sense, we can often "treat" the random variable $X$ as being "non-random", since we can pull it out of the operator $\mathbb{E}[ \cdot \mid X]$, however it still can be a random variable.

ORM has to do with dealing with cases where both $X$ and $Y$ contain measurment error. That is, when $X = X + \epsilon$ and $Y = y + \mu$ where $x,y$ are unknown fixed constants and $\epsilon, \mu$ are independent random variables.

whuber
  • 322,774
measure_theory
  • 565
  • 2
  • 7
  • 1
    Alright!, I understand now why the prof mentioned that the ORM can be more appropriate when there is uncertainties with both variables. – rannoudanames Oct 09 '16 at 00:13
  • My contribution is to add a little correction on Digio contribution. First of all, thank for your answer, it is very clear. It is not a particular comment but the fact that it is not $ X $ we are going to find but the vector $\beta$ who minimize the quantity $$ \mathbb{E} [ (Y - X \beta )^ 2 | X ] $$ – The goat Konan Dec 16 '23 at 03:41
  • @ThegoatKonan Thank you for noticing that. I have fixed this evident typographical error by changing "$X$" to "$\beta.$" – whuber Dec 16 '23 at 17:04