1

I am studying normal linear regression and wanted to ask a question about its utility when working with independent RV. Suppose that we have for $k \in [1,\dots,n]$, $$Y_k = \beta_0 + \beta_1x_{k1} + \beta_2x_{k2} + \dots + \beta_qx_{kq} + \epsilon_k \quad \text{where } \epsilon_k \sim_{ind} N(0,\sigma^2)$$

We can write put all $Y_k$ in a vector $Y$, put the explanatory variables in a matrix $X$ and parameters in a vector $\beta$ and write $Y=X\beta + \epsilon $ where the disturbance vector $\epsilon \sim N(0,\sigma^2 Id)$.

However, in real life thinking, I do not see an utility to this matrix form if we are studying things that have no link. If $Y_1$ is the points you receive when asking a question in this website, $Y_2$ the salary of employees in a company, $Y_3$ the numbers of goals scored by a striker then these three only share parameters $\beta_i$ and explanatory variables can be totally different. So why do we choose to study them jointly ?

Kilkik
  • 345
  • $Y_1$, $Y_2$, $Y_3$ must be of the same type, e.g., all being "points you receive when asking a question in this website". They are individual observations of those points received. I can't say why you choose to study them jointly when they are of different types. It must be a mistake of your class. – Alex Oct 13 '22 at 18:55
  • @Alex Ok ! I think its more a misunderstanding from my side though. I thought that "explanatory variables are different" meant that they literally represent different things for example $x_{21}$ would have represented years of experience and $x_{31}$ a shot power. But if $Y_1, Y_2, Y_3$ are of the same type then $x_{21}$ and $x_{31}$ represent the same concept but with different numerical quantities. I think I got it – Kilkik Oct 13 '22 at 19:04

1 Answers1

0

You are correct, it wouldn’t make sense to have single model explaining completely unrelated things. $(y_i, X_i)$ are all of the same kind. For example $y_i$ is some economic indicator where $i$ is index for different countries and $X_i$ are the independent features for the countries (same for each country). Another example could be where $y_i$ is height of the child and $x_i$ height of the parent, where $i$ is an index for different parent-child pairs, as in the famous example by Sir Francis Galton.

Loosely speaking, this is what is meant by the assumption that the random variables are independent and identically distributed that you will see in different places. What it tells is that they are independent and of the same kind.

Tim
  • 138,066