Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [linear-model]

Refers to any model where a random variable is related to one or more random variables by a function that is linear in a finite number of parameters.

A linear model is any model where a random variable, $Y$, is related to one or more random variables, $X$, by a function that is linear in a finite number of parameters. That is, the parameters being estimated are all coefficients. Note that it does not matter if the resulting function looks like a straight line.

2551 questions
7
votes
1 answer

How to prove that Asymptotic Variance-Covariance matrix of OLS estimator is Positive Definite?

I'm trying to understand why the asymptotic Variance-Covariance($\text{Avar}(b)$) matrix of OLS estimator is Positive Definite(PD), like it's stated in Hayashi's book on page 113. We know that…
An old man in the sea.
  • 5,640
5
votes
3 answers

What makes a GLM estimate the means differently from the actual sample means?

Disclaimer: I'll happily admit to not really knowing what I'm doing when it comes to General Mixed Models but hopefully this thread can bring me a little bit closer to some kind of understanding. If, however, this is an "OMG, you're such a n00b"…
Speldosa
  • 688
5
votes
1 answer

Self-studying the normal linear model: what can I do with it?

I am currently self-studying the normal linear model ($X_i \sim N(A \beta, \sigma^2I)$), and have learned about estimating $\beta$ and $\sigma^2$, testing certain types of hypothesis (although not too complicated ones), creating confidence…
Markelo
  • 51
3
votes
2 answers

Excluding one combination of two factors in a linear model

Imagine you are modelling an outcome based on two discrete factors, factor1 and factor2, which can be yes or no. You could do something like: model <- lm(outcome ~ factor1 + factor2) Now suppose that out of the four combinations of values for the…
nico
  • 4,581
  • 3
  • 32
  • 45
3
votes
1 answer

Why is it theoretically satisfactory to predict a random variable using constants and parameters?

Suppose we model the random variable $Y$ as follows: $$\mathbb{E}[Y]=\beta_0+\beta_1x_1.$$ Now many statistics textbooks treat $\beta_i$ as parameters, which is simply constants (correct me if I am wrong), and $x_1$ as a observable varaibles. My…
Kun
  • 502
1
vote
1 answer

Marginal calculation in linear gaussian model

If we have a linear gaussian model where we have, $P(Z) = \mathcal{N}(\mu_z,\sigma^2_z)$ and $P(X|Z) = \mathcal{N}(w_o+zw,\sigma^2_x)$ How do we calculate the marginal $P(X) = \int_Z P(Z)P(X|Z)$ ?
user36047
  • 113
1
vote
3 answers

How does $y = ax + b + \epsilon$ induce a probability distribution on $R^2$?

This is with regards to the example given in the wikipedia article on statistical models. In the example it is claimed that "Each possible value of $\theta = (b_0, b_1, σ^2)$ determines a distribution on S," but no further explanation is given. I…
John Doe
  • 11
  • 2
1
vote
2 answers

What metrics can I use to compare the predictions of two linear regression models on an independent data set other than RMSE?

Model A was fit using 213 predictors, Model B with 252 predictors, on a data set with n=431. I test both on an independent data set (n=121) and am currently using RMSE to determine which model performs better. I am looking for more measures to…
sumthymes
  • 59
1
vote
1 answer

Why $var(y_a - \hat{y}_a)=\sigma^2(1+ x_a' (X' X)^{-1} x_a)$?

Let $y_a$ be an observed value and $\hat{y}_a$ be predicted value. Then I've read that $$var(y_a- \hat{y}_a)=\sigma^(1+ x_a'(X'X)^{-1}x_a)$$ but what's the proof for this? Additional info: $X_1=x_{a1}, ..., X_p = x_{ap}$ $x_a' = (1, x_{a1}, ...,…
mavavilj
  • 4,109
1
vote
2 answers

Linear model with multiple measures

Consider the following situation: I have several individuals (say 10), and for each of these individuals there is a covariate $x$ which linearly determines an outcome $y$. I can measure $y$ with some normal error, multiple times per individual (say…
Elvis
  • 12,666
1
vote
1 answer

How to treat x=0, y=0 in a linear model with no intercept?

Very often in the research we want to establish a linear relationship without intercept: $y=\beta x + \epsilon$ and the sample has many double zero observations $x=0, y=0$. I am wondering how to deal with these many zeros. On one hand, they are the…
tiantianchen
  • 2,101
0
votes
1 answer

How can I show that none of the other variables in the model were potential mediators

I got a revision for my research paper recently and the following is the reviewer's comment on my paper: the authors should show that for any estimate from the linear regression that is reported in this manuscript, none of the other variables in the…
zhiheng yi
  • 45
  • 4
0
votes
0 answers

Constraint Linear regression in finance

Below you can find the problem that I am trying to understand. My main problem is to understand where do the reparametrization that they propose come from. Alternatively I have try to connect it with the constraint LS KKT solution but I do not see…
glouis
  • 227
0
votes
1 answer

Linear regression in as a conditional expectation

We can show that the regression for two variables can be written as $$ E(\mathbf{Y} \mid \mathbf{X}=\mathbf{x}) = \alpha + \beta X $$ So for example if I consider let the gaussian vector Z=(X,Y) It means we have So Is my reasoning correct ?
glouis
  • 227
0
votes
1 answer

Correcting a linear model with non-significant coefficients

I have created a linear model based on 2 (interacting) independent nominal variables and 1 dependent interval variable. Most parameters I've found were statistically significant, but the parameters related to 1 value of an independent variable came…
HenryP96
  • 3
1
2