5

I have data on variables ($y, x_1, x_2$), a linear model with the following form:

$$ y = \beta_0 + \beta_1x_1 + \beta_2x_2 $$

and a theoretical assumption that the residuals represent a latent variable $x_3$:

$$ \hat{y} - y = x_3 $$

What are the typical dangers of such a theoretical assumption?

jtd
  • 579
  • 1
  • 4
  • 11
  • Are X and Y time depedent? Are you going to use time-series techniques to model x3? I'm asking because I'm considering a similar assumption for a modeling problem with time-based data. – KishKash Mar 11 '15 at 04:20
  • By contrast, try to imagine what might happen without that assumption – shadowtalker Mar 11 '15 at 08:25

1 Answers1

1

I've seen the residuals considered with a prior in the model. Well, in a way. In a paper by a colleague, they show that using bayesian grouping hypotheses over B-splines accounts for perceptual grouping of skeletons really well. But there's a catch. The skeletons are fit with ribs of a certain length. This a necessary parameter in b-splines. The width is sensitive to the scale of the object: how big or small it is. Thus, the width has a gaussian prior over top of it. Though, I believe that they marginalize out such hyper parameters.

There is also another colleague who posits a 'regression to the mean' effect in human memory. In memory tasks, the recalled value of an episodic event (a color value, the height of a person, etc) has larger residuals when the value is further away from the prototypical value of that category (the prototype of blue, or heights of men). Thus, the entire theory revolves around the residuals!

I think the danger in predicting the residuals is not considering all of the alternate hypothesis. For example, perhaps the residuals are measurement error, observational bias, or some other instance which would give rise to deviations from a central tendency. If you plan on this route, I would make sure you model all possible alternative hypothesis and use some form of model comparison, perhaps AIC.

bcmcmahan
  • 86
  • Thanks for your thoughts and links! For you, dangers are: measurement error of IVs and DV away from true. What about confounding effects of other latent variables with effects of $x_3$? The memory study sounds interesting, although I wonder if comparable as it sounds like $X$ may be heteroskedastic and errors definitely biased to mean ($cov(\epsilon,X) \ne 0)? AIC sounds good but doesn't handle latent confounding latent. – jtd Mar 13 '15 at 14:47