10

In many SEM examples, we have noticed that covariances are specified between latent variables.

My questions are:

  1. What are the advantages of specifying covariance in SEM models?
  2. If there is high covariance within 2 variables, does it affect the direct effect?
  3. If there is high covariance within 2 variables, does it affect the indirect effect? Suppose we have 3 variables (x,y & z). x and y both affecting z. And there's covariance between x & y (represented by two-headed arrow in path diagram). Indirect effect means effect of x via y on z. So, if there's covariance between x & y, then will the indirect effect changes as well?
  4. Lastly, is there any literature specifically explaining the use of covariance in SEM models?

Thanks!

Beta
  • 6,334
  • Do you mean that the software is instructed to estimate covariances between factors (latent variables)? Or do you mean that specific values for these covariances are stipulated? – Placidia Jan 07 '15 at 13:04
  • When you say covariance between variables, do you mean between observed or latent variables? Can you be more specific with what you mean with "indirect effect"? – robin.datadrivers Jan 07 '15 at 14:20
  • @Placidia: I mean when we instruct software to estimate covariances between factors (latent variables) using double-headed arrow. – Beta Jan 07 '15 at 15:49
  • @Robin: it doesn't matter between observed or latent. Updated the question with meaning of indirect effect. – Beta Jan 07 '15 at 15:51

2 Answers2

7

The idea behind SEM and confirmatory factor analysis is to explain the covariances between the observed variables in terms of a smaller number of underlying factors -- unseen but presumed to exist. You could fit a model with uncorrelated factors, but allowing the factors to have correlations gives you extra parameters to improve the fit. Unless you have strong, theoretical reasons for positing independence, it makes sense to include the covariances. Otherwise, the model won't fit.

If factors X and Y are correlated (double arrow) and each impacts Z, there is no way to distinguish between the direct effect of X on Z and the effect of Y on Z through X. The effect exists, but it will be confounded with the direct effect of X on Z. However if X depends on Y (single arrow) and Z depends on X (single arrow), the correlation of Z and Y will increase if the correlation of X and Y increases.

The difference between covariances between latent factors and covariances between indicator variables (Observed) should matter to you. The whole point of SEM is to explain covariances between indicator variables in terms of factors. Conditional on the factors, the indicators should be independent. You can tweak the model to add correlations between the indicators if you feel you need them, but these are often a counsel of desperation employed when in truth there are no latent factors.

Exceptions exist. Suppose the indicators are something measured at successive time points. I can believe they might all depend on some underlying factors, and yet influence each other through a learning effect in the subject, say. In that case, it would make sense to have covariances in the indicators and covariances in the factors. But in general, I would want to see a solid, subject matter reason for introducing covariances at the indicator level.

Recommended texts

Sadly, I have yet to find a text on SEM that I really like. I think that Bollen's book, is the classic in this field, but it's pricey and I don't own a copy. I recently ordered Beaujean's book, which is focused on using lavaan in R. My copy hasn't arrived yet, so I don't know if it goes into the detail you need about path analysis. If you have access to a good library, or a lot of cash, I would check out Bollen.

Placidia
  • 14,361
  • Thanks a lot Placidia! The answer has been very thorough and clear.But could you please refer to me some literature on this. Specially the part where you have mentioned " there is no way to distinguish between the direct effect of X on Z and the effect of Y on Z through X". Actually I can see this from my model as well. But I want to know the theoretical underpining of it. – Beta Jan 08 '15 at 06:04
  • @Beta I have edited my response to give a couple of references. – Placidia Jan 08 '15 at 13:16
  • 1
    +1 I enjoyed reading your answer. I have no statistical background and consider myself a beginner in SEM (though I use it in my dissertation as main method). I have Beaujean's book and like it (mostly practical). Also have Kline's book on SEM - also pretty good for non-demanding person like me. But, currently, one of my favorite related books is Harlow's "The essence of multivariate thinking". – Aleksandr Blekh Jan 08 '15 at 13:42
  • 1
    (+1) Yet another reason to introduce correlated errors in a CFA model is the case where item content would result in correlated unique variances between pairs of items, whether those items truly target the same construct or because of poorly worded items (and whether this is assumed prior to collecting the data, or after using modification indices when fitting the hypothesised model). "Correlated uniqueness CFA models" were also used at some point for MultiTrait-MultiMethod (see, e.g., Brown (2006), Confirmatory Factor Analysis for Applied Research, Guilford Press). – chl Jan 08 '15 at 14:30
  • Thanks @chl: But I was not only looking the answer for CFA but also structural equation within SEM. But this reference is also helpful. Thanks again! – Beta Jan 08 '15 at 15:00
3

If you don't include the covariances between latent variables then cross loadings will be biased. That's because cross loadings fit the correlations between measurement items that measure the two latent variables, and so does the the factor covariance. If you mis-specify one of them you will affect the other. From there direct and indirect effects can be biased as well. If you don't have cross loadings you are fine - most everything will be fine due to some conditional model expressions.

TA72
  • 127
  • Thanks TA72 for your answer! But can you provide me the reference on the point you have mentioned. – Beta Jan 08 '15 at 04:49