2

This sort of question has been asked on this site before and I was partly satisfied with the answer here (How does centering the data get rid of the intercept in regression and PCA?)

where the images show that without the demeaning we might have our principal component not necessarily along the direction of maximum variance of our dataset. With that thread, I am still unclear as to why the principle component need to pass through the origin.

My other question is that I have seen comments to the affect that if the data is not demeaned before applying PCA, the first principle component will be the mean itself. Again, it is not clear to me why that should be the case and also why that is a problem exactly.

user42140
  • 399

1 Answers1

1

PCA only turns the axes around the origin. It doesn't move the center around which the turn happens.

Imagine that you have 3 variables. Each observation can be represented with a dot on the 3 dimensional plot. It's like a dot in the space of x,y,z coordinates. What PCA does is it turns the system of coordinates around the origin, so that the first PC points in the direction of the most variance. Each of three principal components will be the new axis. So, the dots do not move, but the system of coordinates turns and becomes PC1,PC2,PC3 instead of x,y,z.

Imagine that the means of your original three variables are so big, that all observations are in the cloud far away from the origin. In this case, turning the original coordinate system x,y,z will do nothing interesting: it'll only point PC1 to the center of the cloud, which is trivial and non-informative. The goal of PCA is not to point to where the cloud of observations is, but to inform us about the shape of the cloud.

However, when you de-mean the original variables, your de-meaned x,y,z system's origin will be inside the cloud of observation. In this case PCA has a better chance to work, because turning the coordinate system around the origin will probably make a difference: it'll inform us about the shape of the cloud of observations.

Aksakal
  • 61,310
  • Thanks for the answer. So if I understand correctly, the new coordinate system coming from PCA will be related through a rotation only? regarding my second question, if we do not perform this demeaning, why would the first component be the mean itself i.e. why would the direction pointing in the direction of the mean describe the maximum variance? – user42140 Oct 02 '17 at 14:19