Let's say cor(A,B) > 0 and cor(B,C)>0. What can we say about cor(A,C)?
So, I am reasoning this way, if A is increasing, B is increasing, since the correlation is >0. Same holds between B and C, i.e. when B increases then C increases.
So according to my logic, cor(A,C)>0. So what's wrong with my intuition?
An explanation is highly appreciated.
First of all, note that it is not true to say that "if A is increasing, B is increasing, since the correlation is >0". You should say that "if A is increasing, the probability of B with greater values is increased, but it is possible for B to decrease".
Anyway, your proof about the relationship between the correlations is not true. It is possible that $cor(A,B)\neq$ 0 and $cor(B,C)\neq 0$ but $cor(A,C)=0$. This can be occurred in the v-structures in Bayesian networks. Consider the following example for the factors affecting the grade of a student in a course:
$\textrm{Student Intelligence} \rightarrow \textrm{Grade} \leftarrow \textrm{Comfortable Exam Environment}$
Obviously, Intelligence and Grade are positively correlated. The Comfortable Environment and Grade are also positively correlated. But, Intelligence and Comfortable Environment are independent (not correlated).
Finally, to find the problem with your intuition note that "correlation does not imply causation". Refer to the Wikipedia page for more information.
In the above example, Grade is correlated with Comfortable Environment, but Grade does not cause Comfortable Environment. So, it breaks the chain of reasoning mentioned in the question.
Let us consider a simple example of 2-dim vectors. We can calculate a correlation between them easy. Let A = (1,0), B = (1, 2), C = (-1,2). Then cor(A,B) = 1*1 + 0*2 = 1 > 0, cor(B,C) = 1*(-1) + 2*2 = 3 > 0, but cor(A,C) = 1*(-1) + 0*2 = -1 < 0. So, if we know a sign of cor(A,B) and cor(B,C) we can say anything about a sign of cor(A,C).
