One of the assumptions of the OLS is that observations of the error terms should not be correlated with each other. This assumption is often denoted as $\text{Corr}(\epsilon_i,\epsilon_j) = 0$.
The confusion I have is the following: if $\epsilon_i$ and $\epsilon_j$ are errors, this should mean that they are numbers, and therefore their correlation should always be zero. If I got it wrong and $\epsilon_i$ and $\epsilon_j$ are random variables, please explain why they are.