I am reading Robert and Casella (2004) on Markov Chain Monte Carlo methods and, in particular, Section 6.7. This contains the ergodic theorem, which is stated as follows, where $S_n(f)$ denotes a partial average under function $f$:
If $(X_n)$ has a $\sigma$-finite invariant measure $\pi$, the following two statements are equivalent:
- If $f, g \in L^1(\pi)$ with $\int g(x) d\pi(x) \neq 0$, then $$\lim_{n \to \infty} \frac{S_n(f)}{S_n(g)} = \frac{\int f(x) d\pi(x)}{\int g(x) d\pi(x)}.$$
- The Markov chain $(X_n)$ is Harris recurrent.
Although the theorem seems to be stated for any sample path (sure convergence), it is later referenced as 'almost sure' convergence (e.g. page 269). However, just above the statement of the theorem, Robert and Casella write `if $(X_n)$ is Harris positive, if $S_n(g)$ converges almost surely [to the corresponding expected value under the stationary measure], this convergence occurs for any initial distribution $\mu$'. Is not this (and the way the theorem is stated) implying that we have sure convergence rather than just almost sure convergence? (The theorem is stated in terms of almost sure convergence in many other papers and books too, which further confuses me.)
