Is the stationary distribution in a Markov chain just an average or will this probability distribution actually be reached?

Question

So I know that a connected Markov chain has a stationary distribution $\pi$ that satisfies

$$\lim_{t \rightarrow \infty} a_t = \pi,$$

where $a_t$ is the average probability distribution at time $t$. So this average converges to our stationary distribution. However, does that mean that $\lim_{t \rightarrow \infty} p_t = \pi$ (where $p_t$ is the probability vector of where the Markov chain is after $t$ steps)?

Just going off of the average definition, couldn't it be that the Markov chain bounces around two or more different probability distributions, so that the average is still $\pi$, but $\pi$ itself is never a probability distribution for the states of the Markov chain?

For example, say we have 3 states and $\pi = (0.2, 0.3, 0.5)$, then $p_{(1)} = (0.1, 0.2, 0.7)$ and $p_{(2)} = (0.3, 0.4, 0.3)$. Say for even $i$, $p_i = p_{(1)}$, and for uneven $i$, $p_i = p_{(1)}$, which means that the distribution of the process oscillates between $p_{(1)}$ and $p_{(2)}$. So the average $\lim_{t \rightarrow \infty} a_t$ is indeed $\pi$, but not $\lim_{t \rightarrow \infty} p_t = \pi$, since $p_t$ always switches between $p_{(1)}$ and $p_{(2)}$.

This is obviously a simple example, but I hope you understand my general question.

Answer 1

Here is a fairly general statement from Meyn & Tweedie (1994) that addresses the question: convergence to the invariant probability measure holds for all Harris recurrent aperiodic chains (without aperiodicity it fails!)

with an additional result about the unicity of the stationary measure provided by recurrence only

and another one about the convergence of the averages

where$$S_n(f)=\sum_{i=1}^n f(\it\Phi_i)$$