Look in any finance textbook or search about CAPM, it will say that CAPM is a model about how portfolios have systematic risk (risk that can't be diversified away) and idiosyncratic risk (risk that can be diversified away).
But it appears to me that this is impossible to justified based on CAPM alone, which is a model about expected returns ($\mu_i-r_f=\beta_i(\mu_m-r_f)$), and this is actually an implication of the single-index model, which is a model about realized returns ($r_i-r_f= \beta_i(r_m-r_f)+\epsilon_i$).
In fact, the definition of idiosyncratic risk relies critically on the error term $\epsilon_i$. Without the error term, how can idiosyncratic risk and diversification even be defined?
It appears to me that CAPM says absolutely nothing about risk. You can't derive any statement about the variance of assets or portfolio returns just from a relation that is only about the mean of the returns.
My question is this: How is it possible to derive that beta is a measure of systematic risk, and that every asset has systemic risk that can't be diversified away and idiosyncratic risk that can be diversified away, based on CAPM alone and without relying on the single-index model? If it's not possible, it would appear that pretty much everyone is wrongly attributing to CAPM a conclusion that actually comes from the single-index model.
Edit: This observation has also been noted before in this question, but was brushed aside without addressing it.