1

I've been looking at a paper for a while that I find interesting. It's essentially a comparative analysis where the authors are comparing PCA/PLS to different machine learning methods. The aim is to predict bond risk premia so they report MSPE and R^2_{OOS} as their main results. But the results are a bit puzzling to me. You see, they report these values for bond maturities 2,3,4 and 5 and both performance measures rise as the maturities do. Isn't this contradictory since a MSPE value is better the closer to 0 it is and R^2_{OOS} is better the higher it gets? Or is there something that I'm missing?

Thanks for reading.

(I've have attached the table of results) [1]: https://i.stack.imgur.com/BFgZm.png

Edit: At the request of a comment: A bond is essentially a loan and the maturity is the remaining lenght of the loan. So if a bond has a maturity of (for instance) 2, there are two years until the bond expires and the principal has to be repaid.

Dave
  • 62,186
Nbs610
  • 13
  • Welcome to Cross Validated! Many statisticians are interested in finance, but this is not a community where it should be expected that everyone knows what bond maturities $2$ through $5$ mean. Could you please clarify in an edit to the original question? – Dave Dec 15 '23 at 16:16

1 Answers1

0

Two components influence the out-of-sample $R^2$, and only one is the MSPE (mean squared prediction error). As the bond maturities change, yes, the MSPE increases, and if the $R^2$ denominators were the same, then the $R^2$ values would decrease. However, the denominator values need not stay the same.

$$ R^2_{oos}=1-\dfrac{ MSPE_{model} }{ MSPE_{baseline} } $$

If the numerator increases but the denominator also increases, then there could be an increase in our-of-sample $R^2$.

As an aside, looking at the linked table, watch about for those $R^2$ values below zero, which indicate terrible performance worse than a simple baseline like predicting the historical mean (likely how the denominator is calculated).

EDIT

Another possibility, though it turns out not to apply here, is that the paper calculates $R^2_{oos}$ by squaring the Pearson correlation between the true and predicted values, which loses the relationship with MSPE, discussed in my answer here and demonstrated graphically in my answer here. That the paper reports values below zero, however, rules out such a calculation of $R^2_{oos}$.

Dave
  • 62,186