11

For example, say I'm creating a forecast for weather patterns and I am looking for a 95% prediction interval (essentially an upper and lower bound for that forecast); what is the name of that 95% parameter?

Researching I've found 'confidence level', 'coverage probability', 'coverage of the interval', and have seen it directly referred to as 'prediction interval' (though seems the term would be overloaded if this were the case).

Are any of these terms correct?

Stephan Kolassa
  • 123,354
Trevor Lowe
  • 111
  • 3
    "Confidence level" is decidedly an error. "Coverage" is a standard term. – whuber Jun 23 '22 at 21:21
  • 2
    @whuber Would you be able to clarify why "Confidence level" would not suffice? – Trevor Lowe Jun 23 '22 at 21:57
  • 1
    Because it's incorrect: prediction intervals are not confidence intervals. However, technical writers do use the term "confidence level" to describe the coverage. (I even quote an authority at https://stats.stackexchange.com/a/26704/919.) That might be why so many people are confused about the distinction. – whuber Jun 24 '22 at 12:09

1 Answers1

13

This is an issue that has been bugging me for the many years I have been active in forecasting (which you seem to be interested in, given your mention of prediction intervals, PIs): there does not seem to be a standard term.

For instance, the M5 uncertainty competition (see also the guidelines here) - one of the largest forecasting competitions, run by experts - requested nine different quantile forecasts, noting that they would give rise to four different central PIs. However, there is no term used for the proportion of data to be contained in such a PI, like 95%.

I personally sometimes simply use "level". I agree with whuber that "confidence level" would be bad, if only because it feeds into the common confusion between confidence intervals and PIs. Contra whuber, I would avoid "coverage", or at least qualify it as in "target coverage", because "coverage" alone is too close to the realized coverage rather than the target coverage or level for my taste. (Until the M5 competition, it was accepted wisdom among forecasters that PIs are usually too narrow, so the realized coverage is usually smaller than the target coverage or level.) Also, in my experience among forecasters, I can't recall having seen "coverage" in the sense of the target level - but other statistical subcultures may have other conventions.

Stephan Kolassa
  • 123,354