1

At my organization, we use a deterministic formula mid-month to predict an end-of-month value. We have some data on the historical performance of this prediction. We know the value of the average prediction percent error ((Predicted - Actual)/Actual) and correspondingly we know the standard deviation of the prediction percent error. Can this historical information be used to produce a prediction interval (let's use 95%) when the predictions are made?

Dan
  • 63
  • 1
    Depending on your tooling, available data, and statistical approach, I would recommend either conformal prediction or fully Bayesian inference. – Galen Jan 25 '24 at 18:34
  • If you make certain assumptions about your conditional distributions, such as homoskedasticity, uncorrelated errors and some particular distribution, then yes, you should be able to transform average percentage errors into prediction intervals. However, percentage errors can be very problematic. Any particular reason why you would not use established methods like straight-up OLS or forecasting methods to create PIs? – Stephan Kolassa Jan 25 '24 at 20:17
  • @StephanKolassa This is a project that I've recently taken over and the aggregate data in use is, in some cases, a pretty complex set of aggregates, derived from dozens of departments. Projections may someday be better estimated using a regression model or some other approach, but for now, management demands additional metrics, so this is going to have to do. Thanks for the link to the MAPE article. FWIW, we're not using the absolute value of the pos and neg errors - I agree that would be problematic. – Dan Jan 25 '24 at 20:31

0 Answers0