2

I am new in analytics field. Our team runs multiple model which does the demand forecasting for multiple product. To check whether the model is performing good or bad we calculate the MAPE and decide about model performance.

I have multiple forecasting run results(last month, this month, or may be 2-3 months back).

My question is basis on the MAPE of each run of the models what should be my criteria of MAPE if my model increased or decreased. for example last month's run gives the MAPE of 2% and this months gives MAPE of 3%.

in that case should i say that my model decreased ?

In general what percent change in MAPE of two versions should i neglect and what percentage change should i be worried off?

RohitM
  • 21

1 Answers1

1

My question is basis on the MAPE of each run of the models what should be my criteria of MAPE if my model increased or decreased. for example last month's run gives the MAPE of 2% and this months gives MAPE of 3%.

in that case should i say that my model decreased ?

This month's model has a larger MAPE. To this extent, it is "worse". However, note that the MAPE has some unintuitive features, which make me personally (and from what I see, the larger forecasting community) rather disinclined to use it: What are the shortcomings of the Mean Absolute Percentage Error (MAPE)?

In general what percent change in MAPE of two versions should i neglect and what percentage change should i be worried off?

We can't say in general. You need to figure out what level of MAPE means that your forecasts yield bad decisions. This will not only depend on the MAPE, but also on other parameters (e.g., logistical constraints, or contractual arrangements - the quality of the forecast may be the least important driver of the quality of subsequent decisions).

Also, note that MAPE is naturally variable, the same as any other statistical measure. It will fluctuate naturally from month to month, and small changes may simply be noise.

Stephan Kolassa
  • 123,354