Hierarchical forecasting - demand classification required for prediction?

Question

I have product sales data for which I would like to predict what will be the sales for each product at the product level, product store level, product store and region level etc.

To solve this problem, I was reading up online about this topic and came across some useful basis tutorials for hierarchical forecasting such as Nixtla hierarchical forecast and DARTS package.

However, in real world, there are products which experience has high demand, some products which has sporadic demand, some products which has erratic demand etc. There could be products which has no demand. So, you will see lot of zeros for demand and some non-zeros for such products

So my question is

should we first do "demand classification" using traditional ABC XYZ technique ([here])1 and then use it as one of levels in hierarchical forecasting? Or do forecasting only for products that has lot of data (stable demand etc)
How does hierarchical forecasting work for products with uneven time series, lot of zeros etc.how should we handle them?

Can someone share your views on this?

score 8 · Accepted Answer · answered Dec 07 '23 at 13:47

8

The first thing to keep in mind is that hierarchical reconciliation methods usually work on expectation forecasts.

On the one hand, this has implications on how to assess their quality (Kolassa, 2023, IJF).
On the other hand, in principle this means we can mix fast and slow selling products to our heart's content.
On the third hand, depending on what the reason for your forecasting is, expectation forecasts may not answer the question you are interested in. For instance, if you want to forecast for supply chain optimization, you are likely interested in safety stocks, for which you need quantile forecasts, which of course are not sum consistent, so "vanilla" hierarchical reconciliation won't help you here. Nowadays, there are concepts for probabilistic hierarchical consolidation in the academic literature, but I don't know whether DARTS or Nixtla's library support those.

Even for reconciliating expectation forecasts, especially if you mix fast and slow movers, some of the reconciled forecasts may end up negative if you use the simpler approaches. Or the reconciliation may adjust all series by roughly the same amount, which may be a small percentage change for fast movers, but a huge percentage change for slow movers. Again, I don't know whether your tools address this.

In the past, I used a nonlinear reconciliation with box constraints (to keep forecasts nonnegative) and a loss function that relied not on the absolute adjustments, but relative ones. Something like this may be useful.

As to whether an ABC/XYZ classification prior to hierarchical forecasting helps: that is an empirical question. I personally would first make sure my reconciled forecasts do not end up negative (your tools may already be taking care of that), and look at that "percentage change" issue. Once you have a working system, you could look at different ways of slicing your hierarchy, including such an analysis. It may improve matters, it may not; I think it would depend on your specific data.

answered Dec 07 '23 at 13:47

Stephan Kolassa

123,354

Is there any difference between the term "forecast" and "expectation forecast"?? – The Great Dec 07 '23 at 13:57
By probabilistic hierarchical reconciliation, I see the tool has these mentioned - "Probabilistic coherent predictions including Normality, Bootstrap, and PERMBU." Is this what you mean? – The Great Dec 07 '23 at 14:08
My intent to do demand classification was to avoid mixing up fast and slow moving parts (who may have different demand patterns). I thought doing demand classification first would prevent the mix up issue that you mentioned..is there any other way to avoid the mix up? – The Great Dec 07 '23 at 14:17
Yes, there are differences, e.g., between expectation forecasts and quantile forecasts, and people can be quite surprised if they don't think about this a bit. For instance, if you evaluate your forecast accuracy using MAE, you are implicitly asking for a median forecast, i.e., a 50% quantile forecast. For intermittent demand, that can mean that a flat zero forecast is "optimal" (as measured by MAE). You should first decide which forecast you want, then pick the correct accuracy measure. – Stephan Kolassa Dec 07 '23 at 14:23
Re probabilistic hierarchical reconciliation, it's a bit hard to judge based on only a snippet from the documentation... Papers such as this one are what I was thinking of. If concepts like these are included in these tools, great! – Stephan Kolassa Dec 07 '23 at 14:25
Re mixup: I would still say that you should first figure out what you need the forecast for, and what requirements that implies on the forecast. It may well be that you need quantile forecasts, but do not have enough data for (data-hungry) probabilistic reconciliation. Get something working first, then think about whether a classification improves accuracy. – Stephan Kolassa Dec 07 '23 at 14:27
to OP: Imagine category A contains widget p & q. E(sales (p) + sales(q)) = E(sales(A)), the equivalent is not true for median and other quantiles – seanv507 Dec 07 '23 at 16:16
1

Interesting citations (+1). These days I am mostly doing full-posterior Bayesian probabilistic forecasts using NUTS, but I would be revising that if I needed near-real-time forecast refreshing. Sometimes it is hard to beat the efficiency of a point estimate. – Galen Dec 08 '23 at 03:06
@TheGreat: I will try to have a look in the next days. – Stephan Kolassa Jan 02 '24 at 11:52

seanv507 · Answer 2 · 2023-12-07T11:18:04.757

so to handle demand problems you should start with a multiplicative model.

Considering clothing categories, perhaps socks are sold 100 x more than jeans, which is 100 times more than jackets.

Brands also have a multiplicative/percentage effect: Levis jeans might sell 10x more than Armani jeans.

Similarly this handles positive quantities only.

By taking logs a multiplicative model becomes a linear model.

$\log(\text{sales}) = \alpha \text{item_category} + \beta \text{brand}$

Now one problem with simply taking logs of sales, is how to handle no sales (ie $\log 0$).

A Poisson model considers $\log(\mathbf {E}(\text{sales}))$, so it does not have this problem (assuming there was at least a small probability that sales might have been non zero).

log-log regression beer sales with price (https://www.dummies.com/article/business-careers-money/business/economics/econometrics-and-the-log-log-model-156386/)

(https://rstudio-pubs-static.s3.amazonaws.com/466937_d13d384d18fe4529b239b8773f194768.html)

Hierarchical forecasting - demand classification required for prediction?

2 Answers2