3

I want to test the prediction capabilities of an sarima model for long sequences. I want to predict the next [24 48 96] datapoints and calculate the mse and rmse. Can you help me find bibliography and any examples?

the pseudocode i use:

Algorithm: Rolling ARIMA Forecasting Input: Time series data Output: Root Mean Square Error (RMSE) of the forecast

  1. Split the input series into training (70%) and testing (30%) sets.

  2. Initialize the ARIMA model on the entire series.

  3. Define the size of the training set as 70% of the total series length.

  4. Assign the first 'size' elements of the series to the training set.

  5. Assign the remaining elements to the testing set.

  6. Initialize an empty list for storing predictions.

  7. For each time window in the testing set:

    a. Fit the SARIMA model on the current history (training data).

    b. Forecast the next prediction_length values using the fitted model.

    c. Append the forecasted values to the predictions list.

    d. Update the history with the actual observed values.

  8. Calculate the Mean Squared Error (MSE) between the test set and predictions.

  9. Compute the RMSE from the MSE.

  10. Return the RMSE.

This will run for each size(series)/prediction_length

Stephan Kolassa
  • 123,354
Creative T
  • 31

1 Answers1

4

Overall, your approach makes sense.

One point I would definitely change: move step 2 (which I presume refers to SARIMA order selection) either between step 6 and step 7, or into step 7.

Why? Selecting the SARIMA order on the entire series is a form of data leakage, so you should not do that. Conversely, you could select a specific order and then only update the parameters as you step through the series (this is what happens if this step is placed between steps 6 and 7), or alternatively select a new order at every new forecast origin (by putting this step within step 7). Both can be done, they just reflect different setups for your experiment.

We have resources on forecasting in this thread: Resources/books for project on forecasting models. Your approach is discussed in this FPP3 section.

Finally, it seems like you are considering seasonal cycles of length 24 (hours of day?). Note that SARIMA has well-known issues with long seasonalities - as, actually, does Exponential Smoothing. In addition, depending on what exactly you are modeling and forecasting, your intra-daily patterns may be combined with intra-weekly patterns, with weekdays showing a different pattern than weekends. This is not really an issue with temperatures (where you would see intra-yearly patterns), but with anything to do with human activities. You may want to take a look at our tag. Its tag wiki contains pointers to literature.

Stephan Kolassa
  • 123,354