Mimicking seasonaldummy with fourier in Arima model

Question

I'm trying to forecast data that has an hourly and weekly pattern. The model I made using predictors created using seasonaldummy does a nice job of picking up the hourly weekly pattern, but it takes a long time to train the model. I tried to create a similar forecast using fourier function, but it doesn't seem to be picking up the hourly pattern as well. Am I setting up fourier correctly to try to achieve the effect I've gotten with seasonaldummy? Should the frequency in ts be something other than 168? My data is hourly. I've provided some sample data below.

My end goal is to combine the predictors for the hourly weekly pattern with other predictors, that's why I'm not just using tbats. I've provided examples below of how I'm trying to combine dummy variables for the hourly weekly pattern with other predictors.

Code:

##BoxCox

TTTlambda <- BoxCox.lambda(tsData)

##Partitioning Time Series
EndTrain<-1344
ValStart<-EndTrain+1
ValEnd<-ValStart+336

tsTrain <-tsData[1:EndTrain]
tsValidation<-tsData[ValStart:ValEnd]
tsTest <- tsData[TestStart:TestEnd]

##Predictors
xregTrain<-dfPredictors[1:EndTrain,]
xregVal<-dfPredictors[ValStart:ValEnd,]
xregTest<-dfPredictors[TestStart:TestEnd,]

##Seasonal Dummies
x=ts(tsData,freq=168) 
dummies=seasonaldummy(x)
xreg2Train<-dummies[1:EndTrain,]
xreg2Val<-dummies[ValStart:ValEnd,]
xreg2Test<-dummies[TestStart:TestEnd,]


##Fourier Terms

tsTTT<-ts(tsData, freq=168)

bestfit <- list(aicc=Inf)
for(i in 1:25)
{
  fit <- auto.arima(tsTTT, xreg=fourier(tsTTT, K=i), seasonal=FALSE)
  if(fit$aicc < bestfit$aicc)
    bestfit <- fit
  else break;
}

bestfit$coef ## K=2

xreg3<-fourier(tsTTT,2)

xreg3Train<-xreg3[1:EndTrain,]
xreg3Val<-xreg3[ValStart:ValEnd,]
xreg3Test<-xreg3[TestStart:TestEnd,]


##hourly weekly
Arima.fit_D <- auto.arima(tsTrain, lambda = TTTlambda, xreg=xreg2Train, stepwise=FALSE, approximation = FALSE, seasonal = FALSE )

Arima.fit_D_P <- auto.arima(tsTrain, lambda = TTTlambda, xreg=cbind(xreg2Train,xregTrain$Predictor), stepwise=FALSE, approximation = FALSE, seasonal = FALSE )

##Fourier hourly weekly
Arima.fit_F <- auto.arima(tsTrain, lambda = TTTlambda, xreg=xreg3Train, stepwise=FALSE, approximation = FALSE, seasonal = FALSE )

Arima.fit_F_P <- auto.arima(tsTrain, lambda = TTTlambda, xreg=cbind(xreg3Train,xregTrain$Predictor), stepwise=FALSE, approximation = FALSE, seasonal = FALSE )

##Forecast Model

Acast_D<-forecast(Arima.fit_D,xreg=xreg2Val, h=336)

Acast_D_P<-forecast(Arima.fit_D_P,xreg=cbind(xreg2Val,xregVal$Predictor), h=336)

Acast_F<-forecast(Arima.fit_F,xreg=xreg3Val, h=336)

Acast_F_P<-forecast(Arima.fit_F_P,xreg=cbind(xreg3Val,xregVal$Predictor), h=336)

Data:

dput(tsData[1:1681])

c(11, 14, 17, 5, 5, 5.5, 8, NA, 5.5, 6.5, 8.5, 4, 5, 9, 10, 11, 7, 6, 7, 7, 5, 6, 9, 9, 6.5, 9, 3.5, 2, 15, 2.5, 17, 5, 5.5, 7, 6, 3.5, 6, 9.5, 5, 7, 4, 5, 4, 9.5, 3.5, 5, 4, 4, 9, 4.5, 6, 10, NA, 9.5, 15, 9, 5.5, 7.5, 12, 17.5, 19, 7, 14, 17, 3.5, 6, 15, 11, 10.5, 11, 13, 9.5, 9, 7, 4, 6, 15, 5, 18, 5, 6, 19, 19, 6, 7, 7.5, 7.5, 7, 6.5, 9, 10, 5.5, 5, 7.5, 5, 4, 10, 7, 5, 12, 6, NA, 4, 2, 5, 7.5, 11, 13, 7, 8, 7.5, 5.5, 7.5, 15, 7, 4.5, 9, 3, 4, 6, 17.5, 11, 7, 6, 7, 4.5, 4, 4, 5, 10, 14, 7, 7, 4, 7.5, 11, 6, 11, 7.5, 15, 23.5, 8, 12, 5, 9, 10, 4, 9, 6, 8.5, 7.5, 6, 5, 8, 6, 5.5, 8, 11, 10.5, 4, 6, 7, 10, 11.5, 11.5, 3, 4, 16, 3, 2, 2, 8, 4.5, 7, 4, 8, 11, 6.5, 7.5, 17, 6, 6.5, 9, 12, 17, 10, 5, 5, 9, 3, 8.5, 11, 4.5, 7, 16, 11, 14, 6.5, 15, 8.5, 7, 6.5, 11, 2, 2, 13.5, 4, 2, 16, 11.5, 3.5, 9, 16.5, 2.5, 4.5, 8.5, 5, 6, 7.5, 9.5, NA, 9.5, 8, 2.5, 4, 12, 13, 10, 4, 6, 16, 16, 13, 8, 12, 19, 19, 5.5, 8, 6.5, NA, NA, NA, 15, 12, NA, 6, 11, 8, 4, 2, 3, 4, 10, 7, 5, 4.5, 4, 5, 11.5, 12, 10.5, 4.5, 3, 4, 7, 15.5, 9.5, NA, 9.5, 12, 13.5, 10, 10, 13, 6, 8.5, 15, 16.5, 9.5, 14, 9, 9.5, 11, 15, 14, 5.5, 6, 14, 16, 9.5, 23, NA, 19, 12, 5, 11, 16, 8, 11, 9, 13, 6, 7, 3, 5.5, 7.5, 19, 6.5, 5.5, 4.5, 7, 8, 7, 10, 11, 13, NA, 12, 1.5, 7, 7, 12, 8, 6, 9, 15, 9, 3, 5, 11, 11, 8, 6, 3, 7.5, 4, 7, 7.5, NA, NA, NA, NA, 6.5, 2, 16.5, 7.5, 8, 8, 5, 2, 7, 4, 6.5, 4.5, 10, 6, 4.5, 6.5, 9, 2, 6, 3.5, NA, 5, 7, 3.5, 4, 4.5, 13, 19, 8.5, 10, 8, 13, 10, 10, 6, 13.5, 12, 11, 5.5, 6, 3.5, 9, 8, NA, 6, 5, 8.5, 3, 12, 10, 9.5, 7, 24, 7, 9, 11.5, 5, 7, 11, 6, 5.5, 3, 4.5, 4, 5, 5, 3, 4.5, 6, 10, 5, 4, 4, 9.5, 5, 7, 6, 3, 13, 5.5, 5, 7.5, 3, 5, 6.5, 5, 5.5, 6, 4, 3, 5, NA, 5, 5, 6, 7, 8, 5, 5.5, 9, 6, 8.5, 9.5, 8, 9, 6, 12, 5, 7, 5, 3.5, 4, 7.5, 7, 5, 4, 4, NA, 7, 5.5, 6, 8.5, 6.5, 9, 3, 2, 8, 15, 6, 4, 10, 7, 13, 14, 9.5, 9, 18, 6, 5, 4, 6, 4, 11.5, 17.5, 7, 8, 10, 4, 7, 5, 9, 6, 5, 4, 8, 4, 2, 1.5, 3.5, 6, 5.5, 5, 4, 8, 10.5, 4, 11, 9.5, 5, 6, 11, 21, 9.5, 11, 13.5, 7.5, 13, 10, 7, 9.5, 6, 10, 5.5, 6.5, 12, 10, 10, 6.5, 2, 8, NA, 10, 5, 4, 4.5, 5, 7.5, 12, 22, 5, 8.5, 2.5, 3, 10.5, 4, 7, 13, 4, 3, 5, 6.5, 3, 9, 9.5, 16, NA, 4, 12, 4.5, 7, 5.5, 8, 14, 3, 8, 12, 14, 7, 8, 6, 8.5, 6, 6.5, 15.5, 13, 3.5, 12, 7, 6, NA, 3, 5.5, 8.5, 9, 12, 13, 8, 6.5, 8, 3, 5, 16.5, 2, 7, 6, 2, 5, 6.5, 3, 3, 7, 2, NA, 13, 7, 16, 13, 12.5, 12, 7, 13, 11, 21.5, 16, 20, 3, 4, 5, 7, 11, 7, 9, 11, 7, 13, 4, 14, 5, 12, 6, 7, 9, 12, 7, 12.5, 6.5, 16, 5, 12, 9, 9.5, 9, 7, 9.5, 3, 13, 8, 7, 7, 7, 9, 6, 6, 11, 15, 9, 6, 19, 10.5, 4, 6, 14.5, 9, 17, 14, 4, 16, 5, 6.5, 10, 9, 17, 11.5, 3, 5, 9, 8, 16, 10, NA, 7, 5, 12.5, 12, 11, 3, 3, 3.5, 14, 12, 7, 4, NA, 6, NA, 6, 10, 8, 10, 2, NA, 4, 5.5, 14, 4, 4.5, 8.5, 13, 21, 10, 11.5, 18, 5, 3, 2, 6, 11, 3, 7.5, 6, 3, 5, 9, 7.5, 7.5, 5, 9, 17, 3, 9.5, 5.5, 9.5, 15, 14.5, 10, 9, 13.5, 12, 12, 3, 11, 6, 4, 8, 17.5, 7.5, 7.5, NA, 7, 4, 6, 6, 6, 6, 6, 5, 8.5, 6, 6, 5, 6, 7, 5, 5, 5, 5, 7, 6, 8, 14, 6.5, 9.5, 5, 18.5, 5, 8, 10, 16, 12, 13, 7, 6, 13, 9, 18, 17, 8, 7, 3, 8, 2, 9, 11, 5, 2, 5.5, 6.5, 7, 10, 2, 3, 2, 3, 5, 4, 5, 6, 3, 5, 3.5, 5, 4, 9, NA, 10.5, 16, NA, 11, 8.5, 13, 4, 12.5, 12, 13, 18.5, 21, 5, 9, 4.5, 3, 3, 4, 3, 4, 4, 2, 8, 4.5, 4, 5, 9, 5, 4.5, 4, 7.5, 6, 7, 22, 5, 8, 5, 7, 4, 8, 5.5, 3, 8, 7, 6, 7.5, 6, 15, 13.5, 10, 7, 2.5, 7.5, 9, 9.5, 8, 19, 8, 8, 10, 6, 9, 5, 4.5, 9, 3.5, 4, 3.5, 8, 5, 3.5, 8.5, 9, 12.5, 7, 8, 10.5, 10, 1.5, 5, 10, 9, 2, 5, 8, 11, 3, 4.5, 2, 8.5, 4, 8, 2, 3, 4, 5.5, 2, 4, 6, 4.5, 6, 6.5, 0, 2, 3.5, 10, 7, 14, 14, 12.5, 3, 7, 8, 3, 7, 12, 12.5, 2, 2.5, 3, 9, 10.5, 8, 6, 6.5, 8.5, 5, 10.5, 9, 3.5, 7, 5, 8, 5, 5, 5.5, 4, 9, 8, 5.5, 5, 6, 10.5, 4, 9, 6, 5, 11, 10.5, 10.5, 4, 11.5, 11, 6, 2, 9, 5, 9, 5, 5.5, 7, 4, 10, 5, 3, 9, 9, 19.5, 13, 6, 15, 7, 10, 8, 10.5, 8, 16, 7, 10.5, 8.5, 10.5, 8, 8, 7, 5, 5, 6, 6, 5, 4, 9, 6.5, 4, 7, 7, 5, 4, 7, 6, 3, 6, 8.5, 8.5, 4, 5.5, 7, 8, 5, 6, 3, 9, 12, 6, 7.5, 4, 3, 5.5, 2, 5.5, 7, NA, 8.5, 2, 5, 8, 8, 4, 3, 6, 4, 4.5, 5, 3, 7.5, 9, 13, 8, 10, 12, 6.5, 3, 3.5, 8.5, 9, NA, 12, 8, 9, 4, 6, 8, 8, 9.5, 8, 6, 5, 4, 10.5, 6.5, 4, 3.5, 5, 7, 7, 5, 9, 6, NA, 6, 6, 5, 10, 7, 9, 9, 5, 4, 5, 4, 6, 8, 5, 3, 2.5, 2.5, 13, 4.5, 2.5, 2, 3, 9.5, 3, 5.5, 6, 10, 9, 10, 13, 14.5, 9, 7, 6, 5, 4, 4, 4, 5, 6.5, 11, 13.5, 11, 12, 3, 3, 14, 11, 6, 8, 5.5, 9, 8, 8, 7, 7, 5.5, 3.5, 10.5, 6, 5.5, 8, 8, 15, 6.5, 8, 9.5, 6.5, 5, 7, 6, 4, 14.5, 4, 2.5, 5, 8, 18, 13, 10, 6, 7, 18, 4.5, 7, 6.5, 5, 17, 7, 3, 5.5, 4, 6.5, 5.5, 6, 8, NA, 9.5, 14, 9, 11, 8, 7, 17, 7, 8, 8, 9, 2, 2, 4, 3, 8, 4, 9, 6, 9, 11, 13, 7.5, 8.5, 6, 6, 10, 17.5, 18.5, 14, 8.5, 4, 5, 6, 3, 2, 4, 4, 12, 11, 5, 2.5, 8, 6, 10, 5, 8, 8, 10.5, 14, 7, 16, 15, 6, 4.5, 10, 19, 3, 3, 4.5, 6.5, 4, 7.5, 8, 6, 20, 6, 7, 13, 13, 4, 10, 6, 5, 4.5, 6, 10, 6, 4, 8.5, 7.5, 3, 3.5, 3, 2, 2, 20.5, 6, 18, 5.5, 7.5, 5, 3.5, 8, 6, 6.5, 3, 4, 8, 5, 15.5, 4, 5, 8, 5, 3, 4, 5, 3, 3, 3, 6, 4, 12, 8, 10, 12, 5.5, 9.5, NA, 5, 4.5, 7, 16, 7, 4.5, 5, 5, 10, 6, 19, 8, 15, 7, 19.5, 10, 7.5, 9, 9, 7, 8, 3, 6, 5.5, 6, 7, 8, 14, 8, 13, 5.5, 3.5, 5, 9, 4.5, 4, 4, 3, 7.5, 4, 5, 6.5, 9, 4, NA, 12, 5.5, 6, 12.5, 6.5, 6.5, 5, 11, 4.5, 8, 2, 4, 5, 5, 3, 2.5, 6, 7, 4, 17, 4, 3, 5, 6, 2, 8, 8.5, 6.5, 4, 10, 12.5, 11, 6.5, 9, 12.5, 5.5, 5, 7.5, 16, 11.5, 4, 5.5, 3.5, 4, 3, 6, 4, NA, 5, 6, 7, 3, 4.5, 7, 5.5, 4, 7, 11, 7, 3, 3, 4, 3.5, 9, 4.5, 8, 5, 6, 8, 5, 5.5, 8, 5, 9, 8, 8, 6.5, 6, 10, 7, 7, 9, 12, 8, 13, 6.5, 6, 4, 5.5, 6, 3, 7, 8, 15, 10, 8, 3, NA, 5, 7, 7, 6, 9, 19, 13, 7, 7.5, 11, 8.5, 4, 7.5, 6, 13.5, 17, 9, 5, 6.5, 6, 4, 5, NA, 3, 6, 10.5, 6, 14, 6, 9.5, 6, 10, 11, 10, 3, 7, 9, 16.5, 5.5, 12.5, 8, 5, 10, 6, 1, 5, 6.5, 10, 8.5, 5, NA, 9.5, 13, 10, 10, 20, 7, 8, 5, 3, 3, 4.5, 3.5, 2, 5, 11, 3, 7.5, NA, 5.5, NA, 6, 6, 11, 12, 7, 5, 15, 11, 6, 17.5, 13.5, 16, 16.5, 5, 4, 3, 5.5, 3, 8, 11, 8, 12, 14, NA, 10, 6, 4, 5, 8, 10, 12.5, 6, 3, 6, 5, 8, 6, 11, 12.5, 7, 6, 9.5, 2, 8.5, 9.5, 8, 8, 2, 7.5, NA, 6, 2.5, 4, 5, 5, 6, 9, 4, 7, 6, 2, 4.5, 3, 4, 4, 5, 4, 3, 7.5, 8.5, NA, 12, 9, 11, 9, 3, 2.5, 7, 4, 4, 7, 8.5, 12.5, 3.5, 6.5, 10, 6, 8, 7, 13, 13.5, 12, 13, 8, NA, 8, 9, 15, NA, 4, 3.5, 2, 7, 8, 7.5, 9.5, 1.5, 5, 4, 8, 11, 5, 12, 4, 3, 11, 8, 7.5, 5.5, 13, 11, NA, 12, 7, 8, 6, 13, 8, 5, 4, 7, 8, 2, 3, 4, 4, 5, 5.5, 5.5, 6, 5, 6, 14, 12, 6, 11.5, 13, 5, 5, 8, 9, 2, 5, 6, 10, 4.5, 4, 7, 7.5, 7, 4, 10, 6.5, 6, 10)

dput(dfPredictors$Predictor[1:1681])

c(2, 6, 3, 5, 3, 2, 2, NA, 2, 6, 12, 11, 9, 10, 13, 9, 11, 7, 12, 8, 6, 4, 10, 6, 2, 7, 2, 1, 3, 2, 1, 3, 8, 7, 7, 8, 13, 13, 13, 11, 12, 4, 12, 18, 12, 7, 5, 4, 6, 4, 3, 3, NA, 4, 2, 8, 8, 8, 7, 3, 5, 3, 7, 8, 7, 7, 11, 8, 10, 3, 10, 6, 5, 5, 3, 1, 2, 1, 1, 3, 4, 8, 8, 5, 9, 12, 12, 11, 8, 5, 9, 10, 7, 8, 4, 6, 4, 1, 3, 1, 3, NA, 2, 1, 4, 10, 7, 13, 6, 9, 6, 16, 12, 11, 10, 12, 9, 7, 7, 7, 6, 2, 3, 1, 1, 2, 2, 3, 11, 10, 9, 8, 9, 13, 6, 6, 10, 9, 11, 10, 8, 7, 6, 4, 2, 3, 5, 3, 2, 4, 4, 4, 8, 5, 12, 8, 7, 12, 9, 12, 12, 12, 13, 12, 9, 8, 9, 10, 4, 7, 4, 2, 2, 4, 1, 7, 6, 6, 8, 11, 11, 5, 7, 6, 9, 12, 15, 9, 11, 5, 10, 5, 4, 4, 2, 3, 3, 2, 5, 4, 7, 8, 6, 6, 5, 12, 10, 8, 10, 10, 4, 13, 12, 6, 8, 6, 3, 1, 4, 2, NA, 4, 3, 2, 6, 5, 8, 10, 4, 13, 2, 13, 8, 11, 13, 8, 9, 10, 9, 5, 1, NA, 1, 1, 2, NA, 1, 7, 6, 10, 7, 8, 12, 12, 9, 5, 6, 8, 13, 13, 13, 8, 8, 1, 5, 7, 6, 2, NA, 2, 1, 2, 7, 9, 12, 12, 10, 10, 10, 6, 8, 2, 8, 3, 4, 5, 6, 2, 2, 1, 4, 1, NA, 3, 1, 3, 8, 8, 11, 11, 12, 5, 7, 14, 9, 10, 14, 11, 8, 6, 8, 7, 5, 4, 3, 4, 9, NA, 2, 4, 5, 8, 2, 12, 8, 15, 12, 8, 9, 12, 9, 9, 12, 7, 7, 8, 7, 5, 4, NA, 1, NA, NA, 4, 9, 8, 8, 8, 12, 13, 7, 11, 8, 14, 12, 13, 15, 8, 6, 4, 4, 5, 2, NA, 2, 5, 4, 5, 6, 15, 11, 10, 16, 10, 5, 5, 10, 13, 10, 9, 8, 7, 5, 4, 5, 6, NA, 2, 5, 4, 1, 6, 5, 8, 4, 3, 10, 11, 8, 12, 10, 10, 10, 12, 10, 10, 7, 5, 7, 3, 4, 3, 3, 3, 3, 8, 4, 8, 10, 5, 10, 10, 10, 11, 10, 11, 7, 10, 7, 6, 7, 7, 3, 3, NA, 3, 6, 5, 3, 3, 5, 6, 6, 13, 14, 14, 7, 13, 9, 10, 4, 9, 10, 8, 3, 6, 10, 5, 2, 1, NA, 3, 4, 4, 12, 12, 11, 12, 11, 13, 10, 9, 11, 11, 14, 10, 13, 10, 7, 11, 1, 3, 1, 4, 1, 2, 2, 3, 9, 6, 9, 9, 8, 9, 7, 12, 17, 13, 9, 10, 8, 8, 10, 2, 3, 3, 6, 2, 2, 1, 6, 8, 7, 9, 5, 11, 8, 8, 12, 13, 14, 10, 7, 5, 11, 11, 8, 5, 7, 3, 2, 3, 5, NA, 1, 3, 3, 4, 9, 12, 12, 3, 5, 12, 10, 9, 14, 15, 12, 7, 8, 7, 3, 4, 1, 5, 6, 4, NA, 5, 9, 6, 7, 8, 15, 13, 9, 12, 9, 7, 7, 6, 7, 8, 8, 6, 4, 5, 4, 1, 5, 1, NA, 5, 4, 12, 7, 20, 12, 14, 10, 11, 11, 12, 6, 6, 11, 5, 6, 7, 4, 7, 5, 1, 2, NA, 2, 7, 16, 9, 4, 12, 14, 12, 9, 8, 12, 7, 6, 11, 9, 15, 9, 4, 4, 3, 3, 2, 5, 2, 1, 6, 8, 3, 12, 11, 14, 9, 6, 3, 12, 11, 10, 14, 10, 10, 12, 2, 3, 3, 5, 3, 2, 3, 3, 5, 9, 5, 10, 14, 9, 14, 11, 9, 12, 9, 15, 13, 12, 15, 11, 4, 7, 3, 3, 3, 2, 5, 5, 11, 4, 2, NA, 3, 6, 10, 8, 5, 9, 9, 10, 11, 8, 9, 8, NA, 3, NA, 1, 1, 4, 3, 3, NA, 4, 8, 3, 9, 6, 12, 9, 7, 11, 6, 6, 12, 5, 4, 11, 7, 1, 2, 3, 2, 4, 8, 2, 6, 5, 9, 3, 7, 8, 8, 8, 14, 10, 12, 5, 12, 9, 13, 7, 3, 5, 3, 4, 2, 4, 2, NA, 5, 5, 9, 8, 7, 11, 9, 5, 6, 10, 13, 10, 9, 16, 11, 7, 5, 6, 2, 5, 3, 5, 2, 2, 6, 5, 11, 7, 13, 6, 10, 7, 9, 7, 8, 9, 12, 7, 7, 5, 3, 5, 3, 3, 5, 3, 1, 2, 10, 11, 8, 1, 9, 10, 14, 12, 7, 11, 11, 10, 7, 5, 9, 8, 6, NA, 4, 2, NA, 3, 4, 4, 6, 10, 9, 6, 8, 9, 10, 9, 14, 12, 8, 11, 16, 16, 13, 8, 7, 4, 4, 1, 3, 4, 2, 2, 5, 9, 9, 3, 8, 12, 6, 11, 10, 6, 8, 15, 12, 12, 7, 6, 7, 3, 2, 1, 3, 2, 2, 5, 7, 11, 8, 3, 4, 5, 5, 9, 6, 10, 9, 7, 17, 12, 3, 8, 6, 4, 4, 4, 3, 4, 2, 1, 4, 7, 12, 5, 4, 8, 7, 15, 8, 6, 6, 10, 5, 10, 7, 3, 4, 4, 1, 5, 3, 4, 5, 2, 1, 8, 6, 8, 9, 7, 13, 11, 11, 6, 9, 9, 9, 9, 8, 1, 8, 1, 3, 2, 2, 1, 3, 2, 3, 8, 9, 10, 12, 6, 9, 12, 7, 8, 8, 14, 10, 10, 8, 10, 4, 4, 4, 4, 3, 2, 4, 2, 3, 16, 3, 9, 8, 15, 9, 11, 10, 12, 12, 7, 15, 13, 8, 9, 3, 8, 1, 2, 2, 3, 3, 3, 10, 7, 5, 10, 4, 8, 8, 10, 8, 9, 9, 6, 7, 7, 4, 6, 8, 4, 5, 5, 1, 1, 1, 5, 7, 3, 3, 12, 8, 7, 10, 7, 12, 11, 7, 6, 14, 13, 9, 14, 3, 4, 6, 3, 2, 3, NA, 2, 3, 7, 6, 9, 7, 5, 9, 8, 10, 7, 7, 6, 11, 11, 9, 7, 5, 2, 3, 2, 2, 5, NA, 2, 5, 6, 7, 8, 14, 11, 6, 9, 10, 10, 9, 8, 16, 9, 6, 6, 3, 2, 5, 1, 1, NA, 3, 3, 2, 5, 10, 9, 10, 13, 11, 8, 17, 13, 5, 11, 10, 11, 6, 9, 6, 4, 5, 6, 5, 2, 4, 1, 2, 5, 9, 12, 10, 10, 18, 9, 13, 16, 8, 13, 5, 16, 11, 8, 10, 3, 6, 3, 1, 2, 2, 6, 2, 12, 9, 5, 8, 10, 11, 4, 10, 9, 9, 9, 19, 11, 8, 9, 4, 4, 4, 7, 5, 2, 2, 1, 2, 6, 10, 11, 12, 9, 9, 12, 9, 8, 14, 8, 5, 3, 5, 7, 6, 3, 4, 6, 3, 3, NA, 2, 2, 7, 12, 11, 14, 7, 10, 10, 13, 12, 5, 8, 8, 9, 10, 4, 9, 4, 5, 3, 1, 4, 2, 1, 7, 9, 10, 10, 11, 9, 8, 6, 3, 8, 10, 9, 9, 10, 8, 11, 4, 1, 1, 3, 1, 1, 3, 6, 2, 3, 10, 4, 9, 6, 10, 11, 6, 7, 8, 6, 11, 8, 4, 8, 5, 5, 1, 7, 1, 3, 3, 3, 6, 6, 8, 8, 9, 7, 6, 10, 9, 10, 6, 13, 10, 20, 7, 9, 2, 6, 3, 2, 1, 1, 2, 1, 3, 13, 7, 6, 7, 11, 7, 9, 7, 9, 10, 9, 10, 6, 11, 7, 5, 5, 5, 4, 6, 4, NA, 4, 4, 11, 11, 9, 8, 9, 9, 12, 8, 11, 12, 3, 9, 12, 10, 8, 7, 3, 5, 2, 2, 3, 2, 3, 4, 8, 13, 6, 8, 7, 4, 7, 13, 10, 9, 11, 10, 10, 5, 12, 4, 3, 7, NA, 2, 2, 1, 6, 2, 8, 8, 9, 6, 11, 7, 7, 9, 11, 9, 6, 8, 11, 10, 6, 5, 3, 5, 5, 1, 1, 2, 2, 5, 7, 13, 11, 7, 17, 10, 4, 10, 9, 10, 6, 5, 8, 4, 6, 6, 3, 5, NA, 5, 1, 3, 1, 5, 3, 8, 11, 6, 8, 9, 7, 11, 5, 12, 10, 10, 8, 9, 8, 7, 3, 2, 4, 5, 1, 3, 1, 2, 9, 10, 4, 7, 10, 11, 6, 11, 8, 8, 7, 10, 9, 9, 11, 9, 2, 3, 4, 3, NA, 2, 5, 5, 8, 13, 7, 14, 11, 4, 7, 10, 15, 10, 15, 8, 6, 9, 5, 4, 2, 1, 3, NA, 1, 1, 2, 3, 13, 6, 8, 6, 12, 10, 13, 5, 14, 11, 14, 14, 10, 10, 5, 3, 4, 2, 4, 2, 3, 2, 2, NA, 10, 9, 6, 14, 10, 7, 7, 12, 3, 13, 12, 12, 14, 8, 11, 3, 6, NA, 2, NA, 3, 2, 5, 5, 11, 3, 5, 7, 7, 12, 8, 5, 11, 5, 2, 8, 9, 6, 7, 3, 3, 1, 1, NA, 1, 3, 1, 3, 5, 10, 10, 7, 3, 5, 8, 11, 9, 11, 7, 9, 9, 10, 5, 10, 4, 3, 3, 1, 2, NA, 3, 4, 6, 6, 7, 11, 9, 11, 9, 6, 6, 8, 7, 9, 12, 12, 9, 5, 4, 3, NA, 1, 2, 1, 2, 2, 2, 6, 9, 8, 8, 8, 12, 8, 13, 7, 12, 7, 8, 12, 8, 5, 5, 1, NA, 4, 3, 1, NA, 5, 6, 10, 7, 15, 12, 12, 6, 11, 12, 12, 10, 16, 10, 8, 11, 8, 5, 4, 2, 1, 2, NA, 5, 2, 8, 9, 7, 9, 14, 7, 13, 3, 6, 9, 13, 10, 8, 6, 4, 6, 4, 5, 1, 3, 3, 2, 1, 4, 4, 9, 11, 4, 8, 9, 11, 8, 8, 9, 19, 9, 7, 7, 11, 6, 8)

I was about to suggest looking at earlier answers on "multiple seasonalities" until I saw that you want "to combine the predictors for the hourly weekly pattern with other predictors". Could you give a few details on what you have in mind here? — Stephan Kolassa, Apr 08 '16 at 06:24
@RobHyndman Hi Professor Hyndman, thank you for looking at this, and for your online textbook, and blog. I've been teaching myself time-series and forecasting from your textbook, blog, and forum posts. I'm in the US, I would love a chance to take an online forecasting class from you. Do you have any online courses available or planned in the future? I've added more sample data, along with code examples of how I'm trying to combine the dummy variables with other predictors. — ndderwerdo, Apr 08 '16 at 17:09
@StephanKolassa Hi Stephan, thank you for looking at this. I've edited the post and added examples of how I'm trying to combine the dummy variables with other predictors. I found the SO post below which has an example using a tbats model in an auto.arima model with predictors. I was wondering if that might work for my case. I've created a tbats forecast and compared it to the forecast I get using seasonaldummy, their very similar. Post: http://stats.stackexchange.com/questions/176129/r-time-series-forecasting-having-issues-selecting-fourier-pairs-for-arima-with — ndderwerdo, Apr 08 '16 at 17:34
@StephanKolassa Example from SO post: y <- msts(ts, c(7,365)) # multiseasonal ts fit <- auto.arima(y, seasonal=F, xreg=fourier(y, K=c(3,30))) fit_f <- forecast(fit, xreg= fourierf(y, K=c(3,30), 180), 180) — ndderwerdo, Apr 08 '16 at 17:35
@RobHyndman Hi Professor Hyndman, I have a lot of questions I would like to ask you related to dynamic regression and forecasting. I have 8 of them listed in the comments below. Please let me know if they need to be moved to separate forum posts. — ndderwerdo, Apr 08 '16 at 18:39
@RobHyndman 1) How much data should you train an Arima model on to forecast 20 hours?

How much data should you train an Arima model on if it has a weekly and hourly pattern, to forecast 20 hours?

Do you know a process for finding good predictors to use in the xreg argument? So far I've been adding predictors, then looking at AICc, ACF plots of residuals, ACCURACY measures (ME, RMSE, MAPE..), and comparing the forecast plot to the actual data. I was wondering what kind of exploratory steps I could do to find predictors to add to the model. — ndderwerdo, Apr 08 '16 at 18:40
@RobHyndman 4) Sometimes I'm finding that the AICc is higher for models with predictors where when I create a forecast with the model all the ACCURACY Error measures are lower. These are simple models with only one predictor. Is AICc the best way to determine if a predictor is good in a model, are there other techniques I should try?

If you are manually fitting an Arima model how do you select models when both ACF and PACF are sinusoidal? — ndderwerdo, Apr 08 '16 at 18:40
@RobHyndman 6) I think this data has level shifts, but I have been unable to detect them using tso. I usually get an error message, or a warning with no data. Do you think this data has level shifts and can you suggest how to detect and address them?

The post below suggests that the Ljung-Box test shouldn't be used to check auto correlation in Arima models. Do you agree? Post: http://stats.stackexchange.com/questions/148004/testing-for-autocorrelation-ljung-box-versus-breusch-godfrey — ndderwerdo, Apr 08 '16 at 18:41
@RobHyndman 8) In dynamic regression where there is an xreg predictor when you look at plots of the residuals and Acf plots of the residuals, are we only trying to check for auto correlation or are we also checking to see if there is any pattern in the residuals not explained by the xreg predictor? — ndderwerdo, Apr 08 '16 at 18:41
@RobHyndman here is the code looking for level shifts and the error I'm getting: library("tsoutliers")
test<-ts(tsData, frequency=168)

product.outlier<-tso(test,types=c("AO","LS","TC"))

Error in filter_(.data, .dots = lazyeval::lazy_dots(...)) : argument ".data" is missing, with no default In addition: Warning message: In auto.arima(x = c(11, 14, 17, 5, 5, 5.5, 8, NA, 5.5, 6.5, 8.5, : Unable to fit final model using maximum likelihood. AIC value approximated — ndderwerdo, Apr 08 '16 at 18:45
Happy to answer your questions at $500 per hour. Seriously, this is a Q&A site, not a place to get free consulting. — Rob Hyndman, Apr 09 '16 at 06:18
@RobHyndman Ha, guess I got a little excited to run across you on the interweb. Anyways, I did pay the $16 for your book which is excellent and would love to take an online course from you if one is going to be available. Do you have any advice on using fourier to achieve to pick up the hourly weekly pattern like I have with seasonaldummy? — ndderwerdo, Apr 09 '16 at 19:25

Mimicking seasonaldummy with fourier in Arima model

0 Answers0