Forecast customer's spending

Question

I am forecasting the spending of customer based on the time series data set. I tried to build to models:

• Arima
• regression. ´

With Arima model, I was able to forecast quite accurately the spending of customers on normal day. However, in both my data set and present reality, on every first day of month, due to monthly promotion, the customers' spending is always significantly higher than that on other days. Hence, with Arima models, at the moment I have not been able to forecast the customers' spending on first day of month accurately. With regression model, the predicted number for the customers' spending on first day of month was more accurate than that predicted with Arima model.

However, with an $R^2$ of only 50%, the fluctuation of predicted values for others days is very high. Can you please advise me methods that I can use to improve my models or is there any other models that can help me to predict the spending accurately on both normal days and first day of month? Here is a small part of my data

Answer 1

Combining ARIMA and dummy indicators is a very good direction. Empirically Finding which days of the month are important is an important feature of software . See the daily demand study in http://autobox.com/cms/index.php/afs-university/intro-to-forecasting/doc_download/53-capabilities-presentation and suumarized here . One needs to robustly integrate these features along with any user-specified predictor series.

If you wish to post your data I will try and help further and I will introduce your data to AUTOBOX (my tool of choice which I helped to develop).

It is naive to try and separate particular days from normal days as one needs to seamlessly incorporate daily/monthly effects , level hist effects, memory effects. holiday effects without being blindsided/affected by anomalies not representing "typical behaviour" thus needing to be cleansed of the anomalous portion/effect.

EDITTED after receipt of data which include two user-suggested "helping variables" viz day1 of the month and a collective holiday indicator. The OP couldn't post his data to SE so he emailed it to me . I would be glad to forward it and any of my analytical results to any interested reader . He sent two typical time series .. One for revenue for prepaid and another for postpaid (1061 consecutive days) . The postpaid series is here resulting in a model that provide the following actual/fit and forecast with 90 day forecasts here . The equation is here in two parts. and . There is a very strong ar(7) structure suggesting a daily memory component along with two monthly indicators.

The prepaid series was a little more interesting . Here it is with forecasts here . The useful model that AUTOBOX automatically provided is here in two parts and here . The stat summary is here . The residuals from the model are here with acf here suggesting model sufficiency. The cleansed plot is interesting . Finally the Actual/Fit and Forecast plot . Note that a number of days in the month were found/discovered to be significant in addition to the user suggested variable day1.

Christmas/Year End is certainly important for both series while the prepaid series is very seasonal (monthly factors within the year) the postpaid series is also very "seasonal" with respect to days within the week.

Most ( if not all !) daily data dealing with human activity has similar characteristics as we are creatures of habit. Daily stock market price data ..... not so much as it is driven in a large part by emotion which is not as quantifiable !

Both models combined memory (arima) and deterministic structure . Note that forecasts foe future holidays were not included in the prepaid study as they were not delivered thus biasing the forecasts to the down side when holiday activity commenced.

One other comment in order to determine if individual holidays are important ( as they usually are) as no one coefficient can explain lead,contempraneous and lag structure around each holiday event , AUTOBOX needs to know the list of holidays (via resident data files) and when they have occurred and will occur in the planning horizon. This data came from Vietnam but we used a generic European/US calendar for this preliminary but highly informational study.

The fact that St. Patrick's day is important for Vietnam assuredly reflects an unspecifed holiday (coincident holiday) arriving around or on the 17th of March.