Comparison of time series sets

Question

I have three sets of time-series data I am looking to compare. They have been taken on 3 separate periods of about 12 days. They are the average, maximum and minimum of head counts taken in a college library during finals weeks. I have had to do mean, max and min because the hourly head counts were not continuous (see Regular data gaps in a time series).

Now the data set looks like this. There is one data point (average, max or min) per evening, for 12 evenings. There are 3 semesters the data was taken for, in only the 12-day periods of concern. So for example, Spring 2010, Fall 2010, and May 2011 each have a set of the 12 points. Here's an example chart:

enter image description here

I have overlaid the semesters because I want to see how the patterns change from semester to semester. However, as I have been told in the linked thread, it's not a good idea to slap the semesters tail-to-head since there is no data in between.

The question is then: What mathematical technique can I use to compare the pattern of attendance for each semester? Is there anything special to time-series that I must do, or can I simply take the percent differences? My goal is to say that library usage over these days is going up or down; I am just not sure what technique(s) I should use to show it.

score 8 · Accepted Answer · edited Jul 12 '11 at 09:54

Fixed-effects ANOVA (or its linear regression equivalent) provides a powerful family of methods to analyze these data. To illustrate, here is a dataset consistent with the plots of mean HC per evening (one plot per color):

       |              Color
   Day |         B          G          R |     Total
-------+---------------------------------+----------
     1 |       117        176         91 |       384 
     2 |       208        193        156 |       557 
     3 |       287        218        257 |       762 
     4 |       256        267        271 |       794 
     5 |       169        143        163 |       475 
     6 |       166        163        163 |       492 
     7 |       237        214        279 |       730 
     8 |       588        455        457 |     1,500 
     9 |       443        428        397 |     1,268 
    10 |       464        408        441 |     1,313 
    11 |       470        473        464 |     1,407 
    12 |       171        185        196 |       552 
-------+---------------------------------+----------
 Total |     3,576      3,323      3,335 |    10,234

ANOVA of count against day and color produces this table:

                       Number of obs =      36     R-squared     =  0.9656
                       Root MSE      =  31.301     Adj R-squared =  0.9454

              Source |  Partial SS    df       MS           F     Prob > F
          -----------+----------------------------------------------------
               Model |  605936.611    13  46610.5085      47.57     0.0000
                     |
                 day |  602541.222    11  54776.4747      55.91     0.0000
           colorcode |  3395.38889     2  1697.69444       1.73     0.2001
                     |
            Residual |  21554.6111    22  979.755051   
          -----------+----------------------------------------------------
               Total |  627491.222    35  17928.3206

The model p-value of 0.0000 shows the fit is highly significant. The day p-value of 0.0000 is also highly significant: you can detect day to day changes. However, the color (semester) p-value of 0.2001 should not be considered significant: you cannot detect a systematic difference among the three semesters, even after controlling for day to day variation.

Tukey's HSD ("honest significant difference") test identifies the following significant changes (among others) in day-to-day means (regardless of semester) at the 0.05 level:

1 increases to 2, 3
3 and 4 decrease to 5
5, 6, and 7 increase to 8,9,10,11
8, 9, 10, and 11 decrease to 12.

This confirms what the eye can see in the graphs.

Because the graphs jump around quite a bit, there's no way to detect day-to-day correlations (serial correlation), which is the whole point of time series analysis. In other words, don't bother with time series techniques: there's not enough data here for them to provide any greater insight.

One should always wonder how much to believe the results of any statistical analysis. Various diagnostics for heteroscedasticity (such as the Breusch-Pagan test) don't show anything untoward. The residuals don't look very normal--they clump into some groups--so all the p-values have to be taken with a grain of salt. Nevertheless, they appear to provide reasonable guidance and help quantify the sense of the data we can get from looking at the graphs.

You can carry out a parallel analysis on the daily minima or on the daily maxima. Make sure to start with a similar plot as a guide and to check the statistical output.

+1, for demonstration of simple yet powerful techniques. I am most curious though how did you manage to extract the values from the graph? Some software, or a punishment for badly behaving student? :) — mpiktas, Jul 12 '11 at 09:55
@mp I digitized points on top of a screenshot of the graphic, extracted their coordinates with GIS software, transformed the coordinates with a spreadsheet, then imported it into a stats package. It takes only a few minutes. This method can be handy when the only data you have are in the form of a chart or map. — whuber, Jul 12 '11 at 12:50
@whuber I wonder what the effect is of having 3 sets of 12 autocorrelated readings as compared to 36 independent observations. I would think that we really don't have 35 degrees of freedom to parcel out. The probabilities you reflect on are premised on the ratio of a non-central chi-square variable to a central chi-square variable. Is there something I am missing here? Nice work to extract the numbers from the plot. Is there a particular program that you can reference in order to aid us in this regard. — IrishStat, Jul 12 '11 at 21:51
@Irish There's little or no evidence of autocorrelation in these data (in part because there aren't enough of them). However, the presence of positive autocorrelation (the most likely kind) would not change the legitimacy of this analysis nor its interpretation; it would merely strengthen the conclusions. Autocorrelation (in a case of limited data like this) is really a modeling decision: do we want to estimate and compare the day-to-day changes or do we want to characterize the second-order moments of some underlying random variable? The former seems more useful and parsimonious. — whuber, Jul 12 '11 at 21:59
@whuber: If a series is positively autocorrelated one underestimates the variance as values can be closer to the mean than they should be. Conversely if you have negative autocorrelation than values can arise disproportionately far from the mean thus one overstates the variance. At least this is what my "experience and education" have taught me. What say you ? I believe that this is treated formally in some stat/math books but I can't put my finger on it. Do have a reference to support your statement. — IrishStat, Jul 12 '11 at 22:16
@Irish Let $x$ be the mean (among color) for one day and $y$ the mean for another. I assume homoscedasticity; that is, $Var(x)=Var(y)=\sigma^2$ ($\sigma$ unknown). The desired comparison ("usage ... going up or down") tests whether $x-y=0$. Without correlation, $Var(x-y)=2\sigma^2$. With correlation $\rho$ between $x$ and $y$, $Var(x-y)=2(1-\rho)\sigma^2$. When $\rho \gt 0$, the variance actually is less than assumed in the ANOVA model. Consequently t-statistics, F-statistics, and the Tukey HSD all should be more significant than they appear. — whuber, Jul 12 '11 at 22:29

score 0 · Answer 2 · answered Jul 11 '11 at 17:56

0

Sarah, Take your 36 numbers (12 values per cycle ; 3 cycles ) and construct a regression model with 11 indicators reflecting possible week-of-the-semester effect and then identify any necessary Intervention Series ( Pulses, Level Shifts ) necessary to render the mean of the residuals to be 0.0 everywhere or at least not statistically significantly different from 0.0. For example if you identify a level shift at period 13 this might be suggest a statistically significant difference between the mean of the first semester i.e. the first 12 values) versus the mean of the last two semesters (last 24 values ). You might be able to draw inference or test the hypothesis of no week of the semester effect. A good time series package might be useful to you in this regard. Failing that you might need to find somebody to provide help in this analytical arena.

answered Jul 11 '11 at 17:56

IrishStat

29,661

1

This sounds like a description of two-way ANOVA (days by cycles) followed by planned tests of 11 pairs of days. Plain old stats software is likely going to be more flexible and powerful to use than specialized time series software; it will certainly be easier. BTW, the indexes are days (into the exam period), not week of the semester. – whuber Jul 11 '11 at 18:13
Can I also use the ANOVA to compare maximums and minimums per day? Or does this only apply to the means? – induvidyul Jul 11 '11 at 20:52
@Sarah It might be applicable to the the minima and maxima. However, those statistics tend to be much more variable than the means, so it is less likely you will be able to detect changes in them over time or between semesters. You graph makes it clear that the means do differ significantly. If you can, make the ANOVA three-way by incorporating the hour of the day and using the original hourly counts rather than their daily means. – whuber Jul 11 '11 at 22:37
@whuber: I have been told that stringing together the hourly data is not usable, since they have only been recorded for 12am to 6am. See my previous question Regular data gaps in a time series. – induvidyul Jul 11 '11 at 22:42
@Sarah I'm talking about something different: model the dependency in terms of three factors: period (3 of them), day into the period (12 of them), and hour of the day (6 of them). You could even account for correlations among the hours, but that might not be necessary for your purposes. Regardless, I am not advocating viewing each period as an interrupted series of 12*24 counts: there are too many missing data. – whuber Jul 11 '11 at 23:24
@whuber Forming it as a time series enables "hypothesis generation" which might lead to a detection of a significant change in the three conditional means.If no level shift is detected one might conclude that there is no statistically significant difference between the 3 semesters.If a level shift is detected at day 13 this would suggest that a significant change in intercept suggesting that the last 24 values had a mean that was significantly different from the 12 days given the day effect.Unusual values could also be detected providing a more robust estimation of the day / semester effects. – IrishStat Jul 12 '11 at 00:46
@Irish Most time series algorithms and software are not going to work on 36 short interrupted segments of a series. ANOVA--optionally with accommodation of some kind of specified correlation structure within the segments (that is, between hours)--will be able to handle everything you mention and more, using familiar techniques and no specialized software or terminology. – whuber Jul 12 '11 at 00:57
@whuber Agreed ! Most time series algorithms and software are not going to work on 36 short interrupted segments of a series, but not all ! – IrishStat Jul 12 '11 at 15:11
@Irish I figured you might have some special capabilities up your sleeve, so I cautiously wrote "most" :-). – whuber Jul 12 '11 at 15:14

Comparison of time series sets

2 Answers2

Linked