Mark Location of Trend Time Series

Question

I am new in the topic of time series data and found out about the STL decomposition which can smoothly show the trend in some time series data. My task is to detect trends given certain data.

At first, I went the complicated way and labelled data, where trends were and trained classification models on it and tested it with new data. Unfortunately this did not perform well. Now after reading more about time series data, I thought it might be a better solution to treat it as a time series problem rather than a classification problem.

Take this STL decomposition as an example:

Is there a way to 'detect' the location of the trend? I mean to put points in the plot of the trend around x=30 until x=52 (upgoing trend) or around x=45 until x=80 (down-going trend)? (Not manually & hardcoded obviously, but with an algorithm that detect a trend)

My data are measurement points of laptop components that are retrieved after each other (with a timestamp) but not necessary periodically with the same time interval. I hope this isn't a problem for this.

My data has this look:

0       0.2362
1       0.3939
2       0.3062
3      -0.2643
...
232    -0.3146
233    -0.1514
234    -0.1881
235    -0.0965

Answer 1

Your problem is more along the line of time series segmentation and changepoint detection. If you care for a readily available solution in R (or Matlab), there are numerous alternatives possible (e.g., strucchange, and changepoint), as summarized in the R CRAN task view on time series: https://cran.r-project.org/web/views/TimeSeries.html. If you want to implement your own, I find the easiest reference as a starting point is Keogh, Eamonn, Selina Chu, David Hart, and Michael Pazzani. "Segmenting time series: A survey and novel approach." (https://www.ics.uci.edu/~pazzani/Publications/survey.pdf). Apparently, assumptions of all sorts are made in different methods. As a practitioner, I think no single method is the best all the time: try and let the results tell which suits your purpose.

Allow me to use one package Rbeast written by me and available in R and Matlab ( more info at https://github.com/zhaokg/Rbeast) to illustrate the basic idea. It accepts regular or irregular data with or without a periodic component. It seeks to decompose a time series into trend and seasonality and simultaneously detect changepoints in the trend and seasonal components.

library(Rbeast)
Nile:           an annual riverflow time series of the River Nile
season='none':  no periodic/seasonal component so no time series decomposition is performed and only changepoint detection is done
o = beast(Nile, season='none')
plot(o)

The time series is divided into two segments, with one changepoint. The time-varying probability of changepoint occurrence in the trend component is shown in the second subplot Pr(tcp). The sgnSlp subplot gives the probability of the slope being positive, zero, or negative over time.

Here is another example using the covid19 daily time series of new cases, with a periodic component of 7 days.

library(Rbeast)
data(covid19)
newcases = covid19$newcases
datenum  = as.numeric(covid19$date)
o       = beast(newcases, start=min(datenum), deltat=1, freq=7)
o$time  = as.Date(o$time, origin='1970-01-01') # Convert from integers to Date.
plot(o)

Here is the output. In this case, the time series was decomposed into the seasona/periodic component and the trend component. The locations of changepoints in the two components are indicated by the dash lines in the season and trend subplots.

Answer 2

Easiest thing for 'plug and play' is to use Meta's Prophet since it fits for these trend changepoints. An example output of the trend from the example in the link: Then just look at the change in the gradients for the trend component in the output dataframe. There are definitely better methods that perform better on benchmarks but it will get you started. If it irregularly sampled just pretend that it is and disable seasonality and reindex your time series to appear regular. Basically it will just be a trend changepoint algo at that point.

Alternatively you could try wild binary segmentation which involves exhaustively segmenting your data into 2 pieces and fitting a trend line to each then choosing the point where segmenting your data minimizes some criterion like mse, then repeating that process fitting on the residuals until a global criterion is minimized like AIC/AICC. If using python this is essentially what a package I wrote up uses if you use a 'linear' trend estimator with 'local' fit type: ThymeBoost.