3

I have a time series that is affected by two (or more) kinds of events. When event $A$ happens, some signal is linearly added to the time series (the signal lasts, for example, for 100 time points). When event $B$ happens, another signal (also of 100 time points length) is linearly added to the time series. The noise is not completely normal - there are some very strong outliers here and there. My goal is to model the two signals.

If the responses to the two events didn't overlap in time, I could simply average all the time points delayed t time points after each event kind. The standard deviation of the values in this subset of time points divided by $\sqrt{n}$ (n is the number of event occurrences) performs well enough in estimating the error: if in one of event $A$'s occurrences there was a strong outlier at time $x$, it will increase the local estimate of error for that particular delay.

My problem is that the two events do overlap in time. Therefor, I have to use regression in order to deconvolve the two responses. This is done by creating a set of pulse predictors of 100 delays from the occurrences of each event type (a Finite Impulse Response model). By OLS I can get a good estimation of the shapes of the two signals. However, I can't get a good estimate of the error: the standard OLS error estimator is assuming homoscedastic error. Hence, a time point at time $t_1$ that was distorted by a strong outlier has the same error estimation as time point $t_2$, which wasn't noised so much. This makes the error estimator useless in telling a true increase in the signal from a momentary noise.

As a crude approximation, I thought about estimating the error for each delay $t$ (and each event kind) by the standard deviation of the particular residuals at that specific delay, divided by $\sqrt{n}$ (n is the number of event occurrences, not total time points as in the usual OLS SE).

I'd appreciate any help with this problem. And in particular, answering these two questions:

  1. Is the latter solution biased? And if it is, in what way?
  2. Is there a better way to estimate the noise in this case? Due to the limitations of my scientific field, it should not involve rich parametric assumptions.
Andy
  • 19,098
Trisoloriansunscreen
  • 1,854
  • 15
  • 27

2 Answers2

4

A standard solution in econometrics for these problems are Newey-West standard errors (Wikipedia, software manual with formulae, random lecture notes with additional explanations). If $e_t$ is the residual at time $t$, and $x_t$ is the vector of regressors, then the general form of this variance estimator is $$ v[\hat\beta] = \Bigl[ \sum_t x_t x_t'\Bigr]^{-1} \Bigl[ \sum_{l=0}^L g(l) \sum_t e_t e_{t-l} (x_t x_{t-l}' + x_{t-l} x_t' ) \Bigr] \Bigl[ \sum_t x_t x_t'\Bigr]^{-1} $$ where the kernel function $g(l)$ declines monotonically from $g(0)=1$ to $g(l)=0, l>L$. You need it to make it more likely that the estimator is positive definite. A common kernel is Bartlett's $g(l) = 1 - l/(L+1)$. Newey-West standard errors take care of (conditional) heteroskedasticity, as well, albeit not in the most efficient way -- but it will be covered, too.

StasK
  • 31,547
  • 2
  • 92
  • 179
0

This is easy to treat with least squares. You just want to weight each point by the inverse variance if you know that or can estimate it. Most regression packages allow you to input either the weights or variance.

Dave31415
  • 1,143
  • I'm not sure that I follow. I don't want to weight data points according to their variability; I'd like to model the variability of the beta values. – Trisoloriansunscreen Mar 03 '14 at 09:35
  • Do you know the shape of the signals and trying to fit for two amplitudes AND two points in time where they turn on? If so, the fitting for the turn-on times is not solvable with OLS. If you know the turn-on times and just want to fit for the two amplitudes, it is indeed a OLS-type problem. But if each signal comes with it's own source of noise (that isn't just Poisson noise), it does become a little bit harder especially when you don't know the noise before hand. One way would be to use bootstrap resampling techniques for errors. That doesn't require that you specify the variances up front. – Dave31415 Mar 03 '14 at 14:35
  • Dave, the turn-on times are given, but the shapes are unknown. I don't believe that each signal has its own source of noise, but I don't want assume homoscedastic noise since I'd like an SNR measure for each delay. For a simpler example, consider a point by point t-test between time courses of two conditions (with multiple trials/ realizations). The error estimate in this case could be computed globally. However, an error estimate computed per delay gives information about the SNR in each point. I'd like to do the same with regression. – Trisoloriansunscreen Mar 04 '14 at 08:12
  • bootstrapping requires assuming that the errors are IID right? Maybe I can bootstrap chunks of observations (trials). – Trisoloriansunscreen Mar 04 '14 at 08:14
  • Bootstrap does not require any assumption about the distribution of errors. It uses the data itself to figure this out. – Dave31415 Mar 04 '14 at 13:11
  • see this question – Trisoloriansunscreen Mar 05 '14 at 06:36