Smoothing methods in data analysis, like splines or kernel smoothers, also regression smoothers like lowess.
See Wikipedia, which has a long list of various smoothing methods, and further references.
Questions tagged [smoothing]
394 questions
6
votes
1 answer
Smoothing dirty data?
I'm trying to do analysis on revenue data and due to circumstances beyond my pay grade, there are large spikes in the data around the time financial reports are due, followed by corresponding low values immediately after. Is there a preferred…
Jeff Sacksteder
- 163
4
votes
2 answers
Rank of smoothing matrix
The solution to the minimizing problem: $$RSS(f, \lambda) = \sum_{i=1}^n (y_i - f(x_i))^2 + \lambda \int (f''(t))^2 dt$$ is written in matrix form as $f = N\theta = \sum_{j = 1}^n N_j(x_i) \theta_j$ where the $N_j(x)$ are a $n$-dimensional set of…
Lejoon
- 175
- 6
3
votes
1 answer
How to filter these extremely bad data points?
I'm looking at data for my company, and basically we have some periods over the last year where the data was not uploaded correctly. In this figure, "mu" is the value of interest, and duration is a time in days. The huge spike near 180 days is an…
thecity2
- 1,955
3
votes
2 answers
How is this seven day rolling average calculated?
Eric Topol posted the below on Twitter claiming that Europe is "turning COVID around" based on the trend in the past <7 days. However, while the case trend hitherto appears smooth, I was curious if the precision of the trend in the tails is somewhat…
AdamO
- 62,637
3
votes
1 answer
Smoothing technique that takes into account subtotals
I have a simple task of fitting a smooth line with monthly points across six years. The line doesn't need to go through any specific point, but the yearly totals need to be specific values.
Is there a statistical technique that can draw such a…
Tom
- 1,771
- 2
- 13
- 18
3
votes
0 answers
How to smooth standard deviations?
So, I have some idea how to smooth a set of points to produce a smooth estimate of the conditional mean.
Suppose that I want to estimate the standard deviation locally. That is, I have a set of points (x,y) and in different regions of the x space,…
dash2
- 235
3
votes
1 answer
strange predicted values from LOESS model
I am trying to predict density function using LOESS in R. However, the predicted values I got are not in the estimated LOESS line.
#Generate data
n<-10000
a1<-a2<-0.1
a3<-a4<-0.2
a0<-0.1
u1 <-rnorm(n,0,1)
u2 <-runif(n,0,1)
u3 <-rbinom(n,1,0.5)…
user37180
- 73
2
votes
1 answer
Does this de-noising algorithm have a name?
I recently was asked to help characterize some surfaces (raised ink dots on paper) that were scanned with a profilometer, yielding height data across a grid of x & y coordinates. The critical question was to characterize the average dot height. The…
Mike Lawrence
- 13,793
1
vote
0 answers
Smoothing of a surface
A land surveyor made a topographic map of a property in the hills by measuring the elevation at different points. The x-y coordinates are more or less "random", meaning no regular grid:
Because the elevation measurements (z-coordinate) seem to be…
matt77
- 11
1
vote
0 answers
Computing a moving average when data points arrive one at a time
I am sure there's a cool python/numpy/pandas way to do this.
I am receiving one data point at a time. I would like to compute something like a moving average over the last n observations, even better with exponential smoothing.
I could have a…
pitosalas
- 963