2

Assume $X$ and $N$ are two sets of observations from two different normal distribution, where $X$ represents clean data and $N$ represents noise; and $A$ a projection matrix of a filter and the relation between $X$ and $Y$ defines as:

$Y=A \times X + N$

considering that $X_i$ are not sequential samples and are just different observations from a normal distribution (there is no time involved), are we allowed to use a Kalman filter to estimate $A$ and parameters of $N$?

PickleRick
  • 195
  • 6

1 Answers1

1

My example in your last post, which solves this problem using a Kalman Filter, shows you can use non-time dependent observations, in any order, to estimate $A$ and $N$. It's called using Kalman Filters for parameter estimation.

spektr
  • 4,238
  • 1
  • 18
  • 19
  • thank's. just wanted to be sure it's theoretically possible. Do you have a reference to add to your answer, please? – PickleRick Jan 25 '16 at 17:05
  • 1
    @Anoosh Why do you question whether it's theoretically possible if I solved your problem with it? That should be experimental evidence it works. Additionally, I don't have a reference to the Kalman filter solving this exact problem. But given your problem is linear, you could look into convergence of Kalman filters for linear dynamical systems. – spektr Jan 26 '16 at 01:49
  • even though you provided a (pretty good) experimental proof, I need the theoretical proof as well. Because in all the references I looked, samples are assumed to be from a time sequence, not just samples from a distribution. would worth it if one can come up with a proof. so far, I wasn't really successful in finding that proof, but keep looking. That's the reason I asked for the code in the email I sent you. To study how you used it exactly and be able to find the reason it is actually working. in addition, some people complained that Kalman filters can't be used in such situations. – PickleRick Jan 26 '16 at 07:46
  • 1
    @Anoosh While typical problems that use Kalman filters are time dependent, parameter estimation and regression using Kalman Filters doesn't necessarily have data that is time dependent. The derivation of the Kalman Filter, as I remember, doesn't use the fact observations can be time dependent. All you should think about is the observations being independent of each other, which they would be in your data set. This means typical Kalman filter proofs of convergence should apply. Additionally, I will try to get my code posted in GitHub later for you and others to look at. – spektr Jan 26 '16 at 14:43
  • thank you @choward look forward to your implementation! Even though what you are saying makes sense to me, I would keep this question for a while, maybe someone comes with an explanation. (according to the definition, Kalman Filter is an algorithm that uses a series of measurements observed over time)

    I've got a lot of vote downs on other sites for trying to use Kalman filters for a problem that is not time-dependent! But it's ok, I look for answers, not reputation!

     – PickleRick Jan 26 '16 at 15:53
  • @Anoosh I posted a link in my original answer to the github. http://scicomp.stackexchange.com/questions/21892/learning-parameters-of-noise-and-filter-coefficients-from-data-where-data-and-no – spektr Jan 27 '16 at 04:03
  • thank you for the time you spent to prepare such a nice answer. – PickleRick Jan 27 '16 at 07:29
  • @choward - if Anoosh provided sufficient content to answer the question, shouldn't it be turned into the question instead of a series of comments? – EngrStudent Feb 22 '16 at 15:59