6

In the hidden Markov model we use two matrices. The first one, called the transition matrix, determines probabilities of transitions from one hidden state to another one (the next one). The second matrix, called the emission matrix, determines probabilities of observations given a hidden state.

In other words, we can imagine a system as being in a state (which is hidden, unobservable) and this hidden state determines the probability of the next hidden state as well as probability of a given observation.

It means that we assume that a "jump" (or transition) to the next hidden state and "generation" of a certain observation are independent events. Is this a limitation of the model? Or, in case of such a dependency, is there a way to reformulate the model (for example, by redefining states) such that these two "steps" (transitions and emissions) are independent?

Alexis
  • 29,850
Roman
  • 584

2 Answers2

6

Yes, it's a limitation of the model.

In most dynamical systems that you might care to think about using an HMM, the act of measurement or observation is usually conceived and/or constructed in such a way that it is assumed not to affect the system under observation. We want to imagine the observer as being passive and completely independent from the system, and then make inferences about how the hidden states would evolve, even if the observer were not present.

An example might help to make this a bit more concrete. Imagine that you are tracking a satellite moving through the sky with a telescope. The "hidden states" in this case would be the position and velocity of the satellite at any given moment in time. The "transition process" is governed by whatever forces are acting on the satellite; certainly gravity at all times, but also possibly other things such as the thrusters in a reaction control system. The "observation" at any given moment consists of two angles, the altitude and azimuth with respect to the horizon. Now ask yourself, how weird would it be if the very act of observing the satellite through a telescope, hundreds or even thousands of miles away, could actually cause it to speed up or slow down (thereby impacting its hidden state) spontaneously?

Lots of dynamical estimate problems are like the satellite and telescope problem: the act of observation itself doesn't affect the hidden states very much, or at all. HMMs, or similar/related concepts, are a perfectly valid and useful way of thinking about these kinds of systems. The independence of the observation and the state transition are a desirable feature of the model.

It's worth noting that there are systems where the act of observation is presumed to affect the thing under observation. This kind of interdependence is actually an integral feature of quantum mechanics. The term of art in quantum mechanics for the moment when an observation actually affects the thing under observation is wave function collapse. To model a quantum mechanical dynamical system in a way that would be analogous to how we use HMMs, you would indeed need to assume that the observation and the state transition are coupled events. It's worth noting that if you google quantum filtering or quantum Markov filter, there are several hits, indicating that it does appear to be an active topic of current research.

stachyra
  • 3,102
  • 15
  • 15
  • does a dependency between observations and transitions to the next state assume that observation affect the hidden state? I am not sure about that. For example we can think of a system that is in a particular state and than system randomly "decide" how to act further, and this "decision" influence the next transition and the observation. So, transition to the next state and observation are kind of bound. – Roman Aug 25 '15 at 09:09
  • I'm not sure I understand what you are asking. In my response, I gave an example involving satellite tracking, which shows how abstract concepts such as "hidden state" and "observation" might be applied to an actual system in real life. Can you give me an example of a real life dynamical system, involving the concept of both hidden states as well as observations, where the state transition and observation are potentially "bound" or "dependent" in the sense that you are trying to ask about here? It might help me to better answer your question if we were speaking less abstractly... – stachyra Aug 25 '15 at 22:55
  • here is a concrete example. Stocks of a perishable item is given by its distribution over the expiration days (for example we have 3 items that will be expired in 2 days, if not sold, and 2 items that will be expired in 1 day if not sold). After one day we jump to another state and observer write-offs (items that expire). The next state (distribution of stocks) and observations (write-offs) are bound (not independent). At this is not the case in which our observation influence the transitions. – Roman Aug 27 '15 at 10:00
  • O.K., I think I understand now. Your example is different from a typical HMM problem in two ways: first, the state isn't hidden, the store owner can read expiration dates and count the items with perfect precision. Second, the system isn't autonomous; rather it is "under control": the store owner decides how many new items to order based upon recent expirations, therefore the observation affects the next state transition. For this situation, I'd suggest a Markov decision process. Use this handy table to decide when to use which model. – stachyra Aug 28 '15 at 03:17
1

In wikipedia article it is stated that next hidden state $x(t+1)$ conditionally depends from previous hidden state $x(t)$:

enter image description here

But we can't say that observation $y(t)$ and next hidden state $x(t+1)$ are independant. Because if you observe $y(t)$ then you have some knowledge about $x(t)$ probability distribution and then knowledge about $x(t+1)$ distribution.

For example, if you have that $y(t)$ just equals hidden state: $y(t) = x(t)$ then knowing $y(t)$ value gives you $x(t)$ value. And then you can derive $x(t+1)$ distribution from $x(t)$ value.

But you can say that $y(t)$ are independant with $x(t+1)$ given $x(t)$. If $x(t)$ is fixed (you don't know the exact value, you just know it is fixed) then $y(t)$ observation gives you nothing about $x(t+1)$.

hvedrung
  • 270
  • @vedrung, when I wrote about "independence" I have understood it in the same way as you described. Given a hidden state, the next hidden state and the observation are independent. So, I completely agree with your interpretation. However, my questions remains: should this independence be considered as a limitation of HMM? What if we have a system in which for a given hidden state the probability of the next hidden state and probability of observations are dependent. Can we still use a HMM? – Roman Aug 19 '15 at 13:52
  • No. HMM construction implies that $x(t+1)$ is independent from $y(t)$ (and other preceding states) if $x(t)$ is given. You have to add an "arrow" from $y(t)$ to $x(t+1)$ if you want implement the desired property i.e. $x(t+1)$ has to have conditional dependence from $x(t)$ AND $y(t)$. And this is not HMM model but it modification. – hvedrung Aug 19 '15 at 14:24