Does dispersion really matter?

Question

Context:
Given a counting process $\{N(t),\,t\ge 0\}$ which tracks the number of events (arrivals) by time $t$, the index of dispersion (for counts) is the variance-to-mean ratio of the cumulative number of events (arrivals) by that point in time. More succinctly, $$I(t)=\frac{\text{Var}(N(t))}{\text{E}[N(t)]}.$$

Comparison to NHPP. In a stochastic process such as a nonhomogenous (nonstationary) Poisson Process (NHPP), $I(t) = 1\;\; \forall t$, since the mean and variance for $N(t)$ are both $$\text{E}[N(t)] = \text{Var}(N(t))=\int_0^t \lambda(s) ds\,,$$ where $\lambda(t)$ is the time-varying rate for $N(t)$.

Q: Does dispersion really matter? In what applications would it matter? How does it matter?

---

I've posted an answer based on my perspective. I'm interested in answers from other branches of OR.

Answer 1

Yes it matters -- the extent varies by several factors.

Applications & Impact
In many contexts using stochastic process-based models, one is well-served to use time-varying models to capture the time-dependent behavior of the system. And sometimes close enough, $I(t)\approx 1$ is good enough.

In healthcare, appointment-based systems often see underdispersion, $I(t) < 1$, which implies staffing models would over staff if an NHPP arrival process was used, even if it captured the deterministic time-dependent behavior. This happens because the model would overestimate the stochastic variability properly about that time-dependent fluctuation. The literature reports appointment-based dispersion levels of $(0.4-0.6)$.

Also in healthcare, emergency rooms often see overdispersion, $I(t) > 1$, as well as some call centers. In this setting, a staffing model would underestimate the requirements to meet a desired performance, or Quality of Service (QoS), target. The literature reports levels $(1.5-2.5)$ as being significant.

Quality of Service (QoS)
How much one should concern oneself with this also depends on the QoS target. If the QoS target is very high, then the level of control required to meet that may require accounting for the dispersion. In low QoS systems, this is less likely to cause problems.

Summary
1. Focus on the time-dependent nature of your arrival process. This is more important to get right than the dispersion.
2. Estimate your dispersion to understand the potential bias of your model. More complexity may not be required, even if the model isn't spot on.
3. If you're quality of service target is high (high performance), then you may want to consider dispersion in the arrival process.
4. Add in dispersion if necessary.

Answer 2

SecretAgentMan has given some specific examples of cases where over- and underdispersion will affect outcomes. I thought it might complement that answer to generalise those examples to some general principles about when over/under-dispersion is likely to happen.

A Poisson process typically represents a count of many low-probability events, each of which is probabilistically independent of each other event that could contribute to the count. A classic example is radioactive gamma-decay: every nucleus has X% chance of decaying in a given time period, but whether it decays has no effect on whether other nuclei decay in the same time period.

Overdispersion is likely to arise when events contributing to the count have positive correlation, i.e. one event occurring makes it more likely that others will occur. Reasons for this could include:

• Counted events can cause other counted events. (Example: a business bankruptcy may lead to flow-on bankruptcies, as other businesses lose a client and invoices go unpaid.)
• A single event not directly included in the count can cause several counted events. (Example: one traffic collision may cause several injuries, and one system outage might cause many helpdesk calls, helping explain the overdispersion that SecretAgentMan notes in emergency rooms and call centres.)

Underdispersion is likely to arise when events contributing to the count have negative correlation, e.g. because events suppress one another or are "spaced" by some mechanism. In the example of an appointment-based medical system, staff will attempt to spread appointments relatively evenly (subject to capacity by time); the more appointments that have already been booked in a given time slot, the less likely it is that other appointments will also be booked for that slot.