Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Gaussian mixture can refer to distributions or models which assumes subpopulations follow Gaussian distributions. A mixed Gaussian distribution $\mathcal{P}(x)$ can be generally written as a weighted sum of individual Gaussians $\mathcal{N}$:

$$\mathcal{P}(x) = \sum_{i=1}^n w_i \mathcal{N}(\mu_i,\sigma_i^2)$$

where $\sum_{i=1}^{n}w_i = 1$

GMMs are often used in unsupervised learning where we don't know from which subpopulation a data point belongs. In this case we seek to maximize the likelihood function (which assumes the data is independent):

$$p(X|w,\mu,\sigma) = \prod_{i=1}^{n} p(x_i|w,\mu,\sigma)$$

The parameters $\{w,\mu,\sigma^2\}$ can be solved using either Expectation-Maximization (EM) or MAP estimation.

It can also be used to simulate outliers and thereby test outlier detection and robust statistical methods

630 questions
20
votes
5 answers

Singularity issues in Gaussian mixture model

In chapter 9 of the book Pattern recognition and machine learning, there is this part about Gaussian mixture model: To be honest I don't really understand why this would create a singularity. Can anyone explain this to me? I'm sorry but I'm just…
Dang Manh Truong
  • 337
  • 1
  • 3
  • 10
6
votes
2 answers

How to tell if a mixture of Gaussians will be multimodal

Suppose I have a mixture of Gaussians and I know the mean and variance of each separate Gaussian. How can I tell whether or not the resulting distribution will be multimodal or, more specifically, that there will be be a number of modes equal to the…
mjnichol
  • 337
  • 1
  • 8
5
votes
1 answer

Introduction to Gaussian mixture models

First of all, I am sorry if this question is not acceptable by some of the readers. However, I really read many, many sources about Gaussian mixture models, but all what I found was a short tutorial or even strong mathematical book. I do not have a…
Maryam
  • 1,620
3
votes
2 answers

fitting a Gaussian mixture with a constraint in python

Suppose I have data and I want to fit a two component Gaussian mixture to it. I don't know how to do it in python but worse than that is that I have an additional constraint that the mean of one component should be less than zero and the mean of the…
mlofton
  • 2,417
3
votes
1 answer

Conditional mean for mixture of multivariate normal distributions

If x = (x_1,x_2,...,x_n) is a vector whose components have a distribution that is a finite mixture of multivariate normals, is the expected value of x_1 still a linear function of the other components, as is the case when the components have a…
3
votes
1 answer

GMM weighting change during marginalization

Suppose one has a multivariate Gaussian Mixture Model: $$ \text{pdf}(\vec{x}) = \sum_{i=1}^N w_i \mathcal{N}(\mu^{(i)}, \Sigma^{(i)}) $$ Suppose $\vec{x} = \{\vec{a},\vec{b}\}$ and we marginalize out $\vec{b}$. For a single Gaussian component we…
Mageek
  • 231
3
votes
1 answer

Estimate Gaussian (mixture) density from a set of weighted samples

I am trying to find if there exists a way to find the spatial distribution over a set of points where the points are weighted. If I have "n" points in the (x,y) space, I can fit a mixture of Gaussians to estimate a distribution. However, there might…
statBeginner
  • 1,581
2
votes
1 answer

Gaussian Mixture Model fails on a simple distribution with a fixed number of components

I'm trying to fit a 2-component 2D Gaussian Mixture Model to some data. I know that there are only two components. The distribution can be seen below in the left plot: The brain can effortlessly pick up that there is a smaller clustered…
Gabriel
  • 4,282
2
votes
1 answer

Is this equivalent to a Gaussian mixture model?

Similar to how you can derive the normal distribution as the distribution where the probability near a point exponentially decays with the square of the number of standard deviations you are away from the mean I was wondering if its possible to…
2
votes
1 answer

Useful separation value in a mixture distribution

Assume we have a distribution that is the mixture of two normal distributions. The pdf of the overall distributions and their single parts may look like the following. In black, the combined distribution and colors are their parts Sometimes it is…
halirutan
  • 145
1
vote
1 answer

Coefficients of Gaussian mixture

This is in context of Gaussian mixtures $$p(\boldsymbol{x}) = \sum_{k=1}^K \pi_k\cal{N}(\boldsymbol{x}|\boldsymbol{\mu_k},\boldsymbol{\Sigma_k})$$ Bishop mentions on Page-111 Also, the requirement that $p(\boldsymbol{x})\ge 0$, together with…
muser
  • 135
1
vote
2 answers

Derivation of gaussian mixture models assuming that hidden variable is known

I saw the following notes from CS229 (screenshotted below). I am confused how the two equations are equivalent. How were they able to distribute the $log$ inside the summation? I don't see how knowing the hidden variable $z$, allows us to write it…
David
  • 705
1
vote
0 answers

Determining number of components in mixtures of normal distributions with common mean

This is a pretty simple question, suppose we want to fit a mixture distribution of multivariate normals with common mean $$y_i \sim \sum_k \pi_k N(\mu, \Sigma_k)$$ What is the preferred approach for deciding the number of components in this…
Xiaomi
  • 2,544
1
vote
1 answer

Decompose/split a single multivariate gauss into random gaussian mixture

Say, there is a single $n$-dimensional multivariate Gaussian. $$Gauss_a(\mu_a,\Sigma_a) $$ $\mu_a$ is $1\times n$ vector and $\Sigma_a$ is $n\times n$ matrix. Is there any easy way to decompose/split a single gaussian $Gauss_a$ into random…
JimSD
  • 413
1
vote
0 answers

Reducing the number of Gaussians in a Gaussian Mixture Model

I build a kernel density estimation (KDE) of Gaussian kernels. I have many samples, but the distribution is not too complicated. I think it should be possible to approximate the resulting KDE by a much smaller number of Gaussian functions. This for…
Make42
  • 565
1
2