5

I'm curious if there are any common parametric distribution models for mixed discrete/continuous data. For illustration, suppose I have two random vectors, $X_c,X_d$, where $X_c$ is continuous and $X_d$ is discrete. I have data consisting of samples of $(X_c,X_d)$, and I'd like to do some density estimation. Ultimately the distribution of $X_c\vert X_d$ is what I would like, but I need to be able to vary $X_d$. I have a fair amount of data but it may be sparse in some areas, so it seems like a parametric model is a good place to start. But are there any standard parametric models for joint continuous/discrete data?

I could obviously go down the generative NN rabbit hole (GANs or VAEs w/ discrete-to-continuous encoding, for instance), but I'm curious about classical approaches as well.

icurays1
  • 502
  • 3
    if you only need $P(X_c|X_d)$, then you don't actually need to do any joint modeling and can get away with just doing regression, and the classical approach is to use a (generalized) linear model. In these approaches, whether $X_d$ is continuous or discrete is not of primary importance. – John Madden May 05 '23 at 19:48
  • How many different values can $X_d$ take? – jbowman May 05 '23 at 21:52
  • 1
    I agree with John Madden. If you're interested in the conditional distribution, the distribution of $X_d$ doesn't matter, so your main issue is a choice (if you need one) of the conditional distribution(s) themselves, and tools related to regression and GLMs would be a good place to start – Glen_b May 05 '23 at 23:24

3 Answers3

5

Assuming that the discrete variables are ordinal or binary, a simple assumption is that there is an underlying latent multivariate normal distribution, and that the discrete variables have been obtained by binning the corresponding normal distributions. Dependence can then be modelled by the correlations of the underlying normal. See R-package polycor and the literature listed on its help pages (particularly function hetcor).

Christian Hennig
  • 23,655
4

Parametric distributions for mixed discrete/continuous random variables can easily be built up as mixtures of standard parametric families of discrete distributions and continuous distributions. There are huge numbers of choices of mixtures you could make, depending on how many distributions you want to mix, and which ones you choose. In general, if you have any parametric discrete distribution with mass function $p_\theta$ and any parametric continuous distribution with density $f_\phi$ then you can create the corresponding parametric mixture distribution with CDF:

$$F(x|\theta, \phi, \lambda) = \lambda \sum_{r \leqslant x} p_\theta(r) + (1-\lambda) \int \limits_{0}^r f_\phi(r) \ dr.$$

While there are infinite ways you can do this, there are a few common cases of mixd random variables that arise in statistical analysis. One extremely common form of model for this is zero-inflation of a continuous random variable, where the continuous distribution is mixed with a point-mass distribution on zero. Another common form occurs when there is censorship of a continuous random variable over some of its range, which yields a mixture case.

Ben
  • 124,856
1

Olkin & Tate (1961) presented a general location model for mixed continuous and categorical variables. The categorical variables are cross-tabulated and an unstructured approach is to put a multinomial distribution over all the table's cells. Then conditional on each cell in the table, the continuous variables are distributed as multivariate normal. You can also structure the model, e.g., log-linear restrictions among the categorical items or a diagonal covariance matrix for the condition multivariate normal distributions, or a pooled, common multivariate normal distribution across all or parts of the contingency table.

Brian Flaherty
  • 21