Why is it complex to write Non-Normal Distributions in Multivariate form?

Question

In statistics, it's pretty straightforward to write a formula for the joint probability distribution of two random Normal Variables:

\begin{align*} x_1 &\sim \mathcal{N}(\mu_1, \sigma_1^2) \\ x_2 &\sim \mathcal{N}(\mu_2, \sigma_2^2) \\ \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} &\sim \mathcal{N}\left(\begin{bmatrix} \mu_1 \\ \mu_2 \end{bmatrix}, \begin{bmatrix} \sigma_1^2 & \rho\sigma_1\sigma_2 \\ \rho\sigma_1\sigma_2 & \sigma_2^2 \end{bmatrix}\right) \end{align*}

Or here is another way to write this that shows this formula in terms of correlations: \begin{align*} x_1 &\sim \mathcal{N}(\mu_1, \sigma_1^2) \\ x_2 &\sim \mathcal{N}(\mu_2, \sigma_2^2) \\ \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} &\sim \mathcal{N}\left(\begin{bmatrix} \mu_1 \\ \mu_2 \end{bmatrix}, \begin{bmatrix} \sigma_1^2 & \sigma_{12} \\ \sigma_{12} & \sigma_2^2 \end{bmatrix}\right) \end{align*}

And here it is for a N variable case:

\begin{align*} x_1 &\sim \mathcal{N}(\mu_1, \sigma_1^2) \\ x_2 &\sim \mathcal{N}(\mu_2, \sigma_2^2) \\ &\vdots \\ x_n &\sim \mathcal{N}(\mu_n, \sigma_n^2) \\ \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} &\sim \mathcal{N}\left(\begin{bmatrix} \mu_1 \\ \mu_2 \\ \vdots \\ \mu_n \end{bmatrix}, \begin{bmatrix} \sigma_1^2 & \sigma_{12} & \cdots & \sigma_{1n} \\ \sigma_{21} & \sigma_2^2 & \cdots & \sigma_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ \sigma_{n1} & \sigma_{n2} & \cdots & \sigma_n^2 \end{bmatrix}\right) \end{align*}

I am trying to understand why no one ever does this for Non-Normal variables (after spending the weekend trying to read about this, I could not find a single example of this for the Non-Normal case).

For example, suppose I had many Exponential random variables (note that the variance of a exponential random variable $x_i$ ~ $exp(\lambda_i)$ is ${1}/{\lambda_i}$):

\begin{align*} X_1 &\sim \text{Exp}(\lambda_1) \\ X_2 &\sim \text{Exp}(\lambda_2) \\ &\vdots \\ X_n &\sim \text{Exp}(\lambda_n) \end{align*}

How come we can't write:

\begin{align*} X_1 &\sim \text{Exp}(\lambda_1) \\ X_2 &\sim \text{Exp}(\lambda_2) \\ &\vdots \\ X_n &\sim \text{Exp}(\lambda_n) \\ \begin{bmatrix} X_1 \\ X_2 \\ \vdots \\ X_n \end{bmatrix} &\sim \text{Exp}\left(\begin{bmatrix} \lambda_1 \\ \lambda_2 \\ \vdots \\ \lambda_n \end{bmatrix}, \begin{bmatrix} \frac{1}{\lambda_1^2} & \sigma_{12} & \cdots & \sigma_{1n} \\ \sigma_{21} & \frac{1}{\lambda_2^2} & \cdots & \sigma_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ \sigma_{n1} & \sigma_{n2} & \cdots & \frac{1}{\lambda_n^2} \end{bmatrix}\right) \end{align*}

Mathematically, why is this incorrect?

And this links shows us how to calculate the covariance of two exponential variables: https://math.stackexchange.com/questions/4101399/finding-covariance-from-exponential-distribution

How come we can do this for the Normal Distribution and not the Exponential Distribution? In my engineering final project, we need to simulate correlated failure times for machines that statistically depend on one another. I am just trying to understand the mathematical//statistical reason why this is so easy for a Multivariate Normal but not for a Multivariate Exponential.

Any hints?

Answer 1

The key problem is that you have not given enough information to specify the joint distribution. With a Normal distribution, the mean vector and covariance matrix specifies the joint distribution. That's also true for an elliptical distribution. It's not true for exponential distributions. Given your mean vector $\lambda$ and your covariance matrix, there might be no multivariate distribution with exponential margins and the specified covariance matrix, or there might be uncountably many different ones. A related issue is that there's no clear way to simulate from this distribution.

When people make up new parametric distribution families (which they do all the time), the distributions are usually specified as one or more of:

• a multivariate density function (eg, the multivariate Normal)
• a recipe for generating the variable from simpler components (eg, "take a multivariate normal and divide by a square root of the covariance matrix)
• some provably unique mathematical characterisation (eg Chernoff's distribution, which is the location of the minimum of a Brownian motion with quadratic drift)

These specifications allow you to do things with the distribution.

There are occasions where you write down just part of a joint distribution and say "don't care" about the rest. For example, the idea behind GEE modelling of longitudinal data was to specify that the marginal distributions followed a generalised linear model and just punt on the rest of the specification. This gives a well-defined semiparametric model. Your "exponential margins" model is also a well-defined semiparametric model; it's not clear whether it's a useful one or not.