How to determine sample space, $\sigma$-algebra and probability measure from the exponential family?

Question

The sample space of binomial distribution is the set $\{0,1\}$ and its $\sigma$-algebra is the power set of $\{0,1\}$ while the sample space of normal distribution is $\mathbb R$ and its $\sigma$-algebra is the Lebesgue measurable set. In these cases, the probability measure of binomial distribution is of the form of exponential family while the radon-nikodym derivative of the probability measure (i.e. pdf) of normal distribution is of the form of exponential family. Their natures are so different and why do people even call it a family?

My questions are:

How do we determine if exponential family gives us the probability measure or the probability density function?
How can we determine the sample space, $\sigma$-algebra and its probability measure from an exponential family?

Any help are welcome.

The sample space of a Binomial $\mathcal B(n,p)$ is ${0,1,\ldots,n}$. — Xi'an, Jul 29 '21 at 03:14

Xi'an · Accepted Answer · 2021-07-29T05:00:45.623

5

Some elements of answer:

The pmf for a distribution with a countable sample space is its density wrt to the counting measure.
Exponential families are made of families of parametric distributions whose density wrt to a given dominating measure $\text d\lambda(x)$ write as $f_\theta(x)=\exp\{R(\theta)^\text{T}S(x)-\psi(\theta)\}$, where $\psi(\theta)$ is the log-normalising constant.$^1$ The Binomial and the Normal distributions belong to different exponential families.$^2$
the sample space, $\mathfrak X$, its σ-algebra $\mathfrak B(\mathfrak X)$ and the dominating measure $\text d\lambda(x)$ are components of the definition of a given exponential family, just as much as the functions $R(\cdot)$ and $S(\cdot)$ above. If they are not provided, the associated exponential family is not defined.$^3$

$^1$I purposedly did not include the traditional $h(x)$ in the representation of $f_\theta(x)$, i.e., $f_\theta(x)=h(x)\exp\{R(\theta)^\text{T}S(x)-\psi(\theta)\}$ to stress the fact that $h(\cdot)$ can as well be part of the dominating measure. Changing $h(\cdot)$ thus amounts to changing the dominating measure.

$^2$In particular, the Binomial distribution is absolutely continuous wrt the counting measure on $\mathfrak X=\mathbb N$ and the Normal distribution is absolutely continuous wrt the Lebesgue measure on $\mathbb R$

$^3$In other words, a probability distribution does not exist independently of its dominating measure, which itself does not exist independently of its σ-algebra $\mathfrak B(\mathfrak X)$, which itself requires a sample space $\mathfrak X$

edited Jul 29 '21 at 05:00

answered Jul 29 '21 at 03:25

Xi'an

105,342

I understand what a measure and a probability measure are, but a 'dominating' measure is new to me. What is it? – Galen Jul 29 '21 at 03:27
1

If you invoke the Radon-Nykodim theorem, you are using dominating measures. – Xi'an Jul 29 '21 at 03:35
1

Thanks for the link. It led me to this definition: "A measure $\mu$ on Borel subsets of the real line is absolutely continuous with respect to the Lebesgue measure $\lambda$ if for every measurable set $A$, $\lambda (A) = 0 \implies \mu (A) = 0$. This is written $\mu << \lambda$. We say $\mu$ is dominated by $\lambda$." – Galen Jul 29 '21 at 03:41
Is there any relation between the 3rd bullet point and the Pitman-Koopman-Darmois theorem besides the superficial analogy? In the first case, we require to specify the domain to make the definition of a distribution meaningful. In the second case, having a fixed support is one of the theorem's assumptions. – paperskilltrees Jul 29 '21 at 04:26
1

Except for the fact that the PKD theorem is about exponential families, I do not see the relation. From its definition, an exponential family distribution necessarily has a sample space that does not vary with the associated parameter. – Xi'an Jul 29 '21 at 04:57
1

Thank you from your clarification. The traditional $h(x)$ bothered me a lot. – fuyutsuki Jul 29 '21 at 11:45

How to determine sample space, $\sigma$-algebra and probability measure from the exponential family?

1 Answers1