1

A common point of confusion in introductory math stats classes is the difference between the joint density (formally a function of the data parameterized by the parameters) and the likelihood (formally a function of the parameters parameterized by the data).

Is there a name for that same expression considered formally as a function both of the parameters and data?

John Madden
  • 4,165
  • 2
  • 20
  • 34
  • 1
    It is a non-negative function with the property that when integrated over the possible data values it gives $1$. Remember that the likelihood is only proportional to this function rather than being equal to it. – Henry Jun 13 '23 at 14:52
  • @Henry that is indeed a property of this function; it has other properties too, like being measurable. But I am curious if this function has a name: I want to explicitly introduce it for pedagogical purposes, the notion being that it may be helpful to students to have this object to contrast with likelihoods and densities, and to be able to visualize that the density and likelihood are each one dimensional slices of this object (in the 1D parameter/sufficient stat case). – John Madden Jun 13 '23 at 14:56
  • 1
    It is the probability density function for the data conditioned on the parameter. The likelihood for the parameter given the data is proportional to it. – Henry Jun 13 '23 at 15:01
  • @Henry Let $\mathcal{X}$ denote the sample space (which we shall assume is independent of the parameter $\theta$), and $\Theta$ denote the parameter space. The density, which we'll assume is a function, is a mapping $\mathcal{X}\to\mathbb{R}^+$. The likelihood is the same expression viewed as a mapping $\Theta\to\mathbb{R}^+$. I am wondering if there is a name for this expression viewed as a $\mathcal{X}\times\Theta\to\mathbb{R}^+$ function. – John Madden Jun 13 '23 at 15:04
  • What is wrong with "the probability density function for the data conditioned on the parameter"? That has $\mathcal{X}$ and $\Theta$ as you wish so $\mathcal{X}\times\Theta\to\mathbb{R}^{\ge 0}$. Only for a fixed $\theta \in \Theta$ is it "the probability density function for the data" and $\mathcal{X}\to\mathbb{R}^{\ge 0}$ – Henry Jun 13 '23 at 15:54
  • @Henry re: "the probability density function for the data conditioned on the parameter" in my view it is ambiguous as to whether that construction considers the parameter to be an input to the function or a parameter of the function, and clearing up this kind of ambiguity is my pedagogical objective. – John Madden Jun 13 '23 at 16:00
  • As an example, you seem to want to distinguish between the function $g(x, \theta)=\theta e^{-\theta x} \mathbf I_{[x>0]}$ and the probability density function $f(x \mid \theta)=\theta e^{-\theta x} \mathbf I_{[x>0]}$ and the likelihood function $L( \theta \mid x)\propto\theta e^{-\theta x} \mathbf I_{[x>0]}$ despite their similarities. I offered you a name for $f(x \mid \theta)$ "the probability density function for the data conditioned on the parameter" which you dislike as too specific, and a name for $g(x,\theta)$ "a non-negative function with some properties" as not specific enough. – Henry Jun 13 '23 at 16:09
  • @Henry I think the name you gave $f(x|\theta)$ is appropriate; I'm looking for a name for $g$. Yes, "a non-negative function with some properties" is indeed too broad, as there is no connection to the underlying probability model. – John Madden Jun 13 '23 at 17:09
  • 2
    The closely related thread on the distinction between likelihood and probability might offer some suggestions. It is, however, not evident that any terminology is needed since we already have "likelihood," "density," "pdf", "probability element," and the associated mathematical symbolism to make everything clear. – whuber Jun 13 '23 at 17:17
  • 2
    The density is measurable as a function of $x$ while the likelihood does not have to be as a function of $\theta$ since there is no default measure over the parameter space. – Xi'an Jun 13 '23 at 17:28
  • @whuber I suppose I am asking if there is a name for the function you denote as $\Lambda$ in your answer to that question. On the basis of this discussion, I am concluding that the answer is no. – John Madden Jun 13 '23 at 17:39
  • To begin with, I would be careful with the use of "joint." In the present context it suggests you are conceiving of the data as vector-valued. Don't do that when introducing the topic! That leaves you with referring to the likelihood as a function of the parameters (namely, functioning as a density) or as a function of the data (which we may fairly term a likelihood). What else might be needed? For a Bayesian analysis, the parameters are now conceived of as random variables and there the term "joint" can differentiate this third idea from the first two. – whuber Jun 13 '23 at 17:47
  • @whuber I use "joint" to refer to the density of an iid sample of a scalar RV, which is at least how likelihoods were introduced to me. As I allude to above, I think it would be useful to visualize $\Lambda$ as 2D function, so I can explicitly show students how the likelihood and density are simply slices of this same function along different axes. I'm planning on leaving Bayes out of this, but I agree that joint makes sense in that context (though in a different sense). – John Madden Jun 13 '23 at 19:19
  • 1
    Again, I think it's good advice to start with a univariate setting. "$x$" generically refers to your observations and you can conceptualize it as a single variable. Similarly, "$\theta$" refers to your parameters and you can conceptualize it as a single parameter. Once this is understood, it's simple and natural to point out that a collection of data can be considered a single vector-valued $x$ and more than one parameter can still be considered a single entity, still called $\theta.$ That gives you permission to draw your 2D graph. – whuber Jun 13 '23 at 19:25
  • @whuber yea that makes sense. – John Madden Jun 13 '23 at 19:32
  • And @whuber, if you'll allow me my esprit de l'escalier: though you question the need for terminology describing the function $\Lambda$, you do after all introduce it as part of your answer describing what a likelihood is :) – John Madden Jun 13 '23 at 20:01
  • I intended to write "additional terminology." I am a believer in using existing terminology where possible and, if there is still a need to make finer distinctions than are commonly available, to feel free to invent new terms. – whuber Jun 13 '23 at 21:07

0 Answers0