2

Background

This is related to the following question: Is an experiment's Sample Space always useful?

I understand that Sample Spaces are sets holding all the possible outcomes of an experiment. Thanks to a great answer here, I understand that a Random Variable assigns a value to each outcome in the sample space. And from this answer, that it's a mapping function from a sample space to the real space:

$$X : S \rightarrow \mathbb{R}$$

However, in my previous question, I was told random variables are used over sample spaces in complex situations. Whilst I'm fine with this in theory, I'm struggling to find a practical example. Whereas I can easily think of examples for using just the sample space, i.e. simple dice roles etc.

Question

When is it easier to solve a problem with a Random Variable than a sample space?

Connor
  • 625
  • From a mathematical perspective this looks like a pointless question, because part of the definition of a random variable $X:S\to\mathbb R$ is its domain $S,$ the sample space. What distinction, then, is implied by "using just the sample space" compared to employing a random variable in problems where the random variable is the object of analysis? – whuber Dec 12 '23 at 15:10
  • @whuber You have a score on Cross Validated of 316.7k. Do you think it's probably fair to say you have a strong intuition and understanding of how everything connects together and where it's useful? Probably better than 99.9999% of people on the planet? My question is based in ignorance that I cannot find an answer for elsewhere. I asked a question previously, that I have linked to. In the answer to that question I was told the following: "Rather in probability theory we tend to work with 'random variables'. ... By using random variables, we make thinking of random experiments a lot simpler." – Connor Dec 12 '23 at 16:09
  • @whuber Perhaps the above is meaningless and I'm being led down a rabbit hole. If you think so, please say. But if they're The Same why do we have random variables? Would you like me to change the question to "When is the Random Variable abstraction that is built on a sample space more helpful than just the sample space?" Or something similar? Please tell me how you want me to word the question and I'll change it. – Connor Dec 12 '23 at 16:18
  • The issue might be, more helpful for what? It sounds like you're trying to get a sense of what kinds of problems are more easily addressed by considering sample spaces and what kinds of problems are more easily addressed using random variables -- but that's that kind of broad, list-of-answers kind of question that doesn't work in our format. – whuber Dec 12 '23 at 17:04
  • Okay, so if I find a single complex problem that uses random variables and I'm not sure why, it would be easier to ask a question about that directly? – Connor Dec 12 '23 at 21:54
  • Conceivably it would be, especially if the definition or choice of the random variable were not obvious and somehow -- it's hard to imagine how, frankly -- the problem could be solved without random variables but using properties of the sample space itself. – whuber Dec 13 '23 at 00:01
  • @whuber It's possible to think of simple problems with dice though where the sample space is all you need, right? Or are you saying that even in those simple situations you're creating a random variable? – Connor Dec 13 '23 at 10:26
  • 1
    A die already is a random variable: its values are plainly written on its faces. The sample space can be modeled as the set of possible upward faces, while the random variable is given by the number of pips on the upward face. – whuber Dec 13 '23 at 15:36

1 Answers1

4

There is no general answer to this, as it depends on each individual situation. In some situations, working with sample spaces is enough, such as computing the probability that a certain card shows up in a random subset of a deck of 52 cards. In other situations, things get much more complicated, such that even random variables are not enough. In this case, you would need to consider the measure-theoretic foundations. To quote section 1 of chapter 1 of A User's Guide to Measure Theoretic Probability by David Pollard:

For a rigorous treatment of probability, the measure theoretic approach is a vast improvement over the arguments usually presented in undergraduate courses. Let me remind you of some difficulties with the typical introduction to probability.

Independence

There are various elementary definitions of independence for random variables. For example, one can require factorization of distribution functions, $$ \Pr(X \leq x, Y \leq y) = \Pr(X \leq x) \cdot \Pr(Y \leq y) \ \text{for all real $x,y$} $$ The problem with this definition is that one needs to be able to calculate distribution functions, which can make it impossible to establish rigorously some desirable properties of independence. For example, suppose $X_1,\dots,X_4$ are independent random variables. How would you show that $$ Y = X_1X_2 \left[\log\left(\frac{X_1^2 + X_2^2}{|X_1| + |X_2|}\right) + \frac{|X_1|^3 + X_2^3}{X_1^4 + X_2^4}\right] $$ is independent of $$ Z = \sin\left[X_3 + X_3^2 + X_3X_4 + X_4^2 + \sqrt{X_3^4 + X_4^4}\right] $$ by means of distribution functions? Somehow you would need to express events $\{Y \leq y,Z \leq z\}$ in terms of the events $\{X_i \leq x_i\}$, which is not an easy task. (If you did figure out how to do it, I could easily make up more taxing examples.) You might also try to define independence via factorization of joint density functions, but I could invent further examples to make your life miserable, such as problems where the joint distribution of the random variables are not even given by densities. And if you could grind out the joint densities, probably by means of horrible calculations with Jacobians, you might end up with the mistaken impression that independence had something to do with the smoothness of the transformations.

The difficulty disappears in a measure theoretic treatment, as you will see in Chapter 4. Facts about independence correspond to facts about product measures.

Random variables are especially useful when the sample space is large. Consider the following random experiment: I toss a coin five times. The sample space can be written as $$ S=\{TTTTT,TTTTH,... , HHHHH\}. $$ Note that here the sample space $S$ has $2^5=32$ elements. Suppose that in this experiment, we are interested in the number of times the coin flip results in 3 heads. We can list all the cases when this happens manually as a subset of the sample space $S$. However, this would be cumbersome. Instead, we could define a random variable $X$ whose value is the number of observed heads. The value of $X$ will be one of 0,1,2,3,4 or 5 depending on the outcome of the random experiment. Then, by computing $X$ for each element of $S$, we get the collection $\{0,1,\dots,5\}$ that contains more than one $3$. We then count the number of $3$'s that are in this collection.

mhdadk
  • 4,940
  • Interesting! So measure theory is the step beyond Random Variables when things get really complicated? If you have a single example of where random variables are easier and why, I'll happily accept this answer! – Connor Dec 12 '23 at 08:40
  • 1
    @Connor Yes, measure theory offers a rigorous foundation for probability theory. It would be difficult to come up with your desired example because it is likely that we have different definitions of easier, unless you can provide an exact definition of what "easier" means here. – mhdadk Dec 12 '23 at 08:41
  • An example where the sample space is secondary in consideration to the Random Variable. Or where the sample space is understood to be a background detail, but you don't need to think about it to solve the problem. – Connor Dec 12 '23 at 08:51
  • @Connor A simple example of this is a coin flip: let $X = 0$ if the coin flip yields a tails, and $X = 1$ if the coin flip yields a heads. The probability mass function of $X$ is Bernoulli. The sample space in this case is ${H,T}$. – mhdadk Dec 12 '23 at 11:46
  • Thank you, could you add more on why this might become useful in a complicated problem? I understand the idea of a random variable and a sample space, but it would be good to see where a random variable is more useful for solving problems. It exists for a reason, right? – Connor Dec 12 '23 at 12:09
  • @Connor I added an example at the end of my answer. – mhdadk Dec 12 '23 at 12:30
  • Thank you again! The only part that feels missing is why that's better? In this case it looks like the random variable is just a transform from the sample space into something that aggregates values. Is the aggregation of values, related to what you're interested, the part that's most useful? And is there an example that is more complicated in infinite sample spaces? – Connor Dec 12 '23 at 13:26