5

Background

I'm working through the Harvard Stat 110 probability course. One of the first topics discussed is the sample space of an experiment.

I understand that the sample space represents all the possible outcomes of the experiment and that outcomes can be collected into distinct events $A$.

Intuitively, doing this for simple experiments with a finite sample space and equally likely outcomes makes sense to me. Particularly, as it allows you to use the Naive Definition of Probability. But when you have an infinite number of unequally likely outcomes, it's not clear to me why defining the sample space here is useful (or possible?).

Question

Does the usefulness of an experiment's sample space break down when you have unequal probabilities or an infinite number of outcomes?

Connor
  • 625

2 Answers2

7

The sample space is useful for easy examples when you are learning probability theory. However, when you have very complex random experiments it becomes very hard to describe the sample space. For instance, there is something called a "branching" process. A branching process is when you have a single-parent, and that parent has a random number of children, those children can themselves have a random number of children, ect. This process might terminate, but it can also continue indefinitely. Here is are some pictures that illustrate one such possible path, enter image description here

In the first and second image the branching process terminated, whereas in the third image the process continued forever. The sample space $\Omega$ will be the collection of all of these trees. This is an absolutely enormous sample space which is very hard to describe.

Rather in probability theory we tend to work with "random variables". A random variable takes each sample point in your sample space and assigns to it a quantity. For example, $X$ can represent the number of generations of each tree. Then we can ask for $P(X=\infty)$, the probability that the branching process continues indefinitely. By using random variables, we make thinking of random experiments a lot simpler.

However, it is important to understand, from a theoretical viewpoint, all random variables are defined on some sample space. Therefore, a mathematician will say $X$ is a random variable defined on the sample space $\Omega$ of all trees. However, we tend not to go back to the sample space. The foundations of probability theory require a sample space, however, when we do computations/calculations there are ways to avoid the sample space.

Nicolas Bourbaki
  • 2,859
  • Okay, so, the sample space is an underlying abstract idea that allows you to work on difficult problems. But the actual tools you use to solve those problems allow you not to think about the sample space? Also, doesn't that mean a random variable is basically a sample space by itself? Or representative of one? – Connor Dec 10 '23 at 09:30
  • 1
    Yes, the random variable $X$ is a sample space itself. In the following sense, $X$ is a mapping which takes a sample point in $\Omega$ and assigns to it a real value in $\mathbb{R}$. Therefore, the possible outputs that we can see under $X$ is some set of numbers, called the "support of X". However, notice by working with the support of X, rather than the sample space itself, you have changed the problem into something more manageable that consists of numbers as opposed to trees. – Nicolas Bourbaki Dec 10 '23 at 21:20
  • Thank you, that's great! To be totally clear then, the support of X is the range of possible probabilities that the mapping function $X$ can output? Also, from a historical point of view, why is it called the support of $X$? – Connor Dec 11 '23 at 12:49
  • 1
    @Connor Yes, supp(X) is the range of X, when you view X as a mapping from the sample space into the set of R (the real numbers). Why is it called the "support"? When you have a function as a mapping from R to R, the support of f, supp(f), is defined as the subset of the domain where the function is non-zero. In a way the support of f is the relevant non-zero part of the function. With random variables "support" is similar, but a bit different, as the case with functions. – Nicolas Bourbaki Dec 11 '23 at 16:09
0

Consider an experiment where you toss a fair coin until you get a head. Let the measured variable $X$ be the number of tosses required to get a head.

If you sampled $TTH$, $X=3$. If you sampled $TTTTH$, $X=5$.

The sample space here is $\{H, TH, TTH, TTTH, ...\}$ It is infinite, and all outcomes have unequal probabilities.

Yet it is useful to have the sample space while doing calculations such as expectation or variance etc.

In order, the probability of the sample is $[(1/2),\; (1/2)^2,\; (1/2)^3,\; ... ]$. In order, the corresponding value of $X$ is $[1,2,3,... ]$.

whoisit
  • 101
  • I would say that in your example you are rather describing a random variable than the sample space. The sample space are infinitely sequences of H and T, and the measure function on that space is complicated. But by assigning a random variable X that counts the waiting time until you get heads it makes the problem more manageable. Usually, we work with random variables than sample spaces directly. – Nicolas Bourbaki Dec 10 '23 at 09:24