A basic question about a randomized test involving the error type I

Question

I have a basic question in the context of testing statistical hypotheses, more specifically, about randomized tests. Suposse that I have two actions (altenatives) about a certain unknown parameter $\theta \in \Theta$: the Null ($H_0$) and altenative hypotheses ($H_1$).

In this case, the sample space is $(0,15) \subset \mathbb{R}$. We know that the critical function is given by $$\phi(x) = P(reject \, \, H_0 \,\,|\,\, x \, \, observed)$$

I don't know exactly if this definition really involves conditional probability. Suposse I have the following critical function

$$ \phi(x)= \begin{cases} 0, \quad x \in (0,2)\\ p, \quad x \in (2,10)\\ 1, \quad x \in (10,15)\\ \end{cases} $$

I can not understand why

$$P( reject \,\, H_0 \,\,|\,\, H_0 \,\, is\,\, true)\, = 0 \times P(x \in (0,2)) + p \times P(x \in (2,10)) + 1 \times P(x \in (10,15))$$

The right side looks a lot like a some expectation. But I can not understand the equality.

Answer 1

Let's parse the notation and then answer your questions.

This "critical function" $\phi$ is a tool to make a decision. Given the value $X$ of a test statistic, independently observe a uniformly distributed variable $U$ (supported on $[0,1]$). Reject the null hypothesis when $U \le \phi(X);$ otherwise, do not reject it.

To be explicit, given an experimental outcome $\omega$ (which is typically a random sample from the population), one often writes $X(\omega)$ for its test statistic. We may, if we choose, consider the "outcome" to include the random value $U$, allowing us to write $U(\omega)$ for the random value used in the decision process.

In these terms, the event "reject $H_0$" consists of all outcomes $\omega$ for which $U(\omega)\le \phi(X(\omega)).$ You can visualize this condition as a set in the $(X,U)$ plane. The set is (of course) limited to the supports of $X$ and $U:$ that is, it's a subset of the rectangle $[0,15]\times [0,1].$

With this plot in hand, making a decision is clear and simple: compute $X$ from the data and generate $U$; plot the point $(X,U)$ on the preceding figure; and reject $H_0$ if and only if the point lies within the shaded region.

Now for the answers:

1. The probability $\Pr(\text{reject }H_0\mid X)$ really is a conditional probability. That is explicit now that "reject $H_0$" has been exhibited as an event and $X$ has been described as a random variable.

2. In this problem the shaded region naturally decomposes into three disjoint rectangles according to how $\phi$ has been expressed: $$\text{Reject } H_0 = [0,2) \times {0} \cup [2,10)\times [0,p] \cup[10,15] \times [0,1].$$ (This decomposition is not unique: a look at the figure may suggest other ways to carve the region up into rectangles.) The chance of any such rectangle is easy to compute using the axioms of probability:

   • Because the rectangles are disjoint, their chances add.

   • Because $X$ and $U$ are (by construction) independent and $U$ has a uniform distribution, the chance of any rectangle $[a,b)\times [c,d]$ is the product of the chances, $$\Pr\left([a,b)\times [c,d]\right) = \Pr(X\in [a,b))\,\Pr(U\in [c,d]) = (d-c) \Pr(X\in [a,b)).$$

   Applying these observations yields $$\eqalign{\Pr(\text{Reject }H_0) &= \Pr(X\in [0,2))\times 0 + \Pr(X\in [2,10))\times (p-0) + \Pr(X\in [10,15])\times (1-0) \\ &= 0\Pr(X\in [0,2)) + p \Pr(X\in [2,10)) + \Pr(X\in [10,15]).}$$

3. The foregoing can be viewed as an expectation in exactly the same way any probability is an expectation, by introducing the indicator function. Define $$I_{\text{Reject}}(\omega) = \left\{ \matrix{1 & H_0\text{ is rejected}\\0 & \text{otherwise.}}\right.$$ Then by definition $$E[I_{\text{Reject}}] = 1\,\Pr(H_0\text{ is rejected}) + 0\,\Pr(H_0\text{ is not rejected}) = \Pr(H_0\text{ is rejected}).$$