Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

10751 questions
66
votes
14 answers

If we fail to reject the null hypothesis in a large study, isn't it evidence for the null?

A basic limitation of null hypothesis significance testing is that it does not allow a researcher to gather evidence in favor of the null (Source) I see this claim repeated in multiple places, but I can't find justification for it. If we perform a…
bkoodaa
  • 1,299
  • 1
  • 12
  • 19
44
votes
4 answers

Do null and alternative hypotheses have to be exhaustive or not?

I saw a lot of times claims that they have to be exhaustive (the examples in such books were always set in such way, that they were indeed), on the other hand I also saw a lot of times books stating they should be exclusive (for example…
greenoldman
  • 663
42
votes
4 answers

Justification of one-tailed hypothesis testing

I understand two-tailed hypothesis testing. You have $H_0 : \theta = \theta_0$ (vs. $H_1 = \neg H_0 : \theta \ne \theta_0$). The $p$-value is the probability that $\theta$ generates data at least as extreme as what was observed. I don't understand…
xyzzyrz
  • 3,161
39
votes
7 answers

Why is the null hypothesis often sought to be rejected?

I hope I am making sense with the title. Often, the null hypothesis is formed with the intention of rejecting it. Is there a reason for this, or is it just a convention?
Prometheus
  • 898
37
votes
3 answers

Is it possible to change a hypothesis to match observed data (aka fishing expedition) and avoid an increase in Type I errors?

It is well known that researchers should spend time observing and exploring existing data and research before forming a hypothesis and then collecting data to test that hypothesis (referring to null-hypothesis significance testing). Many basic…
post-hoc
  • 697
  • 1
  • 6
  • 15
28
votes
1 answer

Multiple hypothesis testing correction with Benjamini-Hochberg, p-values or q-values?

Given a list of p-values generated from independent tests, sorted in ascending order, one can use the Benjamini-Hochberg procedure for multiple testing correction. For each p-value, the Benjamini-Hochberg procedure allows you to calculate the False…
gotgenes
  • 943
  • 2
  • 8
  • 9
25
votes
2 answers

Why is the null hypothesis always a point value rather than a range in hypothesis testing?

This is somewhat related to another question that I asked. The question I have is, when doing hypothesis testing, when the alternative hypothesis is a range, the null hypothesis is still a point value. As an example, when testing whether a…
Can't Tell
  • 581
25
votes
4 answers

Given big enough sample size, a test will always show significant result unless the true effect size is exactly zero. Why?

I am curious about a claim made in Wikipedia's article on effect size. Specifically: [...] a non-null statistical comparison will always show a statistically significant results unless the population effect size is exactly zero I am not sure…
gary
  • 505
24
votes
5 answers

Now that I've rejected the null hypothesis what's next?

I've time and again rejected or failed to reject the null hypothesis. In the failure to reject case, you conclude that there isn't sufficient evidence for rejection and you "move on" (i.e., either gather more data, end the experiment etc.,) But when…
PhD
  • 14,627
21
votes
2 answers

"Investigator intention" and thresholds/p-values

I'm reading John Kruschke's "Doing Bayesian Data Analysis" slides, but actually have a question about his interpretation of t-tests and/or the whole null-hypothesis significance testing framework. He argues that p-values are ill-defined because they…
Matt Krause
  • 21,095
17
votes
5 answers

Why do we need alternative hypothesis?

When we do testing we end up with two outcomes. 1) We reject null hypothesis 2) We fail to reject null hypothesis. We do not talk about accepting alternative hypotheses. If we do not talk about accepting alternative hypothesis, why do we need to…
user1700890
  • 677
17
votes
4 answers

Acceptance of null hypothesis

This is a discussion question on the intersection of statistics and other sciences. I often face the same problem: researchers in my field tend to say that there is no effect when the p-value is not less than the significance level. In the…
German Demidov
  • 1,731
  • 13
  • 27
16
votes
4 answers

How to specify the null hypothesis in hypothesis testing

What is a good rule of thumb for how to choose the question for the null hypothesis. For instance, if I want to check if the hypothesis B is true, should I use B as the null, B as the alternative hypothesis, or NOT B as the null? I hope the question…
Nestor
  • 407
16
votes
2 answers

Is it possible to accept the alternative hypothesis?

I'm aware of several related questions here (e.g., Hypothesis testing terminology surrounding null, Is it possible to prove a null hypothesis?) but I don't know the definitive answer for my question below. Suppose a hypothesis test where we want to…
ckcn
  • 161
15
votes
3 answers

What does Bayesian Hypothesis Testing mean in the framework of inference and decision theory?

My background is mainly in machine learning and I was trying to learn what Bayesian Hypothesis testing meant. I am ok with the bayesian interpretation of probability and I am familiar with it in the context of probabilistic graphical models.…
Charlie Parker
  • 6,866
1
2 3
…
41 42