Questions tagged [p-value]

In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true.

In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true. (Extremity is defined with respect to the likelihood ratio of the alternative vs. the null hypothesis. Hence, extremity depends on the alternative.) When the $p$-value is small, the observed data would be unlikely to occur if the null were true. This fact is then typically used to argue that the null hypothesis is false. That is, a low p-value is typically interpreted as evidence suggesting the null hypothesis is false. The most common cutoff for a "small" $p$-value in research is $.05$.

2901 questions
13
votes
0 answers

How to find p value using estimate and standard error?

I am trying to check statement on p. 23 of Data Analysis Using Regression and Multilevel/Hierarchical Models For example, consider two independent studies with effect estimates and standard errors of 25 ± 10 and 10 ± 10. The first study is…
wildfluss
  • 131
  • 1
  • 1
  • 3
11
votes
2 answers

P-value vs Type 1 Error

I am trying to understand the relationship of a p-value vs Type 1 error rate. My understanding of one interpretation of a p-value is the following: "the p-value tells us the probability of making a type 1 error, conditional on the fact that the null…
Vincent
  • 605
10
votes
6 answers

How do you know if an insignificant P value is because of chance error or because the sample size of the data is too small?

I have always thought that an insignificant P value is the product of chance causing variation in the data. However when I was watching a talk on a study, the presenter mentioned how the insignificant P value meant that the sample size of the data…
8
votes
3 answers

Is a p-value a sample statistic, or a population parameter, or neither?

I would also be interested to know why it is one or the other.
6
votes
3 answers

Is this a valid way to think about p-values?

Would it be accurate to say that a p-value is a random variable whose null distribution is Unif$(0,1)$ which stochastically dominates its distribution under the alternative hypothesis? I realized that I have been thinking about p-values in the sense…
user135912
6
votes
1 answer

Estimating a p-value when you can't compute it for the whole set

I have a two large lists and I want to compute the p-value that they are similar. The similarity algorithm is a black-box, but for this application we will trust it gives accurate p-values. My problem is my list is too large and the algorithm…
Henry B.
  • 1,629
5
votes
1 answer

is this interpretation of the p-value legit?

The interpretation of the p-value is a very difficult thing because what is legit and what is not is very close together. The p-value is the probability of the resulting test statistic if the null hypothesis $H_0$ is true; that means…
duplex
  • 79
5
votes
1 answer

Does rejecting a study's credibility based on sample size make sense even if statistical significance is reached?

I've seen people do this commonly, and to me, it makes no sense because one of the main reasons of having a large enough sample size is that you CAN reach statistical significance in the first place.
Nick Corona
  • 127
  • 6
5
votes
3 answers

question about p values and significance comparison

I am under the impression that you have to set your alpha value before the experiment, and once the experiment is done, your p value, based on the alpha value, is either significant or not. It's dichotomous. There is no "almost significant" or…
Mekok
  • 51
3
votes
1 answer

Are these statements about p-values correct?

I've seen the following equivalent statements about p-values: "It is the probability of wrongly rejecting the null hypothesis if it is in fact true." [1] "The P value or calculated probability is the estimated probability of rejecting the null…
3
votes
1 answer

What p value did this study most likely use?

A study I’ve read shows that all of the results relevant to me were at >0.1 p value. Does it mean that the study used a <0.1 p value as significant? Is there still at possibility the study choose <0.05 as a significant p value, but just wrote the p…
3
votes
4 answers

Do p-values assume an unlikely event has already happened?

According to this thread What is the meaning of p values and t values in statistical tests? a p-value essentially lets me compare the probability of tail-end (unlikely) events occurring and judging that against the significance level $\alpha$ to…
3
votes
2 answers

does p-value distribution indicate the suitability of the statistical test

This post suggests that p-value follows uniform distribution in case of point null hypothesis and continuous data. In my project, I have millions of p-values from actual data. I don't want to get into the details of my data and statistic test. I…
3
votes
1 answer

Can you calculate a p value of less than .05 with an Eta-squared equal to 0?

This is a question for a study guide I can't quite wrap my head around. My thinking is that if Eta-squared is equal to 0, the between group deviations must be equal to 0, therefore it is impossible to have any evidence against the null (thus a p…
ss808
  • 33
3
votes
1 answer

Trying to draw conclusions from p-values

I am still trying to figure out how one can appropriately interpret p-values, and I am running into a bit of an issue with some of the data that I just finished analyzing. I have two types of plots in a common garden: Pure plots that only contains…
user83350
1
2 3 4 5