Can you say that you reject the null at the 95% level?

Question

I know that you reject the null at the 5% significance level. But I read someone writing that they reject at the 95% level. I assume the confidence level. But can you technically say that?

Answer 1

Sure. This is just a case of sloppy language.

You can either say "I reject the null hypothesis with 95% confidence" or "I reject the null at a significance level of .05." Both of these statements are shorthand for this much longer statement:

"If the null hypotheses were true, and I repeated this experiment/survey/analysis a large number times with a different random sample each time, then in less than 5% of those samples would I have found a deviation from the prediction of the null as large or larger than the one I found in the sample I actually have."

When people are talking about stats we sometimes talk about the level of confidence (95%) and sometimes we talk about the error rate (5%), but they're both just different ways of talking about the same level of confidence/significance. The only reason this doesn't end up being that confusing in practice is that no one in their right mind would ever actually claim that they rejected the null at "5% confidence" or with a "95% error rate."

Answer 2

You can* but should not.

You're conflating language of confidence intervals with hypothesis tests. The level of a test is the significance level. "Confidence" levels (coverage) is a property of confidence intervals. There are connections between them but they are NOT the same thing and you should not mix terminology between them.

If you're performing hypothesis tests, use the terminology of hypothesis tests. If you're calculating confidence intervals use the terminology of confidence intervals. What could be easier?

* (I sure can't stop you, these space lasers are useless)