1

I have a set of data parameterized by a single variable which is nearly perfectly linear, and I am trying to quantitatively determine with what confidence we can say a theoretical quadratic term is zero. So far I have done this simply by fitting the data twice, once with a purely linear fit, and once also including a quadratic term.

Though in the quadratic fit I find the quadratic term is consistent with 0, including the quadratic also changes the linear fit term, I think because of a covariance between the two. I am trying to determine if there is a quantitative way in which I can say something to the effect of: "with XX% confidence, I confirm there is no quadratic term, thus fit only up to first order". I have looked into tests such as Neyman–Pearson am not sure if these are fruitful avenues.

For reference, here is an example of the data below. In the legend I show the fit parameters, where the term in the parentheses is the 1-sigma error on that fit parameter.

Edit: I want to clarify that the problem I am ultimately trying to solve is that though the quadratic term is consistent with zero (as evidenced by its standard error), including it or not changes the linear and constant terms - I am trying to quantitatively say with what confidence we can ignore the quadratic term, and fit only up to linear order.

enter image description here

KHAAAAAAAAN
  • 143
  • How did you determine that the quadratic term was "consistent with zero", and why isn't that sufficient for your purposes? – jbowman Oct 26 '22 at 18:56
  • The error bar on the fit parameters is shown in the legend - you can see that the "c" parameter in the quadratic fit has an error bar which overlaps zero. However, even though it is "consistent" with 0, it still changes the value of "a" and "b" to include the parameter "c" in the fit, so I am trying to find a way to justify not including "c" in the first place. – KHAAAAAAAAN Oct 26 '22 at 19:33
  • 1
    Where do the standard errors come from and what happens between X = 300 and X = 400 to increase the standard errors so much? – dipetkov Oct 26 '22 at 19:52
  • Some of the trouble with this is that, even if you can assert your claim about having confidence in not having a quadratic effect, by testing for the quadratic effect, you influence your downstream inferences done on the linear-only model. This is related to (if not an example of) a technique called stepwise regression. – Dave Oct 26 '22 at 19:53
  • 1
    One approach would be to use a t-test of the quadratic coefficient along the lines suggested by @KennethGranahan . Or look to see if confidence intervals for the quadratic coefficient include zero or not. Another approach would be to compare the linear model and the quadratic model using something like adjusted R-squared, AIC, or BIC. ... Did you try anything like these approaches ? – Sal Mangiafico Oct 26 '22 at 20:02
  • I did check that the 1-sigma confidence interval for the quadratic coefficient includes zero, yes - this can be seen in the figure legend. However, my worry is that by including the quadratic coefficient, even if it is consistent with zero, the linear and constant terms are also changed to compensate. My goal is to fit just up to the linear term and quantitatively say "for the data range considered here we have XX% confidence there is no detectable quadratic dependence" – KHAAAAAAAAN Oct 26 '22 at 20:35
  • 1
    The intercept and linear slope change because the meaning of the linear slope and intercept change when you have a quadratic term in the model. If you don't want them to change, use an orthogonal polynomial. This isolates the quadratic term so that its inclusion or exclusion don't change the other terms in the model. You can check to see how much variance is associated with the linear component vs quadratic component more easily (look at my answer on the post). – Noah Oct 26 '22 at 20:52

1 Answers1

1

You can do this using hypothesis testing. In your case you have a proposed quadratic model of the form: \begin{equation} y = a + bx +cx^2 + \epsilon \end{equation}

You've fitted this model to find estimates for each coefficient, call these estimates $\hat{a}, \hat{b}$ and $\hat{c}$. Now what you want to do is investigate if there is a statistically significant difference between $\hat{c}$ and 0.

You do this by

  1. Stating the null and alternative hypotheses (denote $H_0$ and $H_a$ respectively)
  2. Calculating the appropriate test statistic
  3. Choosing a level of significance
  4. Deciding on whether to accept of reject the null hypothesis based on the result

Here we have the hypotheses $H_0: \hat{c}=0$ and $H_a: \hat{c}\neq0$.

The appropriate test statistics is the t-statistic, which in this case is \begin{equation} t =\frac{\hat{c}-0}{se(\hat{c})} = \frac{\hat{c}}{se(\hat{c})} \end{equation} where $se(\hat{c})$ is the standard error of $\hat{c}$.

Once $t$ has been calculated, determine the critical value $t_\alpha$ for your chosen level of significance from a two tailed t distribution table. If $t<t_\alpha$ then you accept the null hypothesis(i.e. $\hat{c}=0$)

Kenneth Granahan
  • 11
  • Welcome to Cross Validated! The trouble with this is that a small sample size can result in failing to reject the null hypothesis, even if there really is a meaningful quadratic effect. In other words, this approach gives no evidence that your study had enough power to detect a quadratic effect of interest. – Dave Oct 26 '22 at 19:50