Linear regression: p-value for linearHypothesis() in R

Question

Let's start with the Davis dataset from car package and fit a simple regression model

data(Davis)
mod.davis <- lm(weight ~ repwt, data = Davis)
summary(mod.davis)
Coefficients:
            Estimate Std. Error t value Pr(>|t|)

(Intercept)   5.3363     3.0369   1.757   0.0806 .

repwt         0.9278     0.0453  20.484   <2e-16 ***

Now I'd like to test \begin{align*} \beta_0 &= 5 \\ \beta_1 &= 1 \end{align*}

linearHypothesis(mod.davis, c("(Intercept) = 5", "repwt = 1"))

what I get is

  Res.Df   RSS Df Sum of Sq      F    Pr(>F)    
1    183 16549                                  
2    181 12828  2      3721 26.251 9.764e-11 ***

The low p-value is quite puzzling for me

confint(mod.davis)
                 2.5 %    97.5 %
(Intercept) -0.6560394 11.328560
repwt        0.8384665  1.017219

especially because of the fact, that for \begin{align*} \beta_0 &= 0 \\ \beta_1 &= 1 \end{align*}

linearHypothesis(mod.davis, c("(Intercept) = 0", "repwt = 1"))
Res.Df   RSS Df Sum of Sq      F Pr(>F)
1    183 13074

2    181 12828  2    245.97 1.7353 0.1793

the data suggests that $H_0$ is true. How is that? $\beta_0 = 5.33$ is the mean of the sampling distribution, so I'd expect a p-value $< 0.05$ rather for testing for $\beta_0=0$.

Answer 1

Let's look at the data and your hypothesis.

The first figure is a plot you should always make: a scatterplot of the data. On it I have drawn a 95% confidence band for the fit (in gray) and your hypothesis in red, the line with intercept 5 and slope 1:

The red line is not a good fit: almost all of the nearly 200 points in the plot lie below it. But does it really deserve a p-value as low as $10^{-10}$?

One way to get some intuition is to simulate these data. This isn't something you would routinely do, but doing it at least once is a good part of any statistical education and is a great way to check your analysis is correct.

Here, I sample from the estimated parameters (slope and intercept) using their estimated variance-covariance matrix. The next plot shows 50 such samples, each drawn as a gray line.

It shows, among other things, that the hypothesized line isn't close to any of these 50 lines: all of those simulated lines tend to go near the "point of averages" near $(66,66),$ whereas the hypothesized line is visibly higher there.

Perhaps a clearer way to see how unlikely the hypothesis is uses the more abstract method of plotting the slopes and intercepts as points. (This is the "parameter space" in which the possible values of (intercept, slope) lie.) This time I show 500 simulated values as faint points and their density as a colored contour plot.

The red point is the hypothesized line. Its extremely low p-value reflects two things:

• The point is visibly far from any of the simulated data, especially in the northeast - southwest direction. That is, it might not be far from any of the intercepts (it's not) and it might not be far from any of the slopes (it's not), but it truly is a "geometric outlier" when examined in the full context. (Its Mahalanobis distance is large.)

• The p-value is computed assuming these points follow a bivariate Normal density. Such densities decay extremely quickly away from their centers (their logarithms, by definition, decrease parabolically -- that is, their decrease accelerates with distance).

If you make a different distributional assumptions about the errors in the data, you will obtain different p-values: but clearly they will all be small, because this hypothesis is a poor description of the data.