Does anyone know how to work out whether points 7, 16 and 29 are influential points or not? I read somewhere that because Cook's distance is lower than 1, they are not. Am, I right?
Asked
Active
Viewed 1.2e+01k times
54
-
1There are various opinions. Some of them relate to the number of observations or to the number of parameters. These are sketched at http://en.wikipedia.org/wiki/Cook%27s_distance#Detecting_highly_influential_observations. – whuber Feb 02 '12 at 15:02
-
@whuber Thanks. This is always a grey area when performing data exploration for me. Data point 16 above massively influences model outcomes, thus increasing Type I errors. – Platypezid Feb 02 '12 at 15:11
-
3One might argue that it increases "Type III" errors as well, which (generically and informally) are errors related to inapplicability of the underlying probability model. – whuber Feb 02 '12 at 15:22
-
@whuber yes, very true! – Platypezid Feb 02 '12 at 15:31
2 Answers
59
Some texts tell you that points for which Cook's distance is higher than 1 are to be considered as influential. Other texts give you a threshold of $4/N$ or $4/(N - k - 1)$, where $N$ is the number of observations and $k$ the number of explanatory variables. In your case the latter formula should yield a threshold around 0.1 .
John Fox (1), in his booklet on regression diagnostics is rather cautious when it comes to giving numerical thresholds. He advises the use of graphics and to examine in closer details the points with "values of D that are substantially larger than the rest". According to Fox, thresholds should just be used to enhance graphical displays.
In your case the observations 7 and 16 could be considered as influential. Well, I would at least have a closer look at them. The observation 29 is not substantially different from a couple of other observations.
(1) Fox, John. (1991). Regression Diagnostics: An Introduction. Sage Publications.
-
10+1 Clear summary. I would add that influential cases are not usually a problem when their removal from the dataset would leave the parameter estimates essentially unchanged: the ones we worry about are those whose presence really does change the results. – whuber Feb 02 '12 at 15:24
-
1@lejohn Very appreciative of your response. Whuber is right excellent clarity in your answer. This is very informative. Might I suggest you highlight Fox's and your opinions in the wikipedia page! – Platypezid Feb 02 '12 at 15:36
35
+1 to both @lejohn and @whuber. I wanted to expand a little on @whuber's comment. Cook's distance can be contrasted with dfbeta. Cook's distance refers to how far, on average, predicted y-values will move if the observation in question is dropped from the data set. dfbeta refers to how much a parameter estimate changes if the observation in question is dropped from the data set. Note that with $k$ covariates, there will be $k+1$ dfbetas (the intercept, $\beta_0$, and 1 $\beta$ for each covariate). Cook's distance is presumably more important to you if you are doing predictive modeling, whereas dfbeta is more important in explanatory modeling.
There is one other point worth making here. In observational research, it is often difficult to sample uniformly across the predictor space, and you might have just a few points in a given area. Such points can diverge from the rest. Having a few, distinct cases can be discomfiting, but merit considerable thought before being relegated outliers. There may legitimately be an interaction amongst the predictors, or the system may shift to behave differently when predictor values become extreme. In addition, they may be able to help you untangle the effects of colinear predictors. Influential points could be a blessing in disguise.
gung - Reinstate Monica
- 145,122
-
6+1 "Cook's distance is presumably more important to you if you are doing predictive modeling, whereas dfbeta is more important in explanatory modeling": this is very useful advice. – Anne Z. Feb 05 '12 at 00:45
-
Hi - interesting discussion. But couldn't it be rational to integrate a dummy-variable to measure the effect from for example observation 16? – Pantera Feb 08 '12 at 09:04
-
-
Hi - if you remove observations, you should make sure that you have "good" argument to do it, for example that the observation is wrongly measured. If we throw out observation because they just make some statistical trouble, then we're close to data-mining. – Pantera Feb 09 '12 at 13:12
-
1Gung great answer! But can you possibly elaborate on why you think: "Cook's distance is presumably more important to you if you are doing predictive modeling, whereas dfbeta is more important in explanatory modeling."? I thought to do predictive work, having robust coeffiecients is very important, no? – rnorouzian Jul 17 '20 at 18:39
-
Seems reasonable, @rnorouzian, but predictive modeling is about getting predicted values ($\hat{y}$s). So if the coefficients weren't robust, but it didn't have an appreciable impact on the predicted values, it wouldn't matter. In practice, both will have similar information, but will be framed in terms of what you care about. – gung - Reinstate Monica Jul 18 '20 at 11:40
-
@rnorouzian, it isn't really appropriate to have these conversations in comments under an answer. I didn't say "with an appreciable impact", I said without, & I said "if". Note the last sentence of my prior comment. – gung - Reinstate Monica Jul 18 '20 at 16:53