3

I am having trouble understanding the concept of 'goodness of fit' w.r.t linear regression. The name suggests that the goodness of fit test is used to determine how well a model fits the data.

But the linear model is already the 'best' fit since we have minimized the sum of the squared error terms. Why do we need to test 'goodness of fit' again? We know it is a good fit because we minimized the sum of the squared errors.

Victor
  • 6,565
  • When you say "the goodness of fit test", which goodness of fit test do you mean? – Glen_b Sep 23 '15 at 00:27
  • I was thinking of chi square test. That is the one I have trouble understanding . – Victor Sep 23 '15 at 00:29
  • The mention of goodness of fit in relation to regression does not relate to the chi-square test of counts in a contingency table but to something else (possibly several other somethings, depending on what is being said). Perhaps you can give the context in which you saw it mentioned in relation to regression? – Glen_b Sep 23 '15 at 00:31
  • Thanks. I read it here:http://www.medicine.mcgill.ca/epidemiology/joseph/courses/EPIB-621/fit.pdf – Victor Sep 23 '15 at 00:35
  • 1
    That document you link to mostly seems to do a decent job of explaining many of the disparate meanings of "goodness of fit" in relation to regression. [It does mention the possibility of constructing a kind of chi-squared test of counts as a way of assessing fit, but then dismisses it; I'd dismiss it for several other reasons]. Either way, it gives a whole list of measures/tests/diagnostics and how they tell you about the fit of the model. Indeed now I've seen it, I'd perhaps be inclined to link to that document as an answer to your question. Perhaps it would bear re-reading] – Glen_b Sep 23 '15 at 00:57

2 Answers2

1

You're right in that you've found the line that best fits the data (in the sense of least squares), but the best line may still not be a good fit

enter image description here

In this case I can do better with a third degree curve

enter image description here

Matthew Drury
  • 35,629
  • So can I use the chi square statistic on the regression line and reject the hypothesis that it is a good fit? My confusion is the name 'goodness of fit' since most examples use chi square to check if a observation came from a particular distribution. I don't see how that relates to 'fit' as in curve fitting. – Victor Sep 23 '15 at 00:33
1

Well when you say that 'linear regression is the best fit' and 'we know it's a good fit because it minimizes the sum of squared errors' then you have all the elements you need: OLS will give you estimates that give you the smallest sum of squared errors. But why did you choose to minimize this sum? Why not minimize the sum of absolute deviations in stead of the sum squared errors ?

Moreover, as @Matthew Drury also said, it is not because you have the best result (according to some criterium) that it is a good result? E.g. In economics one tries to maximize the profit of a firm, but it could be that the maximum (best) profit is negative. Would you then run that firm? I wouldn't, so even if you know that this profit is the best you can get, you will still check ' how good it is' (i.e. Whether it is not negative)?

A similar example, if you have to realise an IT project and you know that you get the best project manager, the best analists, the best programmers, and you see that these best people will need two years to realise the project, but the deadline for realisation is in one year, then you have the 'best solution' but it seems to be 'not good enough', so knowing that you have the best solution (best fit, according to some criterium) does not necessarily mean that it is a good solution (therefore you need to check the goodness of the (best) fit).