1

I understand that the best way to test for non-linearity is to look at the residual plots. However, I have 20,000 or more points and any pattern in the residuals is not easy to spot. Are there approaches other than eyeing the residuals to test for non-linearity in regression. Note that I work in an area where there is little statistical analysis, so there is no theory to build on.

I asked something like this before, but the answer given, to use the residuals won't really work given the shear number of points. So I am looking for the best test, or a way to change the residuals to address this issue. Is there formal test of non-linearity in linear regression?

kjetil b halvorsen
  • 77,844
user54285
  • 1,051
  • 1
    One usually employs robust smooths of the residuals as a start. With such tools it's possible to identify patterns in millions of residuals. It sounds like you don't need or want a formal test: you need methods of exploratory data analysis. There is a lot of literature on that, starting with the methods shown in regression textbooks. – whuber Nov 12 '20 at 20:57
  • thanks whuber. I am not familiar with the robust smooths you mention. Can you suggest a source for this. My primary concern, given a total lack of theory, is to show what the impact of various variables are. My concern with non-linearity (given that I have the whole population) is how it changes the coefficients. Because those I work for are very concerned about which variables are important. I have limited expertise in non-linear data and most that I have looked at don't generate coefficients (I have problems understanding if a loess for example indicates a variable is important or not). – user54285 Nov 12 '20 at 21:04
  • 2
    This is such a huge set of issues to deal with I couldn't even begin addressing them in the small space allowed for comments. Search our site for upvoted posts on Exploratory Data Analysis, regression diagnostics, residuals, and model selection, then go on from there. – whuber Nov 12 '20 at 21:06
  • 1
    thanks very much. I actually have done this a lot, non-linearity continues to cause problems for me. – user54285 Nov 12 '20 at 21:46

0 Answers0