2

I have a set of 2 - 10 explanatory variables which I'd like to use to predict the response variable, distance. The explanatory variables describe the flight of a projectile (velocity, spin, angle) and the conditions of the air it is flying through (temperature, altitude, etc.). I'd like to end up with an interpretable model that makes the best predictions possible, though I know these don't always go hand-in-hand. For example, I want to know much an increase/decrease in sidespin affects distance.

Now, this almost sounds like an ideal problem for linear regression, but the issue is the relationship between distance and angle, for example, is not linear, as if the ball is launched at too high of an angle (say 90 degrees straight up), obviously the distance will be small. In fact, I've tried a linear model already in which I square the angle term (and include some interaction terms), but the predictions are worse than I would like. Furthermore, since linear regression outputs a "line" in multi-dimensional space, it makes some negative predictions, and taking log() still doesn't help the predictions all that much.

Now, I talked to some people and we thought a neural network might be a good choice, since neural nets are very good approximators of functions, and I believe there is some underlying non-linear function at play here. Some of the neural nets I've tried so far give much better predictions than the linear model, however it obviously isn't easy to interpret the main effects.

Does anyone have any thoughts as for the overall approach here? Could it make sense to use a linear model to interpret the influence of the main effects on distance while then using a neural net to make predictions?

mkt
  • 18,245
  • 11
  • 73
  • 172
Jake
  • 199
  • 7
  • Why don't you let physics suggest suitable models? Much is known about this situation, so much so that interest ought to focus on analyzing the residuals from the predictions of a physical model. – whuber Jul 01 '22 at 19:25
  • @whuber What kind of residual analysis are you referring to? – Jake Jul 01 '22 at 19:49
  • The differences between the data and what any model predicts (or fits) are the residuals. If the physical model is any good at all, it will capture the mechanically predictable part of the system, leaving only (likely much smaller) differences due to unknown factors to model statistically. When this works out well, the residuals can be revealing. For instance, a simple analysis of mercury vapor pressure data (a macroscopic variable) is capable of yielding estimates of quantum mechanical forces at atomic scales: see https://stats.stackexchange.com/a/35717/919. – whuber Jul 01 '22 at 20:15

1 Answers1

1

If you can, you would benefit from defining your models based on physical principles, to properly capture the nonlinear relationships you are interested in.

If that is not possible for all predictors, a simple alternative would be to use s (GAMs), which are linear models that allow you to model responses as smooth, nonlinear functions of predictors. So distance can be made a smooth function of angle (and any other predictors), without specifying the functional form. Other predictors can be constrained to follow linear relationships, as in a linear regression.

You can also use neural networks or other machine learning methods, and then generate partial dependence plots to visualise the nonlinear response to specific predictors. This is a key trick in what is called 'interpretable machine learning', for which I recommend this free online book by Christoph Molnar.

mkt
  • 18,245
  • 11
  • 73
  • 172
  • Thanks. Obviously this does sound like a physics problem--the issue is I am a statistician, not a physicist. I may look into some physical models, otherwise GAMs sound interesting as do the partial dependence plots for neural nets. – Jake Jul 01 '22 at 19:47
  • 1
    @Jake Did this address your question? – mkt Jul 06 '22 at 11:23
  • 1
    Yes- in particular it looks like GAMs might be quite useful to me. Thanks for your help, and I've marked your answer as the solution! – Jake Jul 06 '22 at 15:14