When performing a linear multiple regression $ Y = X_1 + X_2 $ is it necessary that $ X_1 $ and $ X_2 $ provide the same scale?
When I think of the variables being represented by an orthogonal axis, it shouldn't matter, or ?
Asked
Active
Viewed 213 times
5
-
With regularisation, it is necessary in most cases. – gunes Jul 23 '20 at 09:25
-
So without it isn't? Regluarisation is only necessary when there are much more variables, right? – Ben Jul 23 '20 at 09:47
-
1related: https://stats.stackexchange.com/questions/208341/scaling-in-linear-regression – Christoph Hanck Aug 25 '20 at 14:50
1 Answers
4
It is sometimes useful, but never (IMO) necessary. A linear rescaling of your predictors amounts precisely to changing the labels on their axes, in a linear way.
If, for instance, you rescale a predictor such that it has a standard deviation of 1, then you can interpret the corresponding regression coefficient estimate as the change in the dependent variable associated with a 1 SD change in the predictor. This may or may not be more enlightening than leaving your predictors as they are and then interpreting the estimates as the change in the DV associated to a change of 1lb or 1 horsepower.
Stephan Kolassa
- 123,354
-
(+1) other than the regularised case, I was trying to think of a necessary situation for standardisation, but couldn't come up with a solid reason. – gunes Jul 23 '20 at 09:30
-
When there is only on variable, scaling doesn't matter, or? Only in regards to regression, not to mention any functional relationship or so. – Ben Jul 23 '20 at 10:03
-
"Doesn't matter" depends on your semantics. Scaling even one predictor will of course change your coefficient estimates (though not $t$ and $p$ value), so in this sense, it does matter. But the exact same thing also holds for scaling multiple predictors. – Stephan Kolassa Jul 23 '20 at 11:03