0

While training a linear regression model, it is advised to standardize the input features using the mean and std deviation of the train input features.

x = (x - x_mean)/ x_std_dev

However, when we want to infer the model on some examples, do we use the same mean and std deviation used during training or the mean and std deviation of the batch of test examples we feed the model? Which one is better and why?

x = (x - x_mean_train) / x_std_dev_train

vs

x = (x - x_mean_test_batch) / x_std_dev_batch
gitartha
  • 1
  • Who advises this? If you want standardized coefficients, you can standardize your predictors, but this is absolutely not something you should do "by default". – Stephan Kolassa Mar 19 '23 at 16:11
  • @StephanKolassa I agree this is not something we should do "by default", however when we are doing it, what's the best way to do. I reckon we should go by the train dataset estimates as we should apply the same transformations during training and testing. Still not sure. – gitartha Mar 19 '23 at 16:42
  • @gitartha Your intuition is a good one that you should standardize according to the training set, not the testing set. However, I think Stephan refers to a lack of necessity to standardize variables. (It’s kind of necessary for deep learning, but that is really because of computational issues and difficulty of optimization convergence when you do not standardize.) – Dave Mar 19 '23 at 17:34
  • @Dave okk got it, thanks for the explaination – gitartha Mar 19 '23 at 18:08

0 Answers0