I am using sklearn Random Forest for a regression problem (sales prediction). I had learned that Tree based models are not insensitive to outliers. However after handling the outliers while preprocessing, I observed that Mean Square Error for the model reduced significantly.
So my question is that if Tree based models are able to handle outliers shouldn't the Mean Squre Error remain almost same? Or is Decision Tree able to handle ouliers just for classification problems and not for regression? Or is there something else I am missing?
I am quite new to ML so I don't have much knowledge on the matter. Any input will be helpful. Thanks in advance!
