2

I am working on a regression task where the target variable cannot be strictly negative. To do the predictions, I am using the LGBM framework (Python) with the RMSE loss. Issue that I am facing is that some of the predictions are negative. I understand that I could apply a log-transformation to the target variable before training the model and making predictions, and then revert to the original one by applying the exponential function.

But I also read that it could be possible to write a custom loss function to deal with this. So my question is what kind of loss function could do such a role ? Can a loss function prevent by itself the prediction of negative values ?

Note: more generally, if you have knowledge/resources which deals with strategies to prevent negative predictions in regression problems (preferably with GBM) I would be glad if you can share them.

yoyoog
  • 307
  • If your outcomes are constrained, then Ordinary Least Squares (which I assume you are using) is likely not the best tool. Better to use something like gamma regression for continuous outcomes, and negative binomial regression for discrete ones. In each case, you maximize the likelihood. Also, if you work on a log scale and back-transform using the exponential only, the result will be biased, you need to adjust for variance. – Stephan Kolassa Sep 18 '20 at 04:58
  • @Stephan Kolassa Thank you for your answer, I will take a look at the Gamma distribution as my task deals with continuous values. It might be a beginner's question, but how should one know what kind loss function is / distribution is the best suitable for one's problem ? by looking at target variable distribution is enough ? – yoyoog Sep 18 '20 at 06:07
  • Perhaps, you can use an indicator function here. You can define an asymmetric loss where you penalize negative part more heavily. Something like $I(x < 0) Penalty1 + I(x > 0) Penalty2$? – user3119750 Sep 18 '20 at 07:09
  • 1
    There is a bit of an art to choosing the right distributional assumptions. Often you need to think about the theory of your problem, possibly look at people who modeled it before. – Stephan Kolassa Sep 18 '20 at 08:43

0 Answers0