1

I am building a boosted decision trees classification model, where the input variables vary smoothly with time.

The problem is that the predictions will always be biased by the most recent entries. I understand that this is happening because these are the points of hyperspace closest to the target points. If I don't use the n last entries, then the predictions are still biased by the last used entries.

How can I avoid this bias and "shake things up"? I expect that the correct model will pick up relations between variables and use them, not blindly looking for the nearest hyperpoints.

The solution could be to use another ML method, but some quick tries show that the problem isn't easy to get rid of, in general. So, any suggestions, either within BDT or with another algorithm, are appreciated.

Helen
  • 309
  • 1
    Your input data is one/ several time series. You should look on how to make your data stationary. You should add time as a feature (lag variable) and change your training/validation approach to predict within a window. – Ciodar Apr 17 '23 at 07:46
  • @Ciodar thanks for your comment ~ a question: how would predicting within a window solve this issue? – Helen Apr 20 '23 at 05:01
  • After having a second thought , it may not be needed to predict within a window, but you need for sure to take account of the time. If your input does vary with time (e.g it increases) and you do nothing, the decision boundary of the tree will soon become not correct, with all your data points ending up in the higher region for future data. If your input varies with a seasonal component, you also need to take that into account. – Ciodar Apr 21 '23 at 11:35

1 Answers1

1

A couple of things that you could do:

  • Use a rolling window approach, limiting the training data to a fixed window of the most recent data points;
  • Add input variavbles that capture temporal relationships between your input variables and the target variable. Lagged variables could capture the impact of past values on the current value, or use rolling averages to smooth out the impact of recent data.
omnp
  • 11