0

Wanted to make sure that I have the proper steps for building a Regression Decision Tree correct. Please let me know if I'm missing anything or if something looks incorrect:

  1. Build a fully grown tree:
  • Divide the predictor space (all X’s) into distinct and non overlapping regions. Where the regions is split is determined by the maximum decrease in RSS. RSS measured by (actual – predicted), and predicted is always the Mean of the train datapoints in whatever region.
  • Repeat until a stopping criteria. Eg. N<5 in each region.
  1. Set up the tree to be pruned to prevent overfitting.
  • Vary Alpha (basically Lasso’s Lambda but for Trees) to create subtrees with different numbers of terminal leaves.
  • Do K fold CV to pick the right Alpha. Tree size with the Lowest MSE wins.
Katsu
  • 911
  • This question is very broad, and I believe you would profit from reading an introductory level textbook. We have a helpful list of free statistical textbooks.. If afterwards you still have more specific questions, then please do ask them here. If you already have read such a textbook, please edit your question to make it more specific. Thank you! – kjetil b halvorsen Jan 20 '23 at 22:18
  • As referenced in many of my questions, I am actually currently reading Introduction to Statistical Learning (2nd edition). I'm just looking to broadly confirm my understanding of how a regression tree is constructed, not have someone type out in great detail each step of the regression tree. Like if im missing a major step in the above outline, or if one of the steps is completely wrong. If not, an answer of "confirmed, that sounds right to me" would be more than sufficient. – Katsu Jan 22 '23 at 01:02

0 Answers0