Decision tree problem

Question

I stumble upon this question. We are solving a classification problem using a Decision Tree. When increasing the depth of the tree the accuracy improves until it stagnates at 90%. Why is this an unexpected behavior? Can you offer a plausible explanation?

I think we re are suffering from underfitting. Am I correct in my thought? Many thanks in advance.

Per the question, we keep on increasing the depth of the tree. So can this be underfitting? — Stephan Kolassa, Jan 11 '23 at 15:32
@StephanKolassa. Therefore you think it is a straight overfitting? — zum77, Jan 11 '23 at 15:34
No, it's not overfitting, either. Consider an extremely simple example. There is only a single predictor with a constant value of 42, and we have 10 observations, 9 of class A and one of class B. What will your trees look like? What accuracy will you get? — Stephan Kolassa, Jan 11 '23 at 15:39
Also, what does this tell you about accuracy as an evaluation metric? — Stephan Kolassa, Jan 11 '23 at 15:40
@StephanKolassa. So this is not unexpected behavior. As per your view model does not suffer neither underfitting nor overfitting? — zum77, Jan 11 '23 at 15:42
Exactly. I would definitely not consider this unexpected. Can you formulate what causes accuracy to stagnate? — Stephan Kolassa, Jan 11 '23 at 15:45
@StephanKolassa. I cannot formulate what causes accuracy to stagnate. Can you help with plausible explanation. — zum77, Jan 11 '23 at 15:48
Accuracy can stagnate because there are multiple observations with identical predictor values but different class memberships. A decision tree (and any other "hard" classifier) will assign the same fitted class to all such instances (because it could only assign different classes if the predictors differ, and they don't). It would be better to output a probabilistic classification and evaluate this using proper scoring rules, see the link in my earlier comment. — Stephan Kolassa, Jan 11 '23 at 15:51
@StephanKolassa. Many thanks. Therefore all hard classifier more or less are going to suffer with such kind of issue. We need a proper scoring rules in order to evaluate them. — zum77, Jan 11 '23 at 15:54
That is exactly my position. Note that this is not unanimously accepted, especially not so among the ML community (see, e.g., here), so if this is in the context of a formal course, be prepared that your instructor might see things differently. — Stephan Kolassa, Jan 11 '23 at 15:58

Decision tree problem

0 Answers0