I've been diving deep into random forests and had a question about terminal nodes.
I know in general when you reach the terminal node, or leaf, of a random forest, the assigned value for that leaf is the mode of the responses that end up there. For example, if you have 5 training examples end up in a terminal (based on the hyperparameters of the model), such that their labels are (A,A,A,B,B), you would set the prediction for that leaf as A.
Is there a reason you wouldn't just have it be a probability distribution? In the above example, you could return A 60% of the time, and B 40% of the time?
My intuition says this would increase the variance of the model, but I'm just looking for some mathematical rigor behind this intuition (or, if this intuition is wrong, an explanation of what I'm not understanding.)