1

When 1 is added to word count in Laplace Smoothing in Naive Bayes, the new probabilities either increase or decrease as shown below. Though the problem of "zero" probability has been solved.
Does the smoothing represents the true picture of probability?
Denominator of Old P(w/c) is 15.
Denominator of New P(w/c) is 21.

Word (w) Count (word) Old P(w/c) Count + 1 New P(w/c) New P(w/c) - Old P(w/c)
w0 0 0.000 1 0.048 0.048
w1 1 0.067 2 0.095 0.029
w2 2 0.133 3 0.143 0.010
w3 3 0.200 4 0.190 -0.010
w4 4 0.267 5 0.238 -0.029
w5 5 0.333 6 0.286 -0.048
Arpit Saxena
  • 11
  • 2

1 Answers1

0

Does the smoothing represents the true picture of probability?

You are using a Bayesian estimator, that besides data considers a prior, so the estimate is biased. In this sense, it is not “true”. But on another hand, are the observed zero counts true zeros? Are those values really impossible? Likely not, so your empirical probabilities were not “true” as well. Laplace smoothing is a trick for overcoming a specific problem, we accept the bias because helps us to solve the bigger problem of dealing with zeros.

Tim
  • 138,066