Estimating the Joint and Conditional probability distributions for discrete variables

Question

I read the proof the of the law of large numbers where its states that the the sample mean converges in probability to the population mean and it its proven by Chebyshev's Inequality Here

I am curious if there is something similar in estimating the joint and conditional probability distributions. I found a couple of articles Here1 and Here2, and my question is is there a way to estimate these probabilities for discrete variables without having to go over the data and count the occurrences and divide by the total number of rows

Answer 1

There are many estimators that can be used to estimate a discrete probability distribution with known range. The empirical distribution (simply taking the observed proportions of occurrences of the values) is one reasonable estimator, but there are also others. A broad class of estimators that are useful in this situation are kernel density estimators (KDEs), which can be used for a discrete distribution with known range by choosing a discrete kernel function over that range. Both the empirical distribution and discrete KDEs have good consistency properties for estimation --- i.e., they will converge to the true distribution as the amount of data becomes large. Discrete KDEs typically give a "smoothed" version of the empirical distribution of a sample, which has some desirable properties for certain estimation properties.