Confusion about the use of the MLE & the posterior in parameter estimation for logistic regression

Question

In classification one usually computes $$ C = \operatorname*{argmax}_k p(C=k\mid X) $$ where $p(C=k\mid X)$ is the posterior distribution.

In a simple logistic regression setting with $C \in \{0, 1\}$ and $$ p(C=1\mid X)=\frac{\exp(\beta_0+\beta_1 x_i)}{1+\exp(\beta_0+\beta_1 x_i)} $$ and therefore $$ p(C=0\mid X)=\frac{1}{1+\exp(\beta_0+\beta_1 x_i)} $$ with $X=\{x_i\},\ i=1,\ldots,N$.

we estimate the parameters $\beta_0, \beta_1$ via the maximum likelihood estimation. To do so one has to compute the product of the likelihood function of all $N$ observations. So far, so normal. However, in all text books the authors plug in the posterior instead of the likelihood (e.g., Bishop, p. 206, Hastie, et al., p. 120): \begin{align} \ell(\beta) &= \log\left(\prod_{i=1}^N p(C_i=k\mid x_i, \beta)\right) \\[8pt] &= \log\left(\prod_{i=1}^N p(C_i=1\mid x_i, \beta)^{C_i}(1-p(C_i=1\mid x_i, \beta))^{1-C_i}\right) \end{align} And even though these probabilities are now conditioned on $\beta$ as well, they are still no likelihood to the posterior $p(C=k\mid X)$. So

1. How come we plug in the MLE just the posterior conditioned on the parameter $\beta$?

2. Why is $p(C=k\mid X)$ a posterior anyway? To me a posterior is a distribution of over a parameter given the observed data. But the class $C$ is to me not a parameter but a target just like the observations $y_i$ in a linear regression setting.

Answer 1

Bit late, but this too gave me headache... Maybe we could argue as follows (two class case). Start with the usual expression for the log-ML, being product of of joint distributions as function of distribution parameters.

\begin{align} \ell(\beta) &= \log\prod_{i=1}^N p(C=1, x_i\mid\beta)^{t_i} \cdot p(C=0, x_i\mid\beta)^{1-t_i} \\[8pt] &= \log\prod_{i=1}^N \left( p(C=1\mid x_i,\beta)~p(x_i)\right)^{t_i} \cdot \left(p(C=0\mid x_i,\beta)~p(x_i)\right)^{1-t_i} \\[8pt] &= \log\prod_{i=1}^N p(C=1\mid x_i,\beta)^{t_i} \cdot p(C=0\mid x_i,\beta)^{1-t_i}~~ p(x_i) \\[8pt] &= \sum_{i=1}^N t_i\cdot \log (p(C=1\mid x_i,\beta))+ (1-t_i)\cdot \log(p(C=0\mid x_i,\beta))+\log(p(x_i)) \\[8pt] \end{align}

Now, as you minimize the expression (where the sigmoid is used as ansatz for the posterior $p(C\mid x)$) with respect to $\beta$ the marginal distribution $p(x_i)$ is dropped and the same minimum will be found independent if you start with the posterior probabilities $p(C\mid x)$ or joint probabilities $p(C,x)$ in the expression for the likelihood. I think this is equivalent to the fact that bigger $p(C\mid x)$ implies bigger $p(C,x)$ for a class.

EDIT: I opened a new post for proposing a consistent problem description/suggested solution. MLE for logistic regression, formal derivation