Conditional entropy $H(X|Y)$ tells us how much the average uncertainty about a channel input $X$ is after observing channel output $Y$, and mutual information $I(X,Y)$ measures how much information about channel input $X$ can be obtained observing channel output $Y$.
Doesn't this mean that $H(X|Y)$ and $I(X,Y)$ complement each other or tell us the same thing? If so, then since mutual information is symmetric $I(X,Y)=I(Y,X)$, shouldn't conditional entropy be symmetric as well?
No,because while mutual information tells you how much uncertainty can a random variable $X$ remove from another random variable $Y$, the conditional entropy of $X$ given $Y$ tells you how much uncertainty remained in X after using the information (given by $I(X;Y)$) that $Y$ gave it.
$H(X|Y)=H(X)-I(X;Y)$
That's why the conditional entropy depends on the value of the entropy before the observation and the mutual information isn't, because it is only the difference ($\delta$) between two entropy states, before and after the observation.
There's a cool Venn diagram from here. This shows clearly that $I(X:Y)$ is independent of the order of $X$ and $Y$.
