How is softmax_cross_entropy_with_logits different from softmax_cross_entropy_with_logits_v2?

Question

Specifically, I suppose I wonder about this statement:

Future major versions of TensorFlow will allow gradients to flow into the labels input on backprop by default.

Which is shown when I use tf.nn.softmax_cross_entropy_with_logits. In the same message it urges me to have a look at tf.nn.softmax_cross_entropy_with_logits_v2. I looked through the documentation but it only states that for tf.nn.softmax_cross_entropy_with_logits_v2:

Backpropagation will happen into both logits and labels. To disallow backpropagation into labels, pass label tensors through a stop_gradients before feeding it to this function.

as opposed to, tf.nn.softmax_cross_entropy_with_logits's:

Backpropagation will happen only into logits.

Being very new to the subject (I'm trying to make my way through some basic tutorials) those statements are not very clear. I have a shallow understanding of backpropagation but what does the previous statement actually mean? How are backpropagation and the labels connected? And how does this change how I work with tf.nn.softmax_cross_entropy_with_logits_v2 as opposed to the original?

Answer 1

You have every reason to be confused, because in supervised learning one doesn't need to backpropagate to labels. They are considered fixed ground truth and only the weights need to be adjusted to match them.

But in some cases, the labels themselves may come from a differentiable source, another network. One example might be adversarial learning. In this case, both networks might benefit from the error signal. That's the reason why tf.nn.softmax_cross_entropy_with_logits_v2 was introduced. Note that when the labels are the placeholders (which is also typical), there is no difference if the gradient through flows or not, because there are no variables to apply gradient to.