0

I have two output tensors whose MSE is really small (0.04) but on checking the distribution of the tensors they are very different.

Distribution of input tensor

Distribution of output tensor

I am using:

torch.nn.MSE(output_tensor, input_tensor) = 0.04

I am just wondering how can this happen?

Edit 1: Here the tensors are the learned representations of words in sentences. So their shape is [batch_size x num_tokens x embed_dimension], for my case [batch_size x 16 x 512]. I wanted to check if two tensors are similar.

Bishwa Karki
  • 121
  • 1
    Keep in mind that the RMSE is $0.2$, which is about a fifth of the range of your observations. That does not sound especially small. $//$ Histograms are not the most useful plots to make for this assessment. – Dave Nov 08 '23 at 20:49
  • @Dave, the RMSE is 4% which is pretty good on the numbers but the actual results are very worst. Thus I checked the distribution which looks like above.

    I am just wondering if I have wrongly used MSE loss or it is actual problem of the model. But as the MSE is very low, the model is not updated and produces same tensor everytime.

     – Bishwa Karki Nov 08 '23 at 21:49
  • I agree with @Dave . The histogram is not the right tool for the comparison, as explained in the linked post. Also, your MSE, not your RMSE, is 4%. – picky_porpoise Nov 10 '23 at 21:01

0 Answers0