6

I am reading this paper about Group Equivariant Convolutional Networks. Basically, it is a CNN whose construction makes the network naturally equivariant to Group transformations (e.g. rotations) of the input.

This is, a GE-CNN trained with the Rotations Group in its architecture, will for instance predict correctly the label of a rotated MNIST digit, even though it was never trained on rotated MNIST digits.

My question here is, are there any situations in which these GE-CNN will not perform better than a regular CNN? In other words: is the property of being equivariant to any transformation of the input always desirable?

puradrogasincortar
  • 199
  • 6

1 Answers1

10

No, it will very much depend on the specific domain that you're applying the model to, whether or not this type of prior will be good or bad. The same applies to any other prior structure/knowledge you could encode in networks.

For example, the basic CNNs implement the prior assumption of shift invariance. Intuitively, you may think of this as saying "if I can learn to recognise a cat (or anything else) in one location of an image, I can generalise from this and immediately also recognise cats in different locations of images". This tends to work well, because it is true: cats do look like cats, regardless of where they are in an image.

The same reasoning applies to the prior assumption of rotation invariance (or equivariance): whether it is helpful or harmful depends on the domain. I would actually argue that the very same MNIST digit recognition problem that you mention as an example in your question is a problem where you do not want rotation invariance. Just think of the digits 6 and 9: if you rotate one of those upside down, it looks identical to the other. So, a neural network with rotation invariance will never be able to distinguish between sixes and nines!

Dennis Soemers
  • 10,264
  • 2
  • 26
  • 69
  • But your example is not correct. Even though intuitively a rotated 6 looks like a 9, the reality is that there are nuances in the human left-to-right handwriting than can make a 90º rotated 6 still be recognised as a 6, even though "it looks like a 9". Indeed, in the Rotated-MNIST classification task, rotation-equivariant networks obtain state of the art results, and can recognise 90º rotated 6s as actual 6s. In other words, rotation equivariance is desirable here. – puradrogasincortar Mar 09 '23 at 17:01
  • This is exactly why I am asking this question: no matter how do you rotate any object, it still is that object; even for 6s and 9s which could appear as the right example. I am trying to find such an example in which for instance, some rotations of the input would change their class, so that those specific rotations are not desirable in the network. – puradrogasincortar Mar 09 '23 at 17:06
  • 2
    Ok, sure, maybe there would still be subtle differences in human handwriting. But then... just imagine exactly the same problem, but with photographs of printed documents (typed on a computer or something) and recognising the text/digits there. In this case the rotated versions of 6s and 9s would (maybe depending on font? not sure) identical under rotation. – Dennis Soemers Mar 09 '23 at 17:40
  • True! That's a good example. The spoiler here is that equivariance to partial rotations is indeed usually better than equivariance to the complete rotations group (see "Learning Partial Equivariances from Data", Romero et al). An example of this is CIFAR10/100. However, I wanted someone to pin me a simpler example, as I can't really reason why CIFAR10 is solved better with partial equivariance (once again, a rotated object in CIFAR10 would still be that object, so why would partial equivariance perform better). – puradrogasincortar Mar 10 '23 at 09:57
  • 2
    @puradrogasincortar If I recall correctly, CIFAR10/100 is probably only/mostly real-life photographs? I imagine there'll be a strong bias, where photographs are maybe not always taken at a perfect straight angle, but usually at least with roughly the same orientation. A blue area near the top of the image is probably usually sky, and blue area near the bottom is probably usually sea. There are probably no/very few upside-down rotated photographs where the sea is at the top and the sky at the bottom. – Dennis Soemers Mar 10 '23 at 10:26