4

I have a very interesting question. It's one month ago Geoffrey Hinton release his CapsNet paper. It's a complete new way for visual computing in the neural network stack. But my question is. Is it possible to use the CapsNet as a GAN. I only understand the concept of the CapsNet but not technical limitations. So could this be possible?

This sounds for me for a nice field to research:)

Christian Frei
  • 351
  • 1
  • 5
  • 2
    I was wondering this too - in principle the capsules should help force consistency between sub-components of an image and avoid some of the weird monstrosities that CNN-based GANs can produce. However, I don't know whether it is even possible to build the generator component using capsules architecture. A CNN-based generator and CapsNet-based discriminator might be more feasible in the near future – Neil Slater Nov 23 '17 at 12:05
  • I am expecting this too. That should worth a sci paper. – Origond Jan 04 '18 at 09:25
  • Here you can find the papers, several articles and tutorials about CapsNet as well as current implementations in different programming languages and frameworks. – loretoparisi Dec 04 '17 at 23:09

2 Answers2

2

It is definitely possible to use them in the Encoder (Discriminator) part of a GAN, since according to Hinton's original paper it's used for classification which means it fits in Discriminators as well even though the proper loss function is a subject to research.

I chose a topic of Capsule Nets in Discriminators of GANs for my Master thesis and I'm working on the model right now. However, I simplified the routing algorithm by not vectorizing the DigitCaps values (otherwise Keras code gets complicated, although I'm not sure if what I've done is 100% correct).

Stephen Rauch
  • 1,783
  • 11
  • 22
  • 34
Guseyn Gadirov
  • 21
  • 2
0

Yes. You can use a CapsuleNet as the discriminator in a GAN. See the following recent paper, which claims this improves over using CNNs as the discriminator:

CapsuleGAN: Generative Adversarial Capsule Network. Ayush Jaiswal, Wael AbdAlmageed, Yue Wu, Premkumar Natarajan. arXiv:1802.06167

The improvement appears to be modest on the tasks they tested. It's not clear how to use a CapsuleNet as the generator, so the paper uses a CNN-based generator network.

D.W.
  • 3,341
  • 17
  • 42