8

I often see neural networks extended to complex-values. Those networks allow complex input, complex parameters, and complex output. My understanding is that the inner products and point nonlinearities are simply extended. I can see it's advantage for signals that are naturally complex (e.g., phase/amplitude decomposition, oscillations, frequency domain processing, ...). However, computationally speaking, is it any different from converting complex signals to real values?

  1. What is the computational advantage of using the complex field compared to the real field?
  2. What is the computational advantage of complex vs real vector space?

References:

  1. Akira Hirose. Complex-Valued Neural Networks. Springer Science & Business Media. 2006, 2012
  2. Danilo P. Mandic, Vanessa Su Lee Goh.Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models. 2009
  3. Tygert, M., Bruna, J., Chintala, S., LeCun, Y., Piantino, S., & Szlam, A. (2016). A Mathematical Motivation for Complex-Valued Convolutional Networks. Neural Computation, 28(5), 815–825.
Memming
  • 870
  • 1
  • 8
  • 19

3 Answers3

1

I don't really see how a complex valued Neural Network would provide anything particularly useful over a real valued Neural Network.

The whole idea of having a Neural Network that operates on complex numbers, uses complex weights, and outputs complex numbers doesn't seem any different than having a real valued Neural Network that has two times as many inputs, outputs, and weights as the complex Neural Network.

For example, if you want a network that takes some input complex number and outputs another complex number, you could represent this with a real valued Neural Network with two inputs and two outputs.

spektr
  • 4,238
  • 1
  • 18
  • 19
1

I also found that book written by Akira Hirose in the library. I have the same question as well, and I found this paper: https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2011-42.pdf

Anyway, we are not even in the "real" world. The input spaces of any neural networks implemented on modern computers are not just countable, but finite. Also the cardinality of real numbers is the same as complex numbers. So I think for the same structure NN, the real-valued NN can solve all the questions that can be solved by complex-valued NN.

Atri Robot
  • 11
  • 1
0

From a strict programming point of view, a complex number is composed of a real part and an imaginary part, that is to say, two real values. So I guess that real arithmetic applies here, except for certain operations like sqrt for a negative number for instance.

A complex number needs twice the storage of a real, because as said before, it is actually two reals. So I assume that a complex-valued neural network will require twice the storage of a real-valued neural network.

Again, since two reals are involved, it is possible that a complex-valued neural network will require twice the arithmetic operations compared to a real-valued neural network, but it may depend on the language, the compiler, some specialists in StackOverflow may give you more details.

Coriolis
  • 629
  • 4
  • 10