11

I'm interested in knowing whether there exist any neural network, that solves (with >=80% accuracy) any nontrivial problem, that uses very few nodes (where 20 nodes is not a hard limit). I want to develop an intuition on sizes of neural networks.

nbro
  • 40,472
  • 12
  • 105
  • 192
Guillermo Mosse
  • 327
  • 1
  • 10

1 Answers1

13

Even if it’s impossible to answer this question properly, as non trivial is not well defined (maybe the author will edit this questions later, to specify it better), I take the opportunity to point out this paper which looks interesting to me

Smallest Neural Network to Learn the Ising Criticality

Assuming you have a general idea of the Ising Model I think the problem of identifying the critical temperature from a data driven perspective can be considered as non trivial and the paper shows how the authors have improved the performance related to solve this task with NN passing from 100 Hidden Neurons, as performed in this paper Machine learning phases of matter from 2017, to only 2 Hidden Neurons

Just my cents:

  • reducing the neurons, while keeping good performance, should help in terms of neural processing interpretability which is notoriously obscure and its complexity grows (exponentially) with the number of neurons
Nicola Bernini
  • 809
  • 6
  • 12
  • 3
    I just wanted examples that the community itself found interesting. I think it is actually non trivial to define non triviality. Right? – Guillermo Mosse Mar 01 '19 at 20:12
  • 1
    @GuillermoMosse No, it's quite trivial to define non-trivial. The definition is more or less arbitrary depending on what best fits the situation. I'm sure a very simple NN could learn to tell whether or not a given number is a non-trivial square root... – forest Mar 02 '19 at 03:24