1

While reading the paper "Backpropagation Applied to Handwritten Zip Code Recognition", one of the original papers attributed to the development of convolutional neural networks, on page 3, section 3.1 the authors write (emphasis mine):

The remainder of the recognition is entirely performed by a multilayer network. All of the connections in the network are adaptive, although heavily constrained, and are trained using backpropagation. This is in contrast with earlier work (Denker et al. 1989) where the first few layers of connections were hand-chosen constants implemented on a neural-network chip. The input of the network is a 16 by 16 normalized image. The output is composed of 10 units (one per class) and uses place coding.

What does place coding mean? A Google search of this term yielded nothing conclusive. Any insights appreciated.

Itamar Mushkin
  • 1,061
  • 4
  • 17
IntegrateThis
  • 119
  • 12

1 Answers1

2

"Place coding" == "one-hot encoding" This is the conventional way of encoding a category at the output of a classifier: for N categories, use an N-dimensional vector where the k-th component is "turned on" and all other turned off to encode the category k. In general, the values for "on" and "off" do not need to be 1 and 0 (or +1 and -1). They could be anything.

The alternatives to "place code" are "coarse coding" and "distributed representation" of which "error-correcting codes" are a special case.

The classifier in the paper uses a form of distributed representation in which similar characters (like o, O, and 0) are represented by similar bit patterns. That way, when the system is not sure if the character is o, O, or 0, it will produce an output bit pattern that still identifies the character as one of those 3 categories, and leave it up to the post-processing (e.g. a language model) to resolve the ambiguity.

Yann LeCun
  • 36
  • 1