From what I've learned about autoencoder is it takes an input and predicts an output almost similar to the input. So, if it outputs the same thing with the same dimensions, what is the benefit of using autoencoder then? We can directly work with the inputs.
Asked
Active
Viewed 278 times
1
-
three terms are you answer: Unsupervised Representation Learning – user3639557 Apr 06 '20 at 12:18
1 Answers
1
You're correct that an auto-encoder outputs the same dimension as the input, but it goes through a smaller hidden layer. Imagine a series of layers, from input to output, each with the following number of neurons:
1000 : 500 : 100 : 500 : 1000
The 1000-dimensional input is squeezed through a 100-dimensional middle layer. Once trained, if you remove the last two layers, and only use:
1000 : 500 : 100
the 100-dimensional middle is a reduced dimensional representation, pure an encoding. You could use that as input to another learning method - another neural network or something else.
Think of it like PCA in which you only use the first few PCs. Principal components regression works in this way, but uses a linear projection of the data to lower dimensions, whereas the auto-encoder uses a non-linear protection.
KirkD_CO
- 1,138
-
-
Is it useful only for the training data? I mean if we take the 100-dimensional middle of an instance outside training would the latent representation be good enough? The decoder didn't see this instance during training so we can say that the latent representation would be good but the decoding part (when reconstructing it) would be bad? – Anton May 22 '22 at 09:58
-
If the new instance came from the same distribution(s) as the original data, it should be fine. Conceptually, using an auto-encoder would be similar to normalizing the training data and then using the means and standard deviations from that to normalize the test data. You would not want to retrain it on the test data. – KirkD_CO May 22 '22 at 13:05