Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [autoencoders]

Feedforward neural networks trained to reconstruct their own input. Usually one of the hidden layers is a "bottleneck", leading to encoder->decoder interpretation.

Autoencoders can be applied to unlabeled data to learn features that best represent the variations in the data distribution. This process is known as unsupervised feature learning or representation learning.

Here is an example of a set of filters learned by an autoencoder on a face recognition dataset:

641 questions
9
votes
1 answer

Why use $L_2$ distance as reconstruction error for an autoencoder as opposed to $L_k$

When training an autoencoder on continuous data (that is, not binary), almost all papers/implementations I've seen minimize the $L_2$ reconstruction error between the feature vector $\mathbf{x}$ and the decoded sparse representation…
the higgs broson
  • 545
7
votes
1 answer

How do I do a classification problem with autoencoders (AEs)

I understand that if I want to fine-tune for classification, I can discard the output layer and link the hidden layer to a classifier, do backpropagation to get an improved weight. So when I want to do testing phase, I only need to feed the hidden…
hiyu144
  • 71
5
votes
1 answer

Architecture of autoencoders

Ordinary autoencoder architectures (not variational autoencoders, stacked denoising autoencoders, etc.) seem to only have three layers: the input, the hidden/code, and the output/reconstruction. Are there any examples of papers which used…
liangjy
  • 666
5
votes
1 answer

How much noise for denoising autoencoder

I was looking at the denoising autoencoder architecture. It seems that the way it works is that it takes corrupted input and tries to restore it using the error between the uncorrupted and corrupted input. However, how much noise is optimal for an…
Goodie123
  • 153
  • 4
3
votes
1 answer

Decoding (reconstruction) with an autoencoder

I would like to reconstruct an image by using autoencoder. I have an autoencoder that I implemented by following the UFLDL Stanford tutorial. I extract random 8x8 patches from images (512x512). the patches are then normalized to [0.1, 0.9]. then…
bubo
  • 131
2
votes
0 answers

VAE latent variables apper highly correlated

I am attempting to train a Beta-VAE on historical interest rate curves to map interest rate curves to a lower dimension latent space. I have chosen to use VAE over PCA or autoencoders because I want a regularized latent space to make a generative…
Alex
  • 21
  • 1
2
votes
0 answers

Why is the Marginal Likelihood Estimator proposed in VAE not based on importance sampling?

I'm interested in evaluating $p(x)$ using a variational autoencoder. I would expect the straightforward way to estimate the marginal likelihood to be based on importance sampling: \begin{align} p(x) &= \int_z p(z) p(x|z) \frac{q(z|x)}{q(z|x)} dz \\ …
Lisa
  • 597
2
votes
1 answer

Decoder of variational autoecnoder

From various explanation (e.g. https://www.youtube.com/watch?v=uaaqyVS9-rM), the decoder part of variational autoencoder corresponds to p(x|z) (or p(x_i|z_i)). However, if we fix z (sample a particular z from latent space), then the output is fixed…
Sam
  • 363
2
votes
1 answer

Optimum Weight of Auto Encoders

I'm completely a noob in auto-encoder and deep learning thing. And now I'm trying to implement a simpe auto-encoder. What I want to ask is: If given same dataset, will auto encoder produce the same optimum weight? Thanks for your help
MCRmy
  • 21
1
vote
1 answer

What is the mechanism that makes possible for VAEs to create meaningful concept vectors rather than purely random ones?

I don't know how to put it better, but if we have n-dimensional data and we want to compress it to m-dimensional space where m<
Krushe
  • 51
1
vote
1 answer

How does VAE generate the value of mean and sd for the latent space?

I am just trying to understand the detailed mechanisms of VAE (variational autoencoder), and even new to Deep Learning. I was wondering, how does the encoder part of the VAE can make sure to generate the value of Gaussian mean and sd? I checked the…
lesdormis
  • 13
1
vote
1 answer

Sparse coding vs. Autoencoder

I was reading the previous Q&As about sparse coding and sparse autoencoder differences but I am still confused what is the point in using sparse coding. It is said that sparse autoencoder give us a way to model the sparse codes (and thus generate…
Shahriar49
  • 321
0
votes
0 answers

Multivariate Variational Autoencoder and Positive Definite Covariance Matrix

This might be a naive question from a non-statistician but here we go. I understand that the challenges that hamper the use multivariate variational encoder where a covariance matrix is used instead of the a vector of standard deviation revolves…
applied_env
  • 1
  • 2
0
votes
1 answer

Activation function selection for autoencoders

I am running an autoencoder model and have to select my activation function. I have structured data (i.e. not an image) which comprises continuous variables which have a range of -4 to +4 for each continuous variable. Which activation function…
chipsin
  • 117
0
votes
0 answers

Do the encoder and decoder architectures of VAE need to be exactly reverse of each other in terms of size of data at each layer?

For example if we have the encoder architecture as follows: (number of channels@ h_size x w_size) 1@28x28 | 32@14x14 | 64@7x7 | 128@4x4 | 256@2x2 | 512@1x1 Should the decoder architecture be such that we get the following: 512@1x1 | 256@2x2…
Curious
  • 421
1
2