Feb 12, 2015 · We introduce a simple modification for autoencoder neural networks that yields powerful generative models. Our method masks the autoencoder's parameters to respect autoregressive constraints: each input is reconstructed only from previous inputs in a …

Oct 25, 2019 · Germain, Gregor & Larochelle $^{[2]}$, posted their findings in the paper MADE: Masked Autoencoder for Density Estimation. In my opion, they made a really elegant observation that, by the definition of the chain rule of probability, …

Sep 10, 2020 · MADE is a straightforward yet efficient approach to estimate probability distribution from a single pass through an autoencoder. It is not capable of generating comparably good images as that of state-of-the-art techniques (GANs), but it has built a very strong base for tractable density estimation models such as PixelRNN/PixelCNN and Wavenet.

MADE: Masked Autoencoder for Distribution Estimation. Paper on arXiv and at ICML2015. This repository is for the original Theano implementation. If you are looking for a PyTorch implementation, thanks to Andrej Karpathy, you can fine one here.

Jul 28, 2020 · This property is formally referred to as “autoregression” (dependence on itself), and is implemented in MADE by introducing masks for the weights of the neural network that is used to estimate the distribution of the variable’s element.

We introduce a simple modification for autoencoder neural networks that yields powerful generative models. Our method masks the autoencoder’s parameters to respect autoregressive constraints: each input is recon-structed only from previous inputs in a given or-dering.

Feb 26, 2021 · While there a lot of different types of autoencoders, the focus of this article will be Masked Autoencoders for (Data) Distribution Estimation (MADE). In mathematical terms we want to estimate \(p_{data}\) from the samples \[\begin{align*} \boldsymbol{x^{(1)}, x^{(2)}, ..., x^{(n)} \sim p_{data}(x)} \end{align*}\]

Masked autoencoder for distribution estimation (MADE) is a well-structured density estimator, which alters a simple autoencoder by setting a set of masks on its connections to satisfy the autoregressive condition.

Jul 6, 2015 · We introduce a simple modification for autoencoder neural networks that yields powerful generative models. Our method masks the autoencoder's parameters to respect autoregressive constraints: each input is reconstructed only from previous inputs in a …

PyTorch implementation of the MADE model (Masked Autoencoder for Distribution Estimation). First download the binarized MNIST dataset. Run the training script. The following script create a model of hidden size (500, 1000), train for 5 epoch, and save the model weight to .data/mnist_made.pt. Create image samples.