NADE: Neural Autoregressive Distribution Estimation. We present Neural Autoregressive Distribution Estimation (NADE) models, which are neural network architectures applied to the problem of unsupervised distribution and density estimation. They leverage the probability product rule and a weight sharing scheme inspired from restricted Boltzmann machines, to yield an estimator that is both tractable and has good generalization performance. We discuss how they achieve competitive performance in modeling both binary and real-valued observations. We also present how deep NADE models can be trained to be agnostic to the ordering of input dimensions used by the autoregressive product rule decomposition. Finally, we also show how to exploit the topological structure of pixels in images using a deep convolutional architecture for NADE.
Keywords for this software
References in zbMATH (referenced in 5 articles , 1 standard article )
Showing results 1 to 5 of 5.
- Brehmer, Johann; Louppe, Gilles; Pavez, Juan; Cranmer, Kyle: Mining gold from implicit models to improve likelihood-free inference (2020)
- Harshvardhan, G. M.; Gourisaria, Mahendra Kumar; Pandey, Manjusha; Rautaray, Siddharth Swarup: A comprehensive survey and analysis of generative models in machine learning (2020)
- Domingues, Rémi; Michiardi, Pietro; Zouaoui, Jihane; Filippone, Maurizio: Deep Gaussian process autoencoders for novelty detection (2018)
- Uria, Benigno; Côté, Marc-Alexandre; Gregor, Karol; Murray, Iain; Larochelle, Hugo: Neural autoregressive distribution estimation (2016)
- Mathieu Germain, Karol Gregor, Iain Murray, Hugo Larochelle: MADE: Masked Autoencoder for Distribution Estimation (2015) arXiv