preprint · arXiv (Cornell University) · 2018

Stochastic Gradient MCMC with Repulsive Forces

Cited by 47Open access

Abstract

We propose a unifying view of two different Bayesian inference algorithms, Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) and Stein Variational Gradient Descent (SVGD), leading to improved and efficient novel sampling schemes. We show that SVGD combined with a noise term can be framed as a multiple chain SG-MCMC method. Instead of treating each parallel chain independently from others, our proposed algorithm implements a repulsive force between particles, avoiding collapse and facilitating a better exploration of the parameter space. We also show how the addition of this noise term is necessary to obtain a valid SG-MCMC sampler, a significant difference with SVGD. Experiments with both synthetic distributions and real datasets illustrate the benefits of the proposed scheme.

Cite this paper

APA 7

Gallego, V., & Insua, D. R. (2018). Stochastic Gradient MCMC with Repulsive Forces. arXiv (Cornell University). https://doi.org/10.48550/arxiv.1812.00071

MLA 9

Gallego, Víctor, and David Ríos Insua. “Stochastic Gradient MCMC with Repulsive Forces.” arXiv (Cornell University), 2018, https://doi.org/10.48550/arxiv.1812.00071.

Chicago

Gallego, Víctor, and David Ríos Insua. 2018. “Stochastic Gradient MCMC with Repulsive Forces.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.1812.00071.

Harvard

Gallego, V. and Insua, D.R. (2018) ‘Stochastic Gradient MCMC with Repulsive Forces’, arXiv (Cornell University). Available at: https://doi.org/10.48550/arxiv.1812.00071.

BibTeX

@misc{gallego2018stochastic,
  title = {{Stochastic Gradient MCMC with Repulsive Forces}},
  author = {Gallego, Víctor and Insua, David Ríos},
  journal = {arXiv (Cornell University)},
  year = {2018},
  doi = {10.48550/arxiv.1812.00071}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.