Jonathan Machta · Physical Review E · 2009
Parallel tempering, also known as replica exchange Monte Carlo, is studied in the context of two simple free-energy landscapes. The first is a double-well potential defined by two macrostates separated by a barrier. The second is a "golf course" potential defined by microstates having two possible energies with exponentially more high-energy states than low-energy states. The equilibration time for replica exchange is analyzed for both systems. For the double-well system, parallel tempering with a number of replicas that scales as the square root of the barrier height yields exponential speedup of the equilibration time. On the other hand, replica exchange yields only marginal speedup for the golf course system. For the double-well system, the free-energy difference between the two wells has a large effect on the equilibration time. Nearly degenerate wells equilibrate much more slowly than strongly asymmetric wells. It is proposed that this difference in equilibration time may lead to a bias in measuring overlaps in spin glasses. These examples illustrate the strengths and weaknesses of replica exchange and may serve as a guide for understanding and improving the method in various applications.
Cited by 44Open access
Hamze, Firas, Dickson, Neil, Karimi, Kamran · arXiv (Cornell University) · 2010
This paper describes an algorithm for selecting parameter values (e.g. temperature values) at which to measure equilibrium properties with Parallel Tempering Monte Carlo simulation. Simple approaches to choosing parameter values can lead to poor equilibration of the simulation, especially for Ising spin systems that undergo $1^st$-order phase transitions. However, starting from an initial set of parameter values, the careful, iterative respacing of these values based on results with the previous set of values greatly improves equilibration. Example spin systems presented here appear in the context of Quantum Monte Carlo.
Cited by 0Open access
Nicola F. Müller, Remco Bouckaert · bioRxiv (Cold Spring Harbor Laboratory) · 2019
Abstract With ever more complex models used to study evolutionary patterns, approaches that facilitate efficient inference under such models are needed. Parallel tempering has long been used to speed up phylogenetic analyses and to make use of multi-core CPUs. Parallel tempering essentially runs multiple MCMC chains in parallel. All chains are heated except for one cold chain that explores the posterior probability space like a regular MCMC chain. This heating allows chains to make bigger jumps in phylogenetic state space. The heated chains can then be used to propose new states for other chains, including the cold chain. One of the practical challenges using this approach, is to find optimal temperatures of the heated chains to efficiently explore state spaces. We here provide an adaptive parallel tempering scheme to Bayesian phylogenetics, where the temperature difference between heated chains is automatically tuned to achieve a target acceptance probability of states being exchanged between individual chains. We first show the validity of this approach by comparing inferences of adaptive parallel tempering to MCMC on several datasets. We then explore where parallel tempering provides benefits over MCMC. We implemented this adaptive parallel tempering approach as an open source package licensed under GPL 3.0 to the Bayesian phylogenetics software BEAST2, available from https://github.com/nicfel/CoupledMCMC .
Cited by 28Open access
Stan E. Dosso, Charles W. Holland · Canadian acoustics · 2012
The method of parallel tempering is applied to achieve efficient and effective sampling of a particularly challenging multi-modal problem involving the inversion of acoustic reverberation data for geoacoustic and scattering parameters. Metropolis-Hastings sampling (MHS) and parallel tempering are compared for Bayesian geoacoustic inversion of simulated (noisy) reverberation data. A range-independent seabed model is assumed for the reverberation inversion problem in which the seabed is represented by an upper sediment layer of thickness 5m, sound velocity 1470 m/s, density 1.4 g/cm 3 , and attenuation 0.5 dB/wavelength. The standard deviation of the data errors 1dB is also considered an unknown parameter in the inversion. Considering the parallel-tempering results, it is found that the multi-modality of the joint marginals are mapped out far better using parallel tempering samples using MHS. There is little practical difference in results for different there is little practical difference in results for different.
Cited by 0Open access
Chai‐Yu Lin, Chin‐Kun Hu, Ulrich H. E. Hansmann · Proteins Structure Function and Bioinformatics · 2003
We report results from all-atom Monte Carlo simulations of the 36-residue villin headpiece subdomain HP-36. Protein-solvent interactions are approximated by an implicit solvent model. The parallel tempering is used to overcome the problem of slow convergence in low-temperature protein simulations. Our results show that this technique allows one to sample native-like structures of small proteins and points out the need for improved energy functions.
Cited by 94
Mark Daniel Rintoul, Steven J. Plimpton, M. P. Sears · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · 2003
We present here the details of the implementation of the parallel tempering Monte Carlo technique into a LAMMPS, a heavily used massively parallel molecular dynamics code at Sandia. This technique allows for many replicas of a system to be run at different simulation temperatures. At various points in the simulation, configurations can be swapped between different temperature environments and then continued. This allows for large regions of energy space to be sampled very quickly, and allows for minimum energy configurations to emerge in very complex systems, such as large biomolecular systems. By including this algorithm into an existing code, we immediately gain all of the previous work that had been put into LAMMPS, and allow this technique to quickly be available to the entire Sandia and international LAMMPS community. Finally, we present an example of this code applied to folding a small protein.
Cited by 1Open access
Alexandre Bouchard-Côté, Trevor Campbell, Nikola Surjanovic et al. · 2022
Cited by 0
Helmut G Katzgraber, Simon Trebst, David A Huse et al. · Journal of Statistical Mechanics Theory and Experiment · 2006
We introduce an algorithm for systematically improving the efficiency of parallel tempering Monte Carlo simulations by optimizing the simulated temperature set. Our approach is closely related to a recently introduced adaptive algorithm that optimizes the simulated statistical ensemble in generalized broad-histogram Monte Carlo simulations. Conventionally, a temperature set is chosen in such a way that the acceptance rates for replica swaps between adjacent temperatures are independent of the temperature and large enough to ensure frequent swaps. In this paper, we show that by choosing the temperatures with a modified version of the optimized ensemble feedback method we can minimize the round-trip times between the lowest and highest temperatures which effectively increases the efficiency of the parallel tempering algorithm. In particular, the density of temperatures in the optimized temperature set increases at the ‘bottlenecks’ of the simulation, such as phase transitions. In turn, the acceptance rates are now temperature dependent in the optimized temperature ensemble. We illustrate the feedback-optimized parallel tempering algorithm by studying the two-dimensional Ising ferromagnet and the two-dimensional fully frustrated Ising model, and briefly discuss possible feedback schemes for systems that require configurational averages, such as spin glasses.
Cited by 244Open access
David Pleydell · 2021
Functions for adaptive parallel tempering (APT) with NIMBLE models. Adapted from 'Lacki' & 'Miasojedow' (2016) <<a href="https://doi.org/10.1007%2Fs11222-015-9579-0" target="_top">doi:10.1007/s11222-015-9579-0</a>> and 'Miasojedow, Moulines and Vihola' (2013) <<a href="https://doi.org/10.1080%2F10618600.2013.778779" target="_top">doi:10.1080/10618600.2013.778779</a>>.
Cited by 0Open access
A. Malakis, Theodoros Papakonstantinou · Physical Review E · 2013
We review several parallel tempering schemes and examine their main ingredients for accuracy and efficiency. The present study covers two selection methods of temperatures and several choices for the exchange of replicas, including a recent novel all-pair exchange method. We compare the resulting schemes and measure specific heat errors and efficiency using the two-dimensional (2D) Ising model. Our tests suggest that an earlier proposal for using numbers of local moves related to the canonical correlation times is one of the key ingredients for increasing efficiency, and protocols using cluster algorithms are found to be very effective. Some of the protocols are also tested for efficiency and ground state production in 3D spin-glass models where we find that a simple nearest-neighbor approach using a local n-fold-way algorithm is the most effective. Finally, we present evidence that the asymptotic limits of the ground state energy for the isotropic case and for an anisotropic case of the 3D spin-glass model are very close and may even coincide.
Cited by 10Open access
David J. Earl, Michael W. Deem · Physical Chemistry Chemical Physics · 2005
We review the history of the parallel tempering simulation method. From its origins in data analysis, the parallel tempering method has become a standard workhorse of physicochemical simulations. We discuss the theory behind the method and its various generalizations. We mention a selected set of the many applications that have become possible with the introduction of parallel tempering, and we suggest several promising avenues for future research.
Cited by 1,271Open access
B. Joó A, Ukqcd Collaboration · 1998
We present results from our study of the Parallel Tempering algorithm. We examine the swapping acceptance rate of a twin subensemble PT system. We use action matching technology in an attempt to maximise the swap acceptance rate. We model the autocorrelation times within Parallel Tempering ensembles in terms of autocorrelation times from Hybrid Monte Carlo. We present estimates for the autocorrelation times of the plaquette operator. 1.
Cited by 0
Stephan Burkhardt · Scholarworks (University of Massachusetts Amherst) · 2023
The efficiency of parallel tempering Monte Carlo is studied for a two-dimensional Ising system of length L with N=L^2 spins. An external field is used to introduce a difference in free energy between the two low temperature states.\nIt is found that the number of replicas R_opt that optimizes the parallel tempering algorithm scales as the square root of the system size N. For two symmetric low temperature states, the time needed for equilibration is observed to grow as L^2.18. If a significant difference in free energy is present between the two states, this changes to L^1.02.\nIt is therefore established that parallel tempering is sped up by a factor of roughly L if an asymmetry is introduced between the low temperature states. This confirms previously made predictions for the efficiency of parallel tempering. These findings should be especially relevant when using parallel tempering for systems like spin glasses, where no information about the degeneracy of low temperature states is available prior to the simulation.
Cited by 0Open access
Stan E. Dosso, Charles W. Holland, Malcolm Sambridge · The Journal of the Acoustical Society of America · 2012
This paper applies parallel tempering within a Bayesian formulation for strongly nonlinear geoacoustic inverse problems. Bayesian geoacoustic inversion consists of sampling the posterior probability density (PPD) of seabed parameters to estimate integral properties, such as marginal probability distributions, based on ocean acoustic data and prior information. This sampling is usually carried out using the Markov-chain Monte Carlo method of Metropolis-Hastings sampling. However, standard sampling methods can be very inefficient for strongly nonlinear problems involving multi-modal PPDs with the potential to miss important regions of the parameter space and to significantly underestimate parameter uncertainties. Parallel tempering achieves efficient/effective sampling of challenging parameter spaces with the ability to transition freely between multiple PPD modes by running parallel Markov chains at a series of increasing sampling temperatures with probabilistic interchanges between chains. The approach is illustrated for inversion of (simulated) acoustic reverberation data for which the PPD is highly multi-modal. While Metropolis-Hastings sampling gives poor results even with very large sample sizes, parallel tempering provides efficient, convergent sampling of the PPD. Methods to enhance the efficiency of parallel tempering are also considered.
Cited by 111
Neuhaus, T. · JuSER (Forschungszentrum Jülich) · 2007
Cited by 0
Latz, Jonas, Madrigal-Cianci, Juan P., Nobile, Fabio et al. · RWTH Publications (RWTH Aachen) · 2021
Cited by 0Open access
Hansmann, U. H. E. · JuSER (Forschungszentrum Jülich) · 2008
Cited by 0
Alexandra Patriksson, David van der Spoel · Physical Chemistry Chemical Physics · 2008
An algorithm is proposed that generates a set of temperatures for use in parallel tempering simulations (also known as temperature-replica exchange molecular dynamics simulations) of proteins to obtain a desired exchange probability Pdes. The input consists of the number of protein atoms and water molecules in the system, information about the use of constraints and virtual sites and the lower temperature limits. The temperatures generated yield probabilities which are very close to Pdes (correlation 97%), independent of force field and over a wide temperature range. To facilitate its use, the algorithm has been implemented as a web server at .
Cited by 503
Theodoros Papakonstantinou, A. Malakis · Journal of Physics Conference Series · 2014
We review parallel tempering schemes and examine their main ingredients for accuracy and efficiency. We discuss two selection methods of temperatures and some alternatives for the exchange of replicas, including all-pair exchange methods. We measure specific heat errors and round-trip efficiency using the two-dimensional (2D) Ising model, and also test the efficiency for the ground state production in 3D spin glass models. We find that the optimization of the GS problem is highly influenced by the choice of the temperature range of the PT process. Finally, we present numerical evidence concerning the universality aspects of an anisotropic case of the 3D spin-glass model.
Cited by 3Open access
Latz, Jonas, Madrigal-Cianci, Juan P., Nobile, Fabio et al. · arXiv (Cornell University) · 2020
In the current work we present two generalizations of the Parallel Tempering algorithm, inspired by the so-called continuous-time Infinite Swapping algorithm. Such a method, found its origins in the molecular dynamics community, and can be understood as the limit case of the continuous-time Parallel Tempering algorithm, where the (random) time between swaps of states between two parallel chains goes to zero. Thus, swapping states between chains occurs continuously. In the current work, we extend this idea to the context of time-discrete Markov chains and present two Markov chain Monte Carlo algorithms that follow the same paradigm as the continuous-time infinite swapping procedure. We analyze the convergence properties of such discrete-time algorithms in terms of their spectral gap, and implement them to sample from different target distributions. Numerical results show that the proposed methods significantly improve over more traditional sampling algorithms such as Random Walk Metropolis and (traditional) Parallel Tempering.
Cited by 0Open access