Radford Neal · 2000
. Markov chain sampling methods that automatically adapt to characteristics of the distribution being sampled can be constructed by exploiting the principle that one can sample from a distribution by sampling uniformly from the region under the plot of its density function. A Markov chain that converges to this uniform distribution can be constructed by alternating uniform sampling in the vertical direction with uniform sampling from the horizontal `slice' defined by the current vertical position, or more generally, with some update that leaves the uniform distribution over this slice invariant. Variations on such `slice sampling' methods are easily implemented for univariate distributions, and can be used to sample from a multivariate distribution by updating each variable in turn. This approach is often easier to implement than Gibbs sampling, and more efficient than simple Metropolis updates, due to the ability of slice sampling to adaptively choose the magnitude of changes made. It...
Cited by 0
Radford M. Neal · The Annals of Statistics · 2003
Markov chain sampling methods that adapt to characteristics of the distribution being sampled can be constructed using the principle that one can ample from a distribution by sampling uniformly from the region under the plot of its density function. A Markov chain that converges to this uniform distribution can be constructed by alternating uniform sampling in the vertical direction with uniform sampling from the horizontal "slice" defined by the current vertical position, or more generally, with some update that leaves the uniform distribution over this slice invariant. Such "slice sampling" methods are easily implemented for univariate distributions, and can be used to sample from a multivariate distribution by updating each variable in turn. This approach is often easier to implement than Gibbs sampling and more efficient than simple Metropolis updates, due to the ability of slice sampling to adaptively choose the magnitude of changes made. It is therefore attractive for routine and automated use. Slice sampling methods that update all variables simultaneously are also possible. These methods can adaptively choose the magnitudes of changes made to each variable, based on the local properties of the density function. More ambitiously, such methods could potentially adapt to the dependencies between variables by constructing local quadratic approximations. Another approach is to improve sampling efficiency by suppressing random walks. This can be done for univariate slice sampling by "overrelaxation," and for multivariate slice sampling by "reflection" from the edges of the slice.
Cited by 1,323Open access
Neal, R M · CERN Bulletin · 2000
Markov chain sampling methods that automatically adapt to characteristics of the distribution being sampled can be constructed by exploiting the principle that one can sample from a distribution by sampling uniformly from the region under the plot of its density function. A Markov chain that converges to this uniform distribution can be constructed by alternating uniform sampling in the vertical direction with uniform sampling from the horizontal `slice' defined by the current vertical position, or more generally, with some update that leaves the uniform distribution over this slice invariant. Variations on such `slice sampling' methods are easily implemented for univariate distributions, and can be used to sample from a multivariate distribution by updating each variable in turn. This approach is often easier to implement than Gibbs sampling, and more efficient than simple Metropolis updates, due to the ability of slice sampling to adaptively choose the magnitude of changes made. It is therefore attractive for routine and automated use. Slice sampling methods that update all variables simultaneously are also possible. These methods can adaptively choose the magnitudes of changes made to each variable, based on the local properties of the density function. More ambitiously, such methods could potentially allow the sampling to adapt to dependencies between variables by constructing local quadratic approximations. Another approach is to improve sampling efficiency by suppressing random walks. This can be done using `overrelaxed' versions of univariate slice sampling procedures, or by using `reflective' multivariate slice sampling methods, which bounce off the edges of the slice.
Cited by 0
Radford M. Neal · 2000
Abstract. Markov chain sampling methods that automatically adapt to characteristics of the distribution being sampled can be constructed by exploiting the principle that one can sample from a distribution by sampling uniformly from the region under the plot of its density function. A Markov chain that converges to this uniform distribution can be constructed by alternating uniform sampling in the vertical direction with uniform sampling from the horizontal ‘slice ’ defined by the current vertical position, or more generally, with some update that leaves the uniform distribution over this slice invariant. Variations on such ‘slice sampling ’ methods are easily implemented for univariate distributions, and can be used to sample from a multivariate distribution by updating each variable in turn. This approach is often easier to implement than Gibbs sampling, and more efficient than simple Metropolis updates, due to the ability of slice sampling to adaptively choose the magnitude of changes made. It is therefore attractive for routine and automated use. Slice sampling methods that update all variables simultaneously are also possible. These methods can adaptively choose the magnitudes of changes made to each variable, based on the local properties of the density function. More ambitiously, such methods could potentially allow the sampling to adapt to dependencies between variables by constructing local quadratic approximations. Another approach is to improve sampling efficiency by suppressing random walks. This can be done using ‘overrelaxed ’ versions of univariate slice sampling procedures, or by using ‘reflective ’ multivariate slice sampling methods, which bounce off the edges of the slice.
Cited by 0
Matthew Heiner, Samuel Johnson, Joshua R. Christensen et al. · arXiv (Cornell University) · 2024
We propose and demonstrate a novel, effective approach to slice sampling. Using the probability integral transform, we first generalize Neal's shrinkage algorithm, standardizing the procedure to an automatic and universal starting point: the unit interval. This enables the introduction of approximate (pseudo-) targets through the factorization used in importance sampling, a technique that popularized elliptical slice sampling, while still sampling from the correct target distribution. Accurate pseudo-targets can boost sampler efficiency by requiring fewer rejections and by reducing skewness in the transformed target. This strategy is effective when a natural, possibly crude approximation to the target exists. Alternatively, obtaining a marginal pseudo-target from initial samples provides an intuitive and automatic tuning procedure. We consider two metrics for evaluating the quality of approximation; each can be used as a criterion to find an optimal pseudo-target or as an interpretable diagnostic. We examine performance of the proposed sampler relative to other popular, easily implemented MCMC samplers on standard targets in isolation, and as steps within a Gibbs sampler in a Bayesian modeling context. We extend the transformation method to multivariate slice samplers and demonstrate with a constrained state-space model for which a readily available forward-backward algorithm provides the target approximation. Supplemental materials and accompanying R package qslice are available online.
Cited by 1Open access
Lu Jingjing, Merrill Liechty · 2008
In Bayesian decision theory, stochastic simulation techniques have been widely used by decision makers. The Markov chain Monte Carlo method (MCMC) plays a key role in stochastic simulation within the framework of Bayesian statistics. The notion of "slice sampling", or employing auxiliary variables in sampling, has been recently suggested in the literature for improving the efficiency of the traditional MCMC methods. In the existing literature, the one dimensional slice sampler has been extensively studied, yet the literature on the multidimensional case is sparse. In this study, we utilize multiple auxiliary variables in our sampling algorithms for multivariate normal distributions, which adapt better to the local properties of the target probability distributions. We show that these methods are flexible enough to allow for truncation to rectangular regions and/or the exclusion of any n dimensional hyper-quadrant. We compare these algorithms for both efficiency and accuracy via simulation experiments. Our results show that our new sampling techniques are accurate and more effective than the traditional single auxiliary variable slice sampler, especially for truncated multivariate distributions. Further, we extend our methods to general multivariate distributions including multivariate student t distributions, multivariate elliptical distributions, multivariate skew normal distributions, multivariate skew t distributions and a general class of multivariate skew elliptical distributions. We also discuss and outline some applications of our algorithms in the real business world, especially in the production and operations management and the finance areas. With regard to production and operations management, we find that the proposed multivariate normal slice sampler can be implemented in a stochastic optimization process for finding the optimal fulfillment rate of an assembly system. This system consists of n components, from which m products are assembled via a periodic review control policy. In finance area, we show that multivariate slice samplers can be used to update model parameters and predict future asset returns in a higher moments Bayesian portfolio optimization model, which is based on a general class of multivariate skew t distributions. The application of a multivariate slice sampler improves the model's ability of handling large data sets and saves computation time for deriving complicated posterior probability distributions.
Cited by 0Open access
Teresa Pietrabissa, Simone Rusconi · Springer proceedings in mathematics & statistics · 2013
Cited by 2
Minas Karamanis, Florian Beutler · Statistics and Computing · 2021
Abstract Slice sampling has emerged as a powerful Markov Chain Monte Carlo algorithm that adapts to the characteristics of the target distribution with minimal hand-tuning. However, Slice Sampling’s performance is highly sensitive to the user-specified initial length scale hyperparameter and the method generally struggles with poorly scaled or strongly correlated distributions. This paper introduces Ensemble Slice Sampling (ESS), a new class of algorithms that bypasses such difficulties by adaptively tuning the initial length scale and utilising an ensemble of parallel walkers in order to efficiently handle strong correlations between parameters. These affine-invariant algorithms are trivial to construct, require no hand-tuning, and can easily be implemented in parallel computing environments. Empirical tests show that Ensemble Slice Sampling can improve efficiency by more than an order of magnitude compared to conventional MCMC methods on a broad range of highly correlated target distributions. In cases of strongly multimodal target distributions, Ensemble Slice Sampling can sample efficiently even in high dimensions. We argue that the parallel, black-box and gradient-free nature of the method renders it ideal for use in scientific fields such as physics, astrophysics and cosmology which are dominated by a wide variety of computationally expensive and non-differentiable models.
Cited by 5Open access
John Skilling, Djc Mackay · Cambridge University Engineering Department Publications Database · 2003
Cited by 11
Topps, David, Cooke, Lara, Armson, Heather et al. · Borealis · 2025
Demo of stratified sampling technique for online CME evaluations. Basic logic and QUestions work. Just crude so far but demonstrates the concept. After the first page, there is a section where the group of questions that you are asked are randomly assigned. There is then a role specific section, which uses an earlier Counter score to determine which questions you will be asked.
Cited by 0Open access
MIRA, ANTONIETTA, ROBERTS GO · IrInSubria (University of Insubria) · 2003
Cited by 0
Guillermina Senn, Nathan Glatt-Holtz, Giulia Carigi et al. · arXiv (Cornell University) · 2026
We introduce Multiproposal Elliptical Slice Sampling, a self-tuning multiproposal Markov chain Monte Carlo method for Bayesian inference with Gaussian priors. Our method generalizes the Elliptical Slice Sampling algorithm by 1) allowing multiple candidate proposals to be sampled in parallel at each self-tuning step, and 2) basing the acceptance step on a distance-informed transition matrix that can favor proposals far from the current state. This allows larger moves in state space and faster self-tuning, at essentially no additional wall clock time for expensive likelihoods, and results in improved mixing. We additionally provide theoretical arguments and experimental results suggesting dimension-robust mixing behavior, making the algorithm particularly well suited for Bayesian PDE inverse problems.
Cited by 0Open access
Tibbits, Matthew M., Groendyke, Chris, Murali Haran et al. · Figshare · 2015
Cited by 0Open access
Josef Leydold · 2003
ePubWU, the institutional repository of the WU Vienna University of Economics and Business, is provided by the University Library and the IT-Services. The aim is to enable open access to the scholarly output of the WU.
Cited by 0
Kevin Bitterlich, Daniel Rudolf, Björn Sprungk · arXiv (Cornell University) · 2025
Slice sampling is a well-established Markov chain Monte Carlo method for (approximate) sampling of target distributions which are only known up to a normalizing constant. The method is based on choosing a new state on a slice, i.e., a superlevel set of the given unnormalized target density (with respect to a reference measure). However, slice sampling algorithms usually require per step multiple evaluations of the target density, and thus can become computationally expensive. This is particularly the case for Bayesian inference with costly likelihoods. In this paper, we exploit deterministic approximations of the target density, which are relatively cheap to evaluate, and propose delayed acceptance versions of hybrid slice samplers. We show ergodicity of the resulting slice sampling methods, discuss the superiority of delayed acceptance (ideal) slice sampling over delayed acceptance Metropolis-Hastings algorithms, and illustrate the benefits of our novel approach in terms improved computational efficiency in several numerical experiments.
Cited by 0Open access
Iain Murray, Matthew M. Graham · arXiv (Cornell University) · 2015
Markov chain Monte Carlo (MCMC) methods asymptotically sample from complex probability distributions. The pseudo-marginal MCMC framework only requires an unbiased estimator of the unnormalized probability distribution function to construct a Markov chain. However, the resulting chains are harder to tune to a target distribution than conventional MCMC, and the types of updates available are limited. We describe a general way to clamp and update the random numbers used in a pseudo-marginal method's unbiased estimator. In this framework we can use slice sampling and other adaptive methods. We obtain more robust Markov chains, which often mix more quickly.
Cited by 6Open access
Tibbits, Matthew M., Groendyke, Chris, Murali Haran et al. · Figshare · 2015
Cited by 0Open access
Tibbits, Matthew M., Groendyke, Chris, Murali Haran et al. · Figshare · 2014
Markov chain Monte Carlo (MCMC) algorithms offer a very general approach for sampling from arbitrary distributions. However, designing and tuning MCMC algorithms for each new distribution can be challenging and time consuming. It is particularly difficult to create an efficient sampler when there is strong dependence among the variables in a multivariate distribution. We describe a two-pronged approach for constructing efficient, automated MCMC algorithms: (1) we propose the “factor slice sampler,” a generalization of the univariate slice sampler where we treat the selection of a coordinate basis (factors) as an additional tuning parameter, and (2) we develop an approach for automatically selecting tuning parameters to construct an efficient factor slice sampler. In addition to automating the factor slice sampler, our tuning approach also applies to the standard univariate slice samplers. We demonstrate the efficiency and general applicability of our automated MCMC algorithm with a number of illustrative examples. This article has online supplementary materials.
Cited by 0Open access
M. Tibbits, Chris Groendyke, Murali Haran et al. · Figshare · 2014
Markov chain Monte Carlo (MCMC) algorithms offer a very general approach for sampling from arbitrary distributions. However, designing and tuning MCMC algorithms for each new distribution can be challenging and time consuming. It is particularly difficult to create an efficient sampler when there is strong dependence among the variables in a multivariate distribution. We describe a two-pronged approach for constructing efficient, automated MCMC algorithms: (1) we propose the “factor slice sampler,” a generalization of the univariate slice sampler where we treat the selection of a coordinate basis (factors) as an additional tuning parameter, and (2) we develop an approach for automatically selecting tuning parameters to construct an efficient factor slice sampler. In addition to automating the factor slice sampler, our tuning approach also applies to the standard univariate slice samplers. We demonstrate the efficiency and general applicability of our automated MCMC algorithm with a number of illustrative examples. This article has online supplementary materials.
Cited by 0Open access
Schär, Philip, Habeck, Michael, Rudolf, Daniel · GoeScholar The Publication Server of the Georg-August-Universität Göttingen (Georg-August-Universität Göttingen) · 2023
Polar slice sampling (Roberts & Rosenthal, 2002) is a Markov chain approach \nfor approximate sampling of distributions that is difficult, if not impossible, \nto implement efficiently, but behaves provably well with respect to the \ndimension. By updating the directional and radial components of chain iterates \nseparately, we obtain a family of samplers that mimic polar slice sampling, and \nyet can be implemented efficiently. Numerical experiments for a variety of \nsettings indicate that our proposed algorithm outperforms the two most closely \nrelated approaches, elliptical slice sampling (Murray et al., 2010) and \nhit-and-run uniform slice sampling (MacKay, 2003). We prove the \nwell-definedness and convergence of our methods under suitable assumptions on \nthe target distribution.
Cited by 0Open access