Tomáš Rajtmajer · Digital Repository (National Repository of Grey Literature) · 2024
The simplest and most common way to generate samples from a given distribution is the inverse transform sampling method. Since this method uses the inverse function of the distribution's cumulative distribution function, it cannot be used in cases where it is impossible to obtain this inverse function. In such cases, we can use the rejection sampling method. This thesis focuses on generating samples from complex distributions using the rejection sampling method. The goal is to introduce this method and describe how it works. In the practical part, we will apply this method to obtain samples from the distribution with the density defined by the shape of Říp Mountain. Using normality tests, we will attempt to demonstrate through that Říp does not have normal distribution. 1
Cited by 0
Tommaso Bozzi · arXiv (Cornell University) · 2026
This thesis presents Regenerative Rejection Sampling (RRS), a novel approximate sampling algorithm inspired by classical Rejection Sampling and Markov Chain Monte Carlo methods. The method constructs a continuous-time regenerative process whose stationary distribution coincides with a target density known only up to a normalizing constant. Unlike standard Rejection Sampling, RRS does not require the existence of a finite constant that upper-bounds the likelihood ratio. As a result, its total variation convergence rate remains exponential for a larger class of scenarios compared to, for example, the Independent Metropolis-Hastings sampler, which requires a finite bounding constant. To explain the workings of the method, we first present a detailed review of renewal and regenerative processes, including their limit theorems, stationary versions, and convergence properties under standard conditions. We explain a coupling proof for exponential convergence of regenerative processes, under the assumption of a spread-out cycle length distribution. We then introduce the RRS algorithm, and derive its convergence rate. Its performance is compared theoretically and empirically with classical MCMC methods. Numerical experiments demonstrate that RRS can exhibit lower autocorrelations and faster effective mixing, both in synthetic examples and in a Bayesian probit regression model applied to a real medical dataset. Moreover, if the algorithm is run until time t, we show that the usual order $O(1/t)$ results for the bias of the time-average estimators, is improved to a bias of $O(1/t^2)$ for the estimator constructed from the RRS method, and provide easy-to-estimate non-asymptotic bounds for this bias.
Cited by 0Open access
Māris Ozols, Martin Roetteler, Jérémie Roland · arXiv (Cornell University) · 2012
Rejection sampling is a well-known method to sample from a target distribution, given the ability to sample from a given distribution. The method has been first formalized by von Neumann (1951) and has many applications in classical computing. We define a quantum analogue of rejection sampling: given a black box producing a coherent superposition of (possibly unknown) quantum states with some amplitudes, the problem is to prepare a coherent superposition of the same states, albeit with different target amplitudes. The main result of this paper is a tight characterization of the query complexity of this quantum state generation problem. We exhibit an algorithm, which we call quantum rejection sampling, and analyze its cost using semidefinite programming. Our proof of a matching lower bound is based on the automorphism principle which allows to symmetrize any algorithm over the automorphism group of the problem. Our main technical innovation is an extension of the automorphism principle to continuous groups that arise for quantum state generation problems where the oracle encodes unknown quantum states, instead of just classical data. Furthermore, we illustrate how quantum rejection sampling may be used as a primitive in designing quantum algorithms, by providing three different applications. We first show that it was implicitly used in the quantum algorithm for linear systems of equations by Harrow, Hassidim and Lloyd. Secondly, we show that it can be used to speed up the main step in the quantum Metropolis sampling algorithm by Temme et al.. Finally, we derive a new quantum algorithm for the hidden shift problem of an arbitrary Boolean function and relate its query complexity to "water-filling" of the Fourier spectrum.
Cited by 25Open access
Aditya Grover, Ramki Gummadi, Miguel Lázaro-Gredilla et al. · arXiv (Cornell University) · 2018
Learning latent variable models with stochastic variational inference is challenging when the approximate posterior is far from the true posterior, due to high variance in the gradient estimates. We propose a novel rejection sampling step that discards samples from the variational posterior which are assigned low likelihoods by the model. Our approach provides an arbitrarily accurate approximation of the true posterior at the expense of extra computation. Using a new gradient estimator for the resulting unnormalized proposal distribution, we achieve average improvements of 3.71 nats and 0.21 nats over state-of-the-art single-sample and multi-sample alternatives respectively for estimating marginal log-likelihoods using sigmoid belief networks on the MNIST dataset.
Cited by 3Open access
Tommaso Bozzi · arXiv (Cornell University) · 2026
This thesis presents Regenerative Rejection Sampling (RRS), a novel approximate sampling algorithm inspired by classical Rejection Sampling and Markov Chain Monte Carlo methods. The method constructs a continuous-time regenerative process whose stationary distribution coincides with a target density known only up to a normalizing constant. Unlike standard Rejection Sampling, RRS does not require the existence of a finite constant that upper-bounds the likelihood ratio. As a result, its total variation convergence rate remains exponential for a larger class of scenarios compared to, for example, the Independent Metropolis-Hastings sampler, which requires a finite bounding constant. To explain the workings of the method, we first present a detailed review of renewal and regenerative processes, including their limit theorems, stationary versions, and convergence properties under standard conditions. We explain a coupling proof for exponential convergence of regenerative processes, under the assumption of a spread-out cycle length distribution. We then introduce the RRS algorithm, and derive its convergence rate. Its performance is compared theoretically and empirically with classical MCMC methods. Numerical experiments demonstrate that RRS can exhibit lower autocorrelations and faster effective mixing, both in synthetic examples and in a Bayesian probit regression model applied to a real medical dataset. Moreover, if the algorithm is run until time t, we show that the usual order $O(1/t)$ results for the bias of the time-average estimators, is improved to a bias of $O(1/t^2)$ for the estimator constructed from the RRS method, and provide easy-to-estimate non-asymptotic bounds for this bias.
Cited by 0Open access
Ala Shayeghi · UWSpace (University of Waterloo) · 2015
Let H be a finite dimensional Hilbert space and ρ, σ ∈ D(H) be quantum states in H \nsuch that S(ρ||σ) is finite. In this thesis, we consider the following communication task involving two parties Alice and Bob. Suppose that Alice is given a classical description of the states ρ and σ. Given an unlimited number of copies of an entangled state whose marginal on Bob’s side is σ, Alice’s goal is to help Bob output a single copy of the state ρ by sending a single message to Bob in a one-way LOCC (short for local operation and classical communication) protocol. We propose a class of one-way LOCC protocols for this task which we refer to as quantum rejection sampling protocols. Inspired by the classical rejection sampling protocol of Harsha, Jain, McAllester, and Radhakrishnan [25] for a similar classical communication task, we introduce the Greedy Quantum Rejection Sampler. We characterize the expected communication cost of the protocol in terms of max-relative entropy of ρ with respect to σ, in the case where the state ρ is a pure state and prove that the Greedy Quantum Rejection Sampler is an optimal quantum rejection sampling protocol in this case. We describe an optimal quantum rejection sampling protocol in terms of a semidefinite program and we find general lower bounds and upper bounds on the expected communication cost of the optimal protocol. We propose an LOCC compression protocol based on the Greedy Quantum Rejection Sampler protocol, for lossless compression of an arbitrary pure state quantum source in the visible compression model and we show an upper bound on the average length of this encoding. The upper bound is always less than or equal to the Shannon entropy of the quantum source and the gap between the two quantities can be arbitrary large. Finally, we introduce a high-entanglement deterministic exact protocol for remote preparation of an arbitrary ensemble of quantum states. Our protocol is based on a quantum rejection sampling protocol which uses a prefix-free encoding for communication of the messages. We establish an upper bound on the expected communication cost of this protocol for the worst case choice of the target state in terms of the max-information in Bob’s output register at the end of the protocol about Alice’s classical input register. Furthermore, this protocol can be used as a non-oblivious universal protocol for exact remote preparation of an arbitrary d-dimensional state at an expected communication cost of at most lg(d) +O (lg(lg(d))).
Cited by 0
Daniel R. Sheldon · 2013
Adaptive rejection sampling (ARS) is an algorithm by Gilks and Wild for drawing samples from a continuous log-concave probability distribution with only black-box access to a function that computes the (unnormalized) density function. The ideas extend in a straightforward way to discrete log-concave distributions, but some details of the extension and its implementation can be tricky. This report provides the details of a discrete ARS algorithm. A companion implementation in C, with a MATLAB interface, accompanies the report. 1
Cited by 0
Costa Maia, Isabella, Congedo, Marco, Coelho Rodrigues, Pedro Luiz et al. · SPIRE - Sciences Po Institutional REpository · 2025
The present work introduces curvature-based rejection sampling (CURS). This is a method for sampling from a general class of probability densities defined on Riemannian manifolds. It can be used to sample from any probability density which "depends only on distance". The idea is to combine the statistical principle of rejection sampling with the geometric principle of volume comparison. CURS is an exact sampling method and (assuming the underlying Riemannian manifold satisfies certain technical conditions) it has a particularly moderate computational cost. The aim of the present work is to show that there are many applications where CURS should be the user's method of choice for dealing with relatively low-dimensional scenarios.
Cited by 0Open access
Isabella Costa Maia, Marco Congedo, Pedro Luiz Coelho Rodrigues et al. · ArXiv.org · 2025
The present work introduces curvature-based rejection sampling (CURS). This is a method for sampling from a general class of probability densities defined on Riemannian manifolds. It can be used to sample from any probability density which ``depends only on distance". The idea is to combine the statistical principle of rejection sampling with the geometric principle of volume comparison. CURS is an exact sampling method and (assuming the underlying Riemannian manifold satisfies certain technical conditions) it has a particularly moderate computational cost. The aim of the present work is to show that there are many applications where CURS should be the user's method of choice for dealing with relatively low-dimensional scenarios.
Cited by 0Open access
Gergely Flamich · 2023
Cited by 0
Jordan Awan, Vinayak Rao · arXiv (Cornell University) · 2021
Differential privacy (DP) offers strong theoretical privacy guarantees, but implementations of DP mechanisms may be vulnerable to side-channel attacks, such as timing attacks. When sampling methods such as MCMC or rejection sampling are used to implement a mechanism, the runtime can leak private information. We characterize the additional privacy cost due to the runtime of a rejection sampler in terms of both $(ε,δ)$-DP as well as $f$-DP. We also show that unless the acceptance probability is constant across databases, the runtime of a rejection sampler does not satisfy $ε$-DP for any $ε$. We show that there is a similar breakdown in privacy with adaptive rejection samplers. We propose three modifications to the rejection sampling algorithm, with varying assumptions, to protect against timing attacks by making the runtime independent of the data. The modification with the weakest assumptions is an approximate sampler, introducing a small increase in the privacy cost, whereas the other modifications give perfect samplers. We also use our techniques to develop an adaptive rejection sampler for log-Hölder densities, which also has data-independent runtime. We give several examples of DP mechanisms that fit the assumptions of our methods and can thus be implemented using our samplers.
Cited by 1Open access
Paulino Perez Rodriguez · 2007
Cited by 0Open access
Brian Caffo · Biometrika · 2002
Rejection sampling thins out samples from a candidate density from which it is easy to simulate, to obtain samples from a more awkward target density. A prerequisite is knowledge of the finite supremum of the ratio of the target and candidate densities. This severely restricts application of the method because it can be difficult to calculate the supremum. We use theoretical argument and numerical work to show that a practically perfect sample may be obtained by replacing the exact supremum with the maximum obtained from simulated candidates. We also provide diagnostics for failure of the method caused by a bad choice of candidate distribution. The implication is that essentially no theoretical work is required to apply rejection sampling in many practical cases.
Cited by 21Open access
Luca Martino · Electronics Letters · 2017
Monte Carlo (MC) methods have become very popular in signal processing during the past decades. The adaptive rejection sampling (ARS) algorithms are well‐known MC techniques which draw efficiently independent samples from univariate target densities. The ARS schemes yield a sequence of proposal functions that converge towards the target, so that the probability of accepting a sample approaches one. However, sampling from the proposal pdf becomes more computationally demanding each time it is updated. The parsimonious ARS method, where an efficient trade‐off between acceptance rate and proposal complexity is obtained, is proposed. Thus, the resulting algorithm is faster than the standard ARS approach.
Cited by 0Open access
Walter R. Gilks, Nicky Best, Keith Tan · Journal of the Royal Statistical Society Series C (Applied Statistics) · 1995
W. R. Gilks, N. G. Best, K. K. C. Tan, Adaptive Rejection Metropolis Sampling within Gibbs Sampling, Journal of the Royal Statistical Society. Series C (Applied Statistics), Vol. 44, No. 4 (1995), pp. 455-472
Cited by 684
Ahmad ALQabandi, Louis J. M. Aslett · 2025
Fast and efficient sampling from general univariate probability density functions. Implements a rejection sampling approach designed to take advantage of modern CPU caches and minimise evaluation of the target density for most samples. Many standard densities are internally implemented in 'C' for high performance, with general user defined densities also supported. A paper describing the methodology will be released soon.
Cited by 0Open access
Mark Jerrum · arXiv (Cornell University) · 2021
Partial Rejection Sampling is an algorithmic approach to obtaining a perfect sample from a specified distribution. The objects to be sampled are assumed to be represented by a number of random variables. In contrast to classical rejection sampling, in which all variables are resampled until a feasible solution is found, partial rejection sampling aims at greater efficiency by resampling only a subset of variables that `go wrong'. Partial rejection sampling is closely related to Moser and Tardos' algorithmic version of the Lovász Local Lemma, but with the additional requirement that a specified output distribution should be met. This article provides a largely self-contained account of the basic form of the algorithm and its analysis.
Cited by 2Open access
Mark Jerrum · Probability Surveys · 2024
Partial Rejection Sampling is an algorithmic approach to obtaining a perfect sample from a specified distribution. The objects to be sampled are assumed to be represented by a number of random variables. In contrast to classical rejection sampling, in which all variables are resampled until a feasible solution is found, partial rejection sampling aims at greater efficiency by resampling only a subset of variables that ‘go wrong’. Partial rejection sampling is closely related to Moser and Tardos’ algorithmic version of the Lovász Local Lemma, but with the additional requirement that a specified output distribution should be met. This article provides a largely self-contained account of the basic form of the algorithm and its analysis. Working within a unified framework allows a clean expression of the running time, and clarifies the scope for nondeterminism in its implementation.
Cited by 2Open access
Ley, Eduardo, Steel, Mark F.J. · e-Archivo (Carlos III University of Madrid) · 1992
Available from Centro de Informacion y Documentacion Cientifica CINDOC. Joaquin Costa, 22. 28002 Madrid. SPAIN / CINDOC - Centro de Informaciòn y Documentaciòn Cientìfica
Cited by 0Open access
Goeruer, D, Teh, YW · Oxford University Research Archive (ORA) (University of Oxford) · 2011
We describe a method for generating independent samples from univariate density functions using adaptive rejection sampling without the log-concavity requirement. The method makes use of the fact that many functions can be expressed as a sum of concave and convex functions. Using a concave-convex decomposition, we bound the logdensity by separately bounding the concave and convex parts using piecewise linear functions. The upper bound can then be used as the proposal distribution in rejection sampling. We demonstrate the applicability of the concave-convex approach on a number of standard distributions and describe an application to the efficient construction of sequential Monte Carlo proposal distributions for inference over genealogical trees. Computer code for the proposed algorithms is available online. © 2011 American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America.
Cited by 0