Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 8,483 papers for “quantization error”

Quantization Error For Vectocardiogram

Spherical Coordinates Piotr, Piotr Augustyniak · 2007

The vectocardiography (VCG) is the methodological extension of electrocardiography (ECG) that allows three-dimensional imaging of the cardiac electrical field. For the lack of the sufficient technological support it was underestimated for years, and currently comes back to the clinical practice thanks to the use of numerically performed spatial transforms. The VCG signal is usually acquired with use of the pseudoorthogonal Frank leads and stored as three simultaneous signals corresponding to three Cartesian coordinates XYZ. This paper addresses the issue of the alternative format for the VCG storage: the Spherical Coordinates.

Cited by 0

Quantization Error in Digital Holography

Mark F. Spencer, Douglas E. Thornton · Optica Advanced Photonics Congress 2022 · 2022

This paper uses the signal-to-noise ratio to analyze the effects of quantization error (aka digitization noise) in digital holography. Assuming a strong reference, the results show that quantization error depends on the pixel-well depth and the number of bits used for digitization.

Cited by 0

Quantization error in stereo imaging

Jeffrey J. Rodrı́guez, J.K. Aggarwal · 2003

In the design of a stereo imaging system, one often chooses the parameters to meet a desired error level. The probability density function of the range estimation error and the expected value of the range error magnitude are derived in terms of the various design parameters. The relative range error is proposed as a better way of quantifying the range resolution of a stereo imaging system.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Cited by 41

Scalar Quantization for Relative Error

John Z. Sun, Vivek K Goyal · 2011

Quantizers for probabilistic sources are usually optimized for mean-squared error. In many applications, maintaining low relative error is a more suitable objective. This measure has previously been heuristically connected with the use of logarithmic companding in perceptual coding. We derive optimal companding quantizers for fixed rate and variable rate under high-resolution assumptions. The analysis shows logarithmic companding is optimal for variable-rate quantization but generally not for fixed-rate quantization. Naturally, the improvement in relative error from using a correctly optimized quantizer can be arbitrarily large. We extend this framework for a large class of nondifference distortions.

Cited by 11

Dynamic Quantization Error Propagation in Encoder-Decoder ASR Quantization

Xinyu Wang, Yajie Luo, Yihong Wu et al. · arXiv (Cornell University) · 2026

Running Automatic Speech Recognition (ASR) models on memory-constrained edge devices requires efficient compression. While layer-wise post-training quantization is effective, it suffers from error accumulation, especially in encoder-decoder architectures. Existing solutions like Quantization Error Propagation (QEP) are suboptimal for ASR due to the model's heterogeneity, processing acoustic features in the encoder while generating text in the decoder. To address this, we propose Fine-grained Alpha for Dynamic Quantization Error Propagation (FADE), which adaptively controls the trade-off between cross-layer error correction and local quantization. Experiments show that FADE significantly improves stability by reducing performance variance across runs, while simultaneously surpassing baselines in mean WER.

Cited by 0Open access

Dynamic Quantization Error Propagation in Encoder-Decoder ASR Quantization

Xinyu Wang, Yajie Luo, Yihong Wu et al. · ArXiv.org · 2026

Running Automatic Speech Recognition (ASR) models on memory-constrained edge devices requires efficient compression. While layer-wise post-training quantization is effective, it suffers from error accumulation, especially in encoder-decoder architectures. Existing solutions like Quantization Error Propagation (QEP) are suboptimal for ASR due to the model's heterogeneity, processing acoustic features in the encoder while generating text in the decoder. To address this, we propose Fine-grained Alpha for Dynamic Quantization Error Propagation (FADE), which adaptively controls the trade-off between cross-layer error correction and local quantization. Experiments show that FADE significantly improves stability by reducing performance variance across runs, while simultaneously surpassing baselines in mean WER.

Cited by 0Open access

Asymptotic quantization error of continuous signals and the quantization dimension

P. Zádor · IEEE Transactions on Information Theory · 1982

Extensions of the limiting qnanfizafion error formula of Bennet are proved. These are of the form <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D_{s,k}(N,F)=N^{-\beta}B</tex> , where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</tex> is the number of output levels, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D_{s,k}(N,F)</tex> is the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</tex> th moment of the metric distance between quantizer input and output, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\beta,B&gt;0,k=s/\beta</tex> is the signal space dimension, and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</tex> is the signal distribution. If a suitably well-behaved <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</tex> -dimensional signal density <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f(x)</tex> exists, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">B=b_{s,k}[\int f^{\rho}(x)dx]^{1/ \rho},\rho=k/(s+k)</tex> , and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b_{s,k}</tex> does not depend on <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</tex> . For <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k=1,s=2</tex> this reduces to Bennett's formula. If <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</tex> is the Cantor distribution on <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[0,1],0&lt;k=s/ \beta=\log 2/ \log 3&lt;1</tex> and this <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</tex> equals the fractal dimension of the Cantor set <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[12,13]</tex> . Random quantization, optimal quantization in the presence of an output information constraint, and quantization noise in high dimensional spaces are also investigated.

Cited by 541

Asymptotic quantization error and cell-conditioned two-stage vector quantization.

Dong‐Hoon Lee · Deep Blue (University of Michigan) · 1990

An asymptotic formula is derived for the probability density of the error produced by scalar and vector quantization (VQ). The r-th moment of this density yields the generalized form of Bennett's formula. Like Bennett's formula, the density formula is an asymptotic result that holds for quantizers with many points. It also shows how the error density is determined by the distribution of quantization cell sizes and the distribution of quantization cell shapes. Moreover, this happens in a way that much about these distributions is revealed in the density itself. Consequently, one may use empirical measurements of the error density to expose geometrical characteristics of a quantizer whose cell sizes and shapes are otherwise unknown. We use the insight gained about the relationship between the error density and quantizer geometrical features to analyze two kinds of structured vector quantization--Tree-Structured Vector Quantization (TSVQ) and Two-Stage Vector Quantization (2VQ)--and to propose an improved version of the latter. For TSVQ, we find that empirical measurements of the error density support the hypothesis that the standard TSVQ design algorithm produces a quantizer whose cells are a mixture of various rectangular shapes with an essentially optimal size distribution. For 2VQ, using the error density formula, we derive asymptotic formulas for its distortion, and also for the increase in its distortion relative to single-stage quantization. We find that a one bit increase in either stage causes a 6 dB gain in the Signal-to-Noise Ratio, and propose a time saving method for designing the second stage. Lastly, we identify the cause of the performance loss of 2VQ and propose a modified form of 2VQ, called Cell-Conditioned Two-Stage Vector Quantization (CC2VQ), that corrects for such. We analyze CC2VQ by deriving an asymptotic formula for its distortion, and show that for high rates, CC2VQ gives as good a performance as single-stage VQ. Moreover, the second stage can be a low complexity lattice quantizer.

Cited by 2

Sigma-Delta Quantization: Number Theoretic Aspects of Refining Quantization Error

Aram Tangboondouangjit · Digital Repository at the University of Maryland (University of Maryland College Park) · 2006

The linear reconstruction phase of analog-to-digital (A/D) conversion in signal processing is analyzed in quantizing finite frame expansions for R^d. The specific setting is a K-level first order Sigma-Delta quantization with step size delta. Based on basic analysis, the d-dimensional Euclidean 2-norm of quantization error of Sigma-Delta quantization with input of elements in R^d decays like O(1/N) as the frame size N approaches infinity; while the L-infinity norm of quantization error of Sigma-Delta quantization with input of bandlimited functions decays like O(T) as the sampling ratio T approaches zero. It has been, however, observed via numerical simulation that, with input of bandlimited functions, the mean square error norm of quantization error seems to decay like O(T^(3/2)) as T approaches zero. Since the frame size N can be taken to correspond to the reciprocal of the sampling ratio T, this belief suggests that the corresponding behavior of quantization error, namely O(1/N^(3/2)), holds in the setting of finite frame expansions in R^d as well. A number theoretic technique involving uniform distribution of sequences of real numbers and approximation of exponential sums is introduced to derive a better quantization error than O(1/N) as N tends to infinity. This estimate is signal dependent.

Cited by 0Open access

Optimum error quantization for LMS adaptation

S.C. Douglas, T.H. Meng · 2002

The authors remove restrictions upon the transition and reconstruction levels of an L-level quantizer and examine error quantization as a design variable. The authors present a general performance factor expressing the misadjustment ratio of the quantized error algorithm with respect to the infinite-precision least-mean-squares (LMS) adaptation for a given convergence rate for Gaussian inputs. It is shown that maximizing this performance factor an an L-level quantizer with arbitrary transition levels and reconstruction levels produces the nonlinear system of equations defining the optimum Lloyd-Max quantizer for the Gaussian error. Simulations verify the theoretical performance loss predicted. The optimum-scaled uniform quantizer is shown to perform at most 0.013 dB worse than the optimum Lloyd-Max quantizer, indicating that in terms of both minimizing complexity and performance loss, uniform error quantization is close to the best for Gaussian inputs.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Cited by 5

Visual Thresholds For Wavelet Quantization Error

Andrew Watson Gloria, Andrew B. Watson, Gloria Y. Yang et al. · 1996

The Discrete Wavelet Transform (DWT) decomposes an image into bands that vary in spatial frequency and orientation. It is widely used for image compression. Measures of the visibility of DWT quantization errors are required to achieve optimal compression. Uniform quantization of a single band of coefficients results in an artifact that is the sum of a lattice of random amplitude basis functions of the corresponding DWT synthesis filter, which we call DWT uniform quantization noise. We measured visual detection thresholds for samples of DWT uniform quantization noise in Y, Cb, and Cr color channels. The spatial frequency of a wavelet is r 2

Cited by 0

QUANTIZATION ERROR IN STEREO IMAGING SYSTEMS

R. Balasubramanian A, Sukhendu Das B, S. Udayabaskaran C et al. · 2001

In this paper a stochastic analysis of the quantization error in a stereo imaging system has been presented. Further the probability density function of the range estimation error and the expected value of the range error magnitude are derived in terms of various design parameters. Further the relative range error is proposed.

Cited by 0

Quantization Error in Stereo Imaging systems

Ravi Balasubramanian, Sukhendu Das, S. Udayabaskaran et al. · International Journal of Computer Mathematics · 2002

In this paper a stochastic analysis of the quantization error in a stereo imaging system has been presented. Further the probability density function of the range estimation error and the expected value of the range error magnitude are derived in terms of various design parameters. Further the relative range error is proposed.

Cited by 17

Quantization error in hexagonal sensory configurations

Behzad Kamgar-Parsi, Behrooz Kamgar-Parsi · IEEE Transactions on Pattern Analysis and Machine Intelligence · 1992

The authors develop mathematical tools for estimating quantization error in hexagonal sensory configurations. These include analytic expressions for the average error and the error distribution of a function of an arbitrary number of independently quantized variables. These two quantities are essential for assessing the reliability of a given algorithm. They can also be used to compare the relative sensitivity of a particular algorithm to quantization error for hexagonal and other spatial samplings, e.g., square, and can have an impact on sensor design. Furthermore, it is shown that the ratio of hexagonal error to square error is bounded between 0.90 and 1.05.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Cited by 11

Error Spectrum Shaping and Vector Quantization

Jon Dattorro, Christine Law · 1997

We view truncation noise and quantization noise in basically the same way, arising from similar phenomena, regardless of bit rate. Truncation error feedback has been employed successfully in the scalar quantization of audio signals for about 20 years. The fundamental idea is to gain control over the spectral shape (the color) of the truncation error noise which

Cited by 0

Quantization error in magnetic resonance imaging

Rodrigo F. Cádiz, Camila Muñoz, Cristián Tejos et al. · Concepts in Magnetic Resonance Part A · 2014

ABSTRACT Due to the special nature of the acquisition domain, the quantization process of magnetic resonance imaging (MRI) data presents challenges that are not present in other medical imaging techniques. In this article, we demonstrate that the quantization error in MRI cannot be assumed to be a random variable with uniform distribution across the entire acquisition domain. Furthermore, the introduced error is not statistically independent of the input signal. On the contrary, we show that this error is correlated with the object that is being scanned, producing perceptually unpleasant artifacts in the image domain. Although the quantization error is not generally a critical issue in two‐dimensional (2D) MRI acquisition, it could be in the case of 3D acquisitions and in noise estimation measurements. © 2014 Wiley Periodicals, Inc. Concepts Magn Reson Part A 43A: 79–89, 2014.

Cited by 2Open access