Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 5,876 papers for “Word error rate”

Word Error Rate

Prof Marilyn Walker, Final Product, Release Of Penn Treebank · 2013

Cited by 0

Minimum Word Error Rate Decoding

G. Evermann · 1999

From the theory of Bayesian pattern recognition it is well known that the maximum a posteriori decision criterion yields a recogniser with the minimum probability of assigning the incorrect label to a pattern, if the correct probability distributions are used. This MAP criterion is also routinely employed in automatic speech recognition system. The problem addressed in this thesis is the fact that in a continuous speech recognition task the MAP criterion only guarantees to minimise the sentence error rate, but for many application the by far more relevant error metric is the word error rate.

Cited by 13

Semantic Word Error Rate for Sentence Similarity

Carmelo Spiccia, Agnese Augello, Giovanni Pilato et al. · 2016

Sentence similarity measures have applications in several tasks, including: Machine Translation, Paraphrase Identification, Speech Recognition, Question-answering and Text Summarization. However, measures designed for these tasks are aimed at assessing equivalence rather than resemblance, partly departing from human cognition of similarity. While this is reasonable for these activities, it hinders the applicability of sentence similarity measures to other tasks. We therefore propose a new sentence similarity measure specifically designed for resemblance evaluation, in order to cover these fields better. Experimental results are discussed.

Cited by 5

Multilingual Word Error Rate Estimation: E-Wer3

Shammur Absar Chowdhury, Ahmed Ali · 2023

The success of the multilingual automatic speech recognition systems empowered many voice-driven applications. However, measuring the performance of such systems remains a major challenge, due to its dependency on manually transcribed speech data in both mono- and multilingual scenarios. In this paper, we propose a novel multilingual framework – eWER3 – jointly trained on acoustic and lexical representation to estimate word error rate. We demonstrate the effectiveness of eWER3 to (i) predict WER without using any internal states from the ASR and (ii) use the multilingual shared latent space to push the performance of the close-related languages. We show our proposed multilingual model outperforms the previous monolingual word error rate estimation method (eWER2) by an absolute 9% increase in Pearson correlation coefficient (PCC), with better overall estimation between the predicted and reference WER.

Cited by 6

Letter recognition error rate (word recognition task) vs. Letter recognition error rate (letter recognition task).

Jean-Baptiste Bernard (2610385), Carlos Aguilar (784459), Eric Castet (2610382) · Figshare · 2016

<p>Letter recognition error rate during the word recognition task (Experiment 3) as a function of letter recognition error rate during the letter recognition task (Experiment 2), averaged across subjects for each letter and both fonts. For instance, the position of the letter 'n' for the Courier font shows that the letter recognition error rate was about 0.31 for the letter recognition task and 0.27 for the word recognition task when the Courier font was used.</p>

Cited by 0Open access

The relationship of word error rate to document ranking

Mang Shou, X., Sanderson, M., Tuffs, N. · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · 2004

This paper describes two experiments that examine the relationship of Word Error Rate (WER) of retrieved
\nspoken documents returned by a spoken document retrieval system. Previous work has demonstrated that
\nrecognition errors do not significantly affect retrieval effectiveness but whether they will adversely affect
\nrelevance judgement remains unclear. A user-based experiment measuring ability to judge relevance from
\nthe recognised text presented in a retrieved result list was conducted. The results indicated that users were
\ncapable of judging relevance accurately despite transcription errors. This lead an examination of the
\nrelationship of WER in retrieved audio documents to their rank position when retrieved for a particular
\nquery. Here it was shown that WER was somewhat lower for top ranked documents than it was for
\ndocuments retrieved further down the ranking, thereby indicating a possible explanation for the success of
\nthe user experiment.

Cited by 0Open access

Automatic speech recognition system-independent word error rate estimation

Park, C., Chen, M., Hain, T. · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · 2024

Word error rate (WER) is a metric used to evaluate the quality of transcriptions produced by Automatic Speech Recognition (ASR) systems. In many applications, it is of interest to estimate WER given a pair of a speech utterance and a transcript. Previous work on WER estimation focused on building models that are trained with a specific ASR system in mind (referred to as ASR system-dependent). These are also domain-dependent and inflexible in real-world applications. In this paper, a hypothesis generation method for ASR System-Independent WER estimation (SIWE) is proposed. In contrast to prior work, the WER estimators are trained using data that simulates ASR system output. Hypotheses are generated using phonetically similar or linguistically more likely alternative words. In WER estimation experiments, the proposed method reaches a similar performance to ASR system-dependent WER estimators on in-domain data and achieves state-of-the-art performance on out-of-domain data. On the out-of-domain data, the SIWE model outperformed the baseline estimators in root mean square error and Pearson correlation coefficient by relative 17.58% and 18.21%, respectively, on Switchboard and CALLHOME. The performance was further improved when the WER of the training set was close to the WER of the evaluation dataset.

Cited by 0Open access

Word Error Rate Estimation for Speech Recognition: e-WER

Ahmed Ali, Steve Renals · 2018

Measuring the performance of automatic speech recognition (ASR) systems requires manually transcribed data in order to compute the word error rate (WER), which is often time-consuming and expensive. In this paper, we propose a novel approach to estimate WER, or e-WER, which does not require a gold-standard transcription of the test set. Our e-WER framework uses a comprehensive set of features: ASR recognised text, character recognition results to complement recognition output, and internal decoder features. We report results for the two features; black-box and glass-box using unseen 24 Arabic broadcast programs. Our system achieves 16.9% WER root mean squared error (RMSE) across 1,400 sentences. The estimated overall WER e-WER was 25.3% for the three hours test set, while the actual WER was 28.5%.

Cited by 114Open access

Speech Models Training Technologies Comparison Using Word Error Rate

Roman Yakubovskyi, Yuriy Morozov · Advances in Cyber-Physical Systems · 2023

The main purpose of this work is to analyze and compare several technologies used for training speech models, including traditional approaches as Hidden Markov Models (HMMs) and more recent methods as Deep Neural Networks (DNNs). The technologies have been explained and compared using word error rate metric based on the input of 1000 words by a user with 15 decibel background noise. Word error rate metric has been ex- plained and calculated. Potential replacements for com- pared technologies have been provided, including: Atten- tion-based, Generative, Sparse and Quantum-inspired models. Pros and cons of those techniques as a potential replacement have been analyzed and listed. Data analyzing tools and methods have been explained and most common datasets used for HMM and DNN technologies have been described. Real life usage examples of both methods have been provided and systems based on them have been ana- lyzed.

Cited by 2Open access

Minimizing Word Error Rate In Textual Summaries Of Spoken Language

Klaus Zechner, Alex Waibel · KITopen · 2000

Automatic generation of text summaries for spoken language faces the problem of containing incorrect words and passages due to speech recognition errors. This paper describes comparative experiments where passages with higher speech recognizer confidence scores are favored in the ranking process. Results show that a relative word error rate reduction of over 10% can be achieved while at the same time the accuracy of the summary improves markedly. 1 Introduction The amount of audio data on-line has been growing rapidly in recent years, and so methods for efficiently indexing and retrieving non-textual information have become increasingly important (see, e.g., the TREC-7 branch for "Spoken Document Retrieval " (Garofolo et al., 1999)). One way of compressing audio information is the automatic creation of textual summaries which can be skimmed much faster and stored much more efficiently than the audio itself. There has been plenty of research in the area of summarizing written languag...

Cited by 87Open access

Decoding linear block codes for minimizing word error rate (Corresp.)

Tai-Yang Hwang · IEEE Transactions on Information Theory · 1979

Two modifications of the basic correlation decoding approach are presented. One of them yields a nonexhaustive optimum word decoding algorithm whose complexity depends upon the "projecting" structure of the code. This algorithm is then modified to yield a second decoding algorithm which, while not optimum, has simpler complexity. Applications to the AWGN channel are discussed and performance curves are given for (24, 12) and (31, 15) codes.

Cited by 45

Toward Zero Oracle Word Error Rate on the Switchboard Benchmark

Arlo Faria, Adam Janin, Sidhi Adkoli et al. · Interspeech 2022 · 2022

The “Switchboard benchmark” is a very well-known test set in automatic speech recognition (ASR) research, establishing record-setting performance for systems that claim human-level transcription accuracy. This work highlights lesser-known practical considerations of this evaluation, demonstrating major improvements in word error rate (WER) by correcting the reference transcriptions and deviating from the official scoring methodology. In this more detailed and reproducible scheme, even commercial ASR systems can score below 5% WER and the established record for a research system is lowered to 2.3%. An alternative metric of transcript precision is proposed, which does not penalize deletions and appears to be more discriminating for human vs. machine performance. While commercial ASR systems are still below this threshold, a research system is shown to clearly surpass the accuracy of commercial human speech recognition. This work also explores using standardized scoring tools to compute oracle WER by selecting the best among a list of alternatives. A phrase alternatives representation is compared to utterance-level N-best lists and word-level data structures; using dense lattices and adding out-of-vocabulary words, this achieves an oracle WER of 0.18%.

Cited by 4

Toward Zero Oracle Word Error Rate on the Switchboard Benchmark

Arlo Faria, Adam Janin, Korbinian Riedhammer et al. · arXiv (Cornell University) · 2022

The "Switchboard benchmark" is a very well-known test set in automatic speech recognition (ASR) research, establishing record-setting performance for systems that claim human-level transcription accuracy. This work highlights lesser-known practical considerations of this evaluation, demonstrating major improvements in word error rate (WER) by correcting the reference transcriptions and deviating from the official scoring methodology. In this more detailed and reproducible scheme, even commercial ASR systems can score below 5% WER and the established record for a research system is lowered to 2.3%. An alternative metric of transcript precision is proposed, which does not penalize deletions and appears to be more discriminating for human vs. machine performance. While commercial ASR systems are still below this threshold, a research system is shown to clearly surpass the accuracy of commercial human speech recognition. This work also explores using standardized scoring tools to compute oracle WER by selecting the best among a list of alternatives. A phrase alternatives representation is compared to utterance-level N-best lists and word-level data structures; using dense lattices and adding out-of-vocabulary words, this achieves an oracle WER of 0.18%.

Cited by 0Open access

Beyond Levenshtein: Leveraging Multiple Algorithms for Robust Word Error Rate Computations And Granular Error Classifications

Korbinian Kuhn, Verena Kersken, Gottfried Zimmermann · arXiv (Cornell University) · 2024

The Word Error Rate (WER) is the common measure of accuracy for Automatic Speech Recognition (ASR). Transcripts are usually pre-processed by substituting specific characters to account for non-semantic differences. As a result of this normalisation, information on the accuracy of punctuation or capitalisation is lost. We present a non-destructive, token-based approach using an extended Levenshtein distance algorithm to compute a robust WER and additional orthographic metrics. Transcription errors are also classified more granularly by existing string similarity and phonetic algorithms. An evaluation on several datasets demonstrates the practical equivalence of our approach compared to common WER computations. We also provide an exemplary analysis of derived use cases, such as a punctuation error rate, and a web application for interactive use and visualisation of our implementation. The code is available open-source.

Cited by 1Open access

Efficient estimation of the word error rate of digitizing waveform recorders

Eulalia Balestrieri, Sergio Rapuano, CATELANI, MARCANTONIO et al. · Florence Research (University of Florence) · 2011

The paper proposes a new method for Word Error Rate (WER) measurement for digitizing waveform recorders. In particular, a WER estimation procedure, based on Student's t distribution, is provided to estimate the WER and its confidence interval, minimizing the acquisition record length. The proposed test procedure has been experimentally verified by means of WER measurements on actual digitizing waveform recorders, where the test results have been compared to those achieved by implementing the method described in the IEEE Std.1057. Two measurement set-ups have been adopted in order to prove the validity of the proposed approach.

Cited by 0