article · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · 2024

Automatic speech recognition system-independent word error rate estimation

Park, C., Chen, M., Hain, T.

Cited by 0Open access

Abstract

Word error rate (WER) is a metric used to evaluate the quality of transcriptions produced by Automatic Speech Recognition (ASR) systems. In many applications, it is of interest to estimate WER given a pair of a speech utterance and a transcript. Previous work on WER estimation focused on building models that are trained with a specific ASR system in mind (referred to as ASR system-dependent). These are also domain-dependent and inflexible in real-world applications. In this paper, a hypothesis generation method for ASR System-Independent WER estimation (SIWE) is proposed. In contrast to prior work, the WER estimators are trained using data that simulates ASR system output. Hypotheses are generated using phonetically similar or linguistically more likely alternative words. In WER estimation experiments, the proposed method reaches a similar performance to ASR system-dependent WER estimators on in-domain data and achieves state-of-the-art performance on out-of-domain data. On the out-of-domain data, the SIWE model outperformed the baseline estimators in root mean square error and Pearson correlation coefficient by relative 17.58% and 18.21%, respectively, on Switchboard and CALLHOME. The performance was further improved when the WER of the training set was close to the WER of the evaluation dataset.

Cite this paper

APA 7

Park, C., Chen, M., & Hain, T. (2024). Automatic speech recognition system-independent word error rate estimation. White Rose Research Online (University of Leeds, The University of Sheffield, University of York).

MLA 9

Park, C., et al. “Automatic speech recognition system-independent word error rate estimation.” White Rose Research Online (University of Leeds, The University of Sheffield, University of York), 2024.

Chicago

Park, C., M. Chen, and T. Hain. 2024. “Automatic speech recognition system-independent word error rate estimation.” White Rose Research Online (University of Leeds, The University of Sheffield, University of York).

Harvard

Park, C., Chen, M. and Hain, T. (2024) ‘Automatic speech recognition system-independent word error rate estimation’, White Rose Research Online (University of Leeds, The University of Sheffield, University of York).

BibTeX

@article{park2024automatic,
  title = {{Automatic speech recognition system-independent word error rate estimation}},
  author = {Park, C. and Chen, M. and Hain, T.},
  journal = {White Rose Research Online (University of Leeds, The University of Sheffield, University of York)},
  year = {2024}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.