Improving ASR in Reverberant Environments

Yen Lun Liao; Chi Han Lin; Ren Yuan Lyu; Jyh Shing Roger Jang

doi:10.1109/ISCSLP57327.2022.10038145

Improving ASR in Reverberant Environments

Yen Lun Liao^*
, Chi Han Lin
, Ren Yuan Lyu
, Jyh Shing Roger Jang^*

^*Corresponding author for this work

Department of Computer Science and Information Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Automatic Speech Recognition (ASR) significantly reduces the effort to create audio transcripts. Despite its convenience, the performance of ASR is unstable in disturbing environments; for instance, indoor signals are usually corrupted by reverberation (reverb), resulting in diminished performance in ASR. A type of solution is to construct an acoustic dereverberation (dereverb) model to pre-process the original signals before submitting them to ASR. However, the acoustic properties of the output signal of the dereverb model differ from that of the training dataset for ASR, resulting in a decline in performance. This paper optimizes the aforementioned structure from four aspects: signal classification, reverberation removal, data mismatch removal in ASR, and ensemble algorithms. With the proposed sentence-level fusion (SLF) and word-level fusion (WLF) ensemble algorithms, a CER of 7.23% was reached in the mixture test set of the reverberated and clean Aishell1 compared to the single model, achieving a reduction in the CER by 20.72%.

Original language	English
Title of host publication	2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022
Editors	Kong Aik Lee, Hung-yi Lee, Yanfeng Lu, Minghui Dong
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	165-169
Number of pages	5
ISBN (Electronic)	9798350397963
DOIs	https://doi.org/10.1109/ISCSLP57327.2022.10038145
State	Published - 2022
Event	13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022 - Singapore, Singapore Duration: 11 12 2022 → 14 12 2022

Publication series

Name	2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022

Conference

Conference	13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022
Country/Territory	Singapore
City	Singapore
Period	11/12/22 → 14/12/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Keywords

automatic speech recognition
dereverberation
model ensemble
new structure
string confusion network

Access to Document

10.1109/ISCSLP57327.2022.10038145

Cite this

Liao, Y. L., Lin, C. H., Lyu, R. Y., & Jang, J. S. R. (2022). Improving ASR in Reverberant Environments. In K. A. Lee, H. Lee, Y. Lu, & M. Dong (Eds.), 2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022 (pp. 165-169). (2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISCSLP57327.2022.10038145

Liao, Yen Lun ; Lin, Chi Han ; Lyu, Ren Yuan et al. / Improving ASR in Reverberant Environments. 2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022. editor / Kong Aik Lee ; Hung-yi Lee ; Yanfeng Lu ; Minghui Dong. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 165-169 (2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022).

@inproceedings{9e2214d745834520865c76f3abb65ae3,

title = "Improving ASR in Reverberant Environments",

abstract = "Automatic Speech Recognition (ASR) significantly reduces the effort to create audio transcripts. Despite its convenience, the performance of ASR is unstable in disturbing environments; for instance, indoor signals are usually corrupted by reverberation (reverb), resulting in diminished performance in ASR. A type of solution is to construct an acoustic dereverberation (dereverb) model to pre-process the original signals before submitting them to ASR. However, the acoustic properties of the output signal of the dereverb model differ from that of the training dataset for ASR, resulting in a decline in performance. This paper optimizes the aforementioned structure from four aspects: signal classification, reverberation removal, data mismatch removal in ASR, and ensemble algorithms. With the proposed sentence-level fusion (SLF) and word-level fusion (WLF) ensemble algorithms, a CER of 7.23\% was reached in the mixture test set of the reverberated and clean Aishell1 compared to the single model, achieving a reduction in the CER by 20.72\%.",

keywords = "automatic speech recognition, dereverberation, model ensemble, new structure, string confusion network",

author = "Liao, \{Yen Lun\} and Lin, \{Chi Han\} and Lyu, \{Ren Yuan\} and Jang, \{Jyh Shing Roger\}",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022 ; Conference date: 11-12-2022 Through 14-12-2022",

year = "2022",

doi = "10.1109/ISCSLP57327.2022.10038145",

language = "英语",

series = "2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "165--169",

editor = "Lee, \{Kong Aik\} and Hung-yi Lee and Yanfeng Lu and Minghui Dong",

booktitle = "2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022",

address = "美国",

}

Liao, YL, Lin, CH, Lyu, RY & Jang, JSR 2022, Improving ASR in Reverberant Environments. in KA Lee, H Lee, Y Lu & M Dong (eds), 2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022. 2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022, Institute of Electrical and Electronics Engineers Inc., pp. 165-169, 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022, Singapore, Singapore, 11/12/22. https://doi.org/10.1109/ISCSLP57327.2022.10038145

Improving ASR in Reverberant Environments. / Liao, Yen Lun; Lin, Chi Han; Lyu, Ren Yuan et al.
2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022. ed. / Kong Aik Lee; Hung-yi Lee; Yanfeng Lu; Minghui Dong. Institute of Electrical and Electronics Engineers Inc., 2022. p. 165-169 (2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Improving ASR in Reverberant Environments

AU - Liao, Yen Lun

AU - Lin, Chi Han

AU - Lyu, Ren Yuan

AU - Jang, Jyh Shing Roger

PY - 2022

Y1 - 2022

N2 - Automatic Speech Recognition (ASR) significantly reduces the effort to create audio transcripts. Despite its convenience, the performance of ASR is unstable in disturbing environments; for instance, indoor signals are usually corrupted by reverberation (reverb), resulting in diminished performance in ASR. A type of solution is to construct an acoustic dereverberation (dereverb) model to pre-process the original signals before submitting them to ASR. However, the acoustic properties of the output signal of the dereverb model differ from that of the training dataset for ASR, resulting in a decline in performance. This paper optimizes the aforementioned structure from four aspects: signal classification, reverberation removal, data mismatch removal in ASR, and ensemble algorithms. With the proposed sentence-level fusion (SLF) and word-level fusion (WLF) ensemble algorithms, a CER of 7.23% was reached in the mixture test set of the reverberated and clean Aishell1 compared to the single model, achieving a reduction in the CER by 20.72%.

AB - Automatic Speech Recognition (ASR) significantly reduces the effort to create audio transcripts. Despite its convenience, the performance of ASR is unstable in disturbing environments; for instance, indoor signals are usually corrupted by reverberation (reverb), resulting in diminished performance in ASR. A type of solution is to construct an acoustic dereverberation (dereverb) model to pre-process the original signals before submitting them to ASR. However, the acoustic properties of the output signal of the dereverb model differ from that of the training dataset for ASR, resulting in a decline in performance. This paper optimizes the aforementioned structure from four aspects: signal classification, reverberation removal, data mismatch removal in ASR, and ensemble algorithms. With the proposed sentence-level fusion (SLF) and word-level fusion (WLF) ensemble algorithms, a CER of 7.23% was reached in the mixture test set of the reverberated and clean Aishell1 compared to the single model, achieving a reduction in the CER by 20.72%.

KW - automatic speech recognition

KW - dereverberation

KW - model ensemble

KW - new structure

KW - string confusion network

UR - https://www.scopus.com/pages/publications/85148636937

U2 - 10.1109/ISCSLP57327.2022.10038145

DO - 10.1109/ISCSLP57327.2022.10038145

M3 - 会议稿件

AN - SCOPUS:85148636937

T3 - 2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022

SP - 165

EP - 169

BT - 2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022

A2 - Lee, Kong Aik

A2 - Lee, Hung-yi

A2 - Lu, Yanfeng

A2 - Dong, Minghui

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022

Y2 - 11 December 2022 through 14 December 2022

ER -

Liao YL, Lin CH, Lyu RY, Jang JSR. Improving ASR in Reverberant Environments. In Lee KA, Lee H, Lu Y, Dong M, editors, 2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022. Institute of Electrical and Electronics Engineers Inc. 2022. p. 165-169. (2022 13th International Symposium on Chinese Spoken Language Processing, ISCSLP 2022). doi: 10.1109/ISCSLP57327.2022.10038145

Improving ASR in Reverberant Environments

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this