Efficient Speech Quality Assessment Using Self-Supervised Framewise Embeddings

Karl El Hajal; Zihan Wu; Neil Scheidwasser-Clow; Gasser Elbanna; Milos Cernak

doi:10.1109/ICASSP49357.2023.10095132

Efficient Speech Quality Assessment Using Self-Supervised Framewise Embeddings

Karl El Hajal^*, Zihan Wu, Neil Scheidwasser-Clow, Gasser Elbanna, Milos Cernak

^*Corresponding author for this work

Section of Epidemiology

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

2 Citations (Scopus)

Abstract

Automatic speech quality assessment is essential for audio researchers, developers, speech and language pathologists, and system quality engineers. The current state-of-the-art systems are based on framewise speech features (hand-engineered or learnable) combined with time dependency modeling. This paper proposes an efficient system with results comparable to the best performing model in the ConferencingSpeech 2022 challenge. Our proposed system is characterized by a smaller number of parameters (40-60x), fewer FLOPS (100x), lower memory consumption (10-15x), and lower latency (30x). Speech quality practitioners can therefore iterate much faster, deploy the system on resource-limited hardware, and, overall, the proposed system contributes to sustainable machine learning. The paper also concludes that framewise embeddings outperform utterance-level embeddings and that multi-task training with acoustic conditions modeling does not degrade speech quality prediction while providing better interpretation.

Original language	English
Title of host publication	Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing
Number of pages	5
Publisher	IEEE
Publication date	2023
ISBN (Electronic)	978-1-7281-6327-7
DOIs	https://doi.org/10.1109/ICASSP49357.2023.10095132
Publication status	Published - 2023
Event	48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023 - Rhodes Island, Greece Duration: 4 Jun 2023 → 10 Jun 2023

Conference

Conference	48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023
Country/Territory	Greece
City	Rhodes Island
Period	04/06/2023 → 10/06/2023
Sponsor	IEEE, IEEE Signal Processing Society

Series	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume	2023-June
ISSN	1520-6149

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

audio embeddings
deep neural networks
self-supervised learning
speech quality assessment
transformers

Access to Document

10.1109/ICASSP49357.2023.10095132

Cite this

El Hajal, K., Wu, Z., Scheidwasser-Clow, N., Elbanna, G., & Cernak, M. (2023). Efficient Speech Quality Assessment Using Self-Supervised Framewise Embeddings. In Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing IEEE. ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings Vol. 2023-June https://doi.org/10.1109/ICASSP49357.2023.10095132

Efficient Speech Quality Assessment Using Self-Supervised Framewise Embeddings. / El Hajal, Karl; Wu, Zihan; Scheidwasser-Clow, Neil; Elbanna, Gasser; Cernak, Milos.

Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 2023. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, Vol. 2023-June).

Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review

El Hajal, K, Wu, Z, Scheidwasser-Clow, N, Elbanna, G & Cernak, M 2023, Efficient Speech Quality Assessment Using Self-Supervised Framewise Embeddings. in Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, vol. 2023-June, 48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023, Rhodes Island, Greece, 04/06/2023. https://doi.org/10.1109/ICASSP49357.2023.10095132

El Hajal K, Wu Z, Scheidwasser-Clow N, Elbanna G, Cernak M. Efficient Speech Quality Assessment Using Self-Supervised Framewise Embeddings. In Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE. 2023. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, Vol. 2023-June). https://doi.org/10.1109/ICASSP49357.2023.10095132

El Hajal, Karl ; Wu, Zihan ; Scheidwasser-Clow, Neil ; Elbanna, Gasser ; Cernak, Milos. / Efficient Speech Quality Assessment Using Self-Supervised Framewise Embeddings. Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 2023. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, Vol. 2023-June).

@inproceedings{2db38fc9f96d413bbf46cd1d5e6e6798,

title = "Efficient Speech Quality Assessment Using Self-Supervised Framewise Embeddings",

abstract = "Automatic speech quality assessment is essential for audio researchers, developers, speech and language pathologists, and system quality engineers. The current state-of-the-art systems are based on framewise speech features (hand-engineered or learnable) combined with time dependency modeling. This paper proposes an efficient system with results comparable to the best performing model in the ConferencingSpeech 2022 challenge. Our proposed system is characterized by a smaller number of parameters (40-60x), fewer FLOPS (100x), lower memory consumption (10-15x), and lower latency (30x). Speech quality practitioners can therefore iterate much faster, deploy the system on resource-limited hardware, and, overall, the proposed system contributes to sustainable machine learning. The paper also concludes that framewise embeddings outperform utterance-level embeddings and that multi-task training with acoustic conditions modeling does not degrade speech quality prediction while providing better interpretation.",

keywords = "audio embeddings, deep neural networks, self-supervised learning, speech quality assessment, transformers",

author = "{El Hajal}, Karl and Zihan Wu and Neil Scheidwasser-Clow and Gasser Elbanna and Milos Cernak",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023 ; Conference date: 04-06-2023 Through 10-06-2023",

year = "2023",

doi = "10.1109/ICASSP49357.2023.10095132",

language = "English",

series = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",

publisher = "IEEE",

booktitle = "Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing",

}

TY - GEN

T1 - Efficient Speech Quality Assessment Using Self-Supervised Framewise Embeddings

AU - El Hajal, Karl

AU - Wu, Zihan

AU - Scheidwasser-Clow, Neil

AU - Elbanna, Gasser

AU - Cernak, Milos

PY - 2023

Y1 - 2023

N2 - Automatic speech quality assessment is essential for audio researchers, developers, speech and language pathologists, and system quality engineers. The current state-of-the-art systems are based on framewise speech features (hand-engineered or learnable) combined with time dependency modeling. This paper proposes an efficient system with results comparable to the best performing model in the ConferencingSpeech 2022 challenge. Our proposed system is characterized by a smaller number of parameters (40-60x), fewer FLOPS (100x), lower memory consumption (10-15x), and lower latency (30x). Speech quality practitioners can therefore iterate much faster, deploy the system on resource-limited hardware, and, overall, the proposed system contributes to sustainable machine learning. The paper also concludes that framewise embeddings outperform utterance-level embeddings and that multi-task training with acoustic conditions modeling does not degrade speech quality prediction while providing better interpretation.

AB - Automatic speech quality assessment is essential for audio researchers, developers, speech and language pathologists, and system quality engineers. The current state-of-the-art systems are based on framewise speech features (hand-engineered or learnable) combined with time dependency modeling. This paper proposes an efficient system with results comparable to the best performing model in the ConferencingSpeech 2022 challenge. Our proposed system is characterized by a smaller number of parameters (40-60x), fewer FLOPS (100x), lower memory consumption (10-15x), and lower latency (30x). Speech quality practitioners can therefore iterate much faster, deploy the system on resource-limited hardware, and, overall, the proposed system contributes to sustainable machine learning. The paper also concludes that framewise embeddings outperform utterance-level embeddings and that multi-task training with acoustic conditions modeling does not degrade speech quality prediction while providing better interpretation.

KW - audio embeddings

KW - deep neural networks

KW - self-supervised learning

KW - speech quality assessment

KW - transformers

U2 - 10.1109/ICASSP49357.2023.10095132

DO - 10.1109/ICASSP49357.2023.10095132

M3 - Article in proceedings

AN - SCOPUS:85164576460

T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

BT - Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing

PB - IEEE

T2 - 48th IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2023

Y2 - 4 June 2023 through 10 June 2023

ER -