Publications - SFB 1102

Zaitova, Iuliia; Abdullah, Badr M.; Xue, Wei; Klakow, Dietrich; Möbius, Bernd; Avgustinova, Tania

It’s not a walk in the park! Challenges of idiom translation in speech-to-text systems Inproceedings

Che, Wanxiang; Nabende, Joyce; Shutova, Ekaterina; Taher Pilehvar, Mohammad (Ed.): Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, Association for Computational Linguistics, pp. 31310-31322, Vienna, Austria, 2025, ISBN 979-8-89176-251-0.

Abstract
|
Links
|
BibTeX

Idioms are defined as a group of words with a figurative meaning not deducible from their individual components. Although modern machine translation systems have made remarkable progress, translating idioms remains a major challenge, especially for speech-to-text systems, where research on this topic is notably sparse. In this paper, we systematically evaluate idiom translation as compared to conventional news translation in both text-to-text machine translation (MT) and speech-to-text translation (SLT) systems across two language pairs (German to English, Russian to English). We compare state-of-the-art end-to-end SLT systems (SeamlessM4T SLT-to-text, Whisper Large v3) with MT systems (SeamlessM4T SLT-to-text, No Language Left Behind), Large Language Models (DeepSeek, LLaMA) and cascaded alternatives. Our results reveal that SLT systems experience a pronounced performance drop on idiomatic data, often reverting to literal translations even in higher layers, whereas MT systems and Large Language Models demonstrate better handling of idioms. These findings underscore the need for idiom-specific strategies and improved internal representations in SLT architectures.

@inproceedings{Zaitova/etal:2025,
title = {It’s not a walk in the park! Challenges of idiom translation in speech-to-text systems},
author = {Iuliia Zaitova and Badr M. Abdullah and Wei Xue and Dietrich Klakow and Bernd M{\"o}bius and Tania Avgustinova},
editor = {Wanxiang Che and Joyce Nabende and Ekaterina Shutova and Mohammad Taher Pilehvar},
url = {https://aclanthology.org/2025.acl-long.1512/},
year = {2025},
date = {2025},
booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics},
isbn = {979-8-89176-251-0},
pages = {31310-31322},
publisher = {Association for Computational Linguistics},
address = {Vienna, Austria},
abstract = {

Idioms are defined as a group of words with a figurative meaning not deducible from their individual components. Although modern machine translation systems have made remarkable progress, translating idioms remains a major challenge, especially for speech-to-text systems, where research on this topic is notably sparse. In this paper, we systematically evaluate idiom translation as compared to conventional news translation in both text-to-text machine translation (MT) and speech-to-text translation (SLT) systems across two language pairs (German to English, Russian to English). We compare state-of-the-art end-to-end SLT systems (SeamlessM4T SLT-to-text, Whisper Large v3) with MT systems (SeamlessM4T SLT-to-text, No Language Left Behind), Large Language Models (DeepSeek, LLaMA) and cascaded alternatives. Our results reveal that SLT systems experience a pronounced performance drop on idiomatic data, often reverting to literal translations even in higher layers, whereas MT systems and Large Language Models demonstrate better handling of idioms. These findings underscore the need for idiom-specific strategies and improved internal representations in SLT architectures.

},
pubstate = {published},
type = {inproceedings}
}