Publications

Lemke, Tyll Robin; Schäfer, Lisa; Reich, Ingo

Can identity conditions on ellipsis be explained by processing principles? Inproceedings Forthcoming

Proceedings of Linguistic Evidence 2020, 2022.

@inproceedings{lemke.etalidentity,
title = {Can identity conditions on ellipsis be explained by processing principles?},
author = {Tyll Robin Lemke and Lisa Sch{\"a}fer and Ingo Reich},
year = {2022},
date = {2022},
booktitle = {Proceedings of Linguistic Evidence 2020},
pubstate = {forthcoming},
type = {inproceedings}
}

Copy BibTeX to Clipboard

Project:   B3

Lemke, Tyll Robin; Reich, Ingo; Schäfer, Lisa

Questions under discussion, salience and the acceptability of fragments Incollection Forthcoming

Konietzko, Andreas; Winkler, Susanne;  (Ed.): Information Structure and Discourse in Generative Grammar: Mechanisms and Processes, De Gruyter Mouton, Berlin; Boston, 2022.

@incollection{lemke.etalquestions,
title = {Questions under discussion, salience and the acceptability of fragments},
author = {Tyll Robin Lemke and Ingo Reich and Lisa Sch{\"a}fer},
editor = {Andreas Konietzko and Susanne Winkler},
year = {2022},
date = {2022},
booktitle = {Information Structure and Discourse in Generative Grammar: Mechanisms and Processes},
publisher = {De Gruyter Mouton},
address = {Berlin; Boston},
pubstate = {forthcoming},
type = {incollection}
}

Copy BibTeX to Clipboard

Project:   B3

Calvillo, Jesús; Brouwer, Harm; Crocker, Matthew W.

Semantic Systematicity in Connectionist Language Production Journal Article

Information (Special issue on Neural Natural Language Generation), 12, pp. 329, 2021.

Decades of studies trying to define the extent to which artificial neural networks can exhibit systematicity suggest that systematicity can be achieved by connectionist models but not by default. Here we present a novel connectionist model of sentence production that employs rich situation model representations originally proposed for modeling systematicity in comprehension. The high performance of our model demonstrates that such representations are also well suited to model language production. Furthermore, the model can produce multiple novel sentences for previously unseen situations, including in a different voice (actives vs. passive) and with words in new syntactic roles, thus demonstrating semantic and syntactic generalization and arguably systematicity. Our results provide yet further evidence that such connectionist approaches can achieve systematicity, in production as well as comprehension. We propose our positive results to be a consequence of the regularities of the microworld from which the semantic representations are derived, which provides a sufficient structure from which the neural network can interpret novel inputs.

@article{Calvillo2021semantic,
title = {Semantic Systematicity in Connectionist Language Production},
author = {Jesús Calvillo and Harm Brouwer and Matthew W. Crocker},
url = {https://doi.org/10.3390/info12080329},
doi = {https://doi.org/10.3390/info12080329},
year = {2021},
date = {2021},
journal = {Information (Special issue on Neural Natural Language Generation)},
pages = {329},
volume = {12},
number = {8},
abstract = {Decades of studies trying to define the extent to which artificial neural networks can exhibit systematicity suggest that systematicity can be achieved by connectionist models but not by default. Here we present a novel connectionist model of sentence production that employs rich situation model representations originally proposed for modeling systematicity in comprehension. The high performance of our model demonstrates that such representations are also well suited to model language production. Furthermore, the model can produce multiple novel sentences for previously unseen situations, including in a different voice (actives vs. passive) and with words in new syntactic roles, thus demonstrating semantic and syntactic generalization and arguably systematicity. Our results provide yet further evidence that such connectionist approaches can achieve systematicity, in production as well as comprehension. We propose our positive results to be a consequence of the regularities of the microworld from which the semantic representations are derived, which provides a sufficient structure from which the neural network can interpret novel inputs.},
pubstate = {published},
type = {article}
}

Copy BibTeX to Clipboard

Project:   C3

Yuen, Ivan; Xu Rattanasone, Nan; Schmidt, Elaine; Macdonald, Gretel; Holt, Rebecca; Demuth, Katherine

Five-year-olds produce prosodic cues to distinguish compounds from lists in Australian English Journal Article

Journal of Child Language, 48, Cambridge University Press, pp. 110-128, 2021.

Although previous research has indicated that five-year-olds can use acoustic cues to disambiguate compounds (N1 + N2) from lists (N1, N2) (e.g., ‘icecream’ vs. ‘ice, cream’) (Yoshida & Katz, 2004, 2006), their productions are not yet fully adult-like (Wells, Peppé & Goulandris, 2004). The goal of this study was to examine this issue in Australian English-speaking children, with a focus on their use of F0, word duration, and pauses. Twenty-four five-year-olds and 20 adults participated in an elicited production experiment. Like adults, children produced distinct F0 patterns for the two structures. They also used longer word durations and more pauses in lists compared to compounds, indicating the presence of a boundary in lists. However, unlike adults, they also inappropriately inserted more pauses within the compound, suggesting the presence of a boundary in compounds as well. The implications for understanding children’s developing knowledge of how to map acoustic cues to prosodic structures are discussed.

@article{YUENetal2020cues,
title = {Five-year-olds produce prosodic cues to distinguish compounds from lists in Australian English},
author = {Ivan Yuen and Nan Xu Rattanasone and Elaine Schmidt and Gretel Macdonald and Rebecca Holt and Katherine Demuth},
url = {https://doi.org/10.1017/S0305000920000227},
doi = {https://doi.org/10.1017/S0305000920000227},
year = {2021},
date = {2021},
journal = {Journal of Child Language},
pages = {110-128},
publisher = {Cambridge University Press},
volume = {48},
number = {1},
abstract = {

Although previous research has indicated that five-year-olds can use acoustic cues to disambiguate compounds (N1 + N2) from lists (N1, N2) (e.g., ‘ice-cream’ vs. ‘ice, cream’) (Yoshida & Katz, 2004, 2006), their productions are not yet fully adult-like (Wells, Pepp{\'e} & Goulandris, 2004). The goal of this study was to examine this issue in Australian English-speaking children, with a focus on their use of F0, word duration, and pauses. Twenty-four five-year-olds and 20 adults participated in an elicited production experiment. Like adults, children produced distinct F0 patterns for the two structures. They also used longer word durations and more pauses in lists compared to compounds, indicating the presence of a boundary in lists. However, unlike adults, they also inappropriately inserted more pauses within the compound, suggesting the presence of a boundary in compounds as well. The implications for understanding children's developing knowledge of how to map acoustic cues to prosodic structures are discussed.
},
pubstate = {published},
type = {article}
}

Copy BibTeX to Clipboard

Project:   C1

Gessinger, Iona; Möbius, Bernd; Le Maguer, Sébastien; Raveh, Eran; Steiner, Ingmar

Phonetic accommodation in interaction with a virtual language learning tutor: A Wizard-of-Oz study Journal Article

Journal of Phonetics, 86, pp. 101029, 2021.

We present a Wizard-of-Oz experiment examining phonetic accommodation of human interlocutors in the context of human-computer interaction. Forty-two native speakers of German engaged in dynamic spoken interaction with a simulated virtual tutor for learning the German language called Mirabella. Mirabella was controlled by the experimenter and used either natural or hidden Markov model-based synthetic speech to communicate with the participants. In the course of four tasks, the participants’ accommodating behavior with respect to wh-question realization and allophonic variation in German was tested. The participants converged to Mirabella with respect to modified wh-question intonation, i.e., rising F0 contour and nuclear pitch accent on the interrogative pronoun, and the allophonic contrast [ɪç] vs. [ɪk] occurring in the word ending -ig. They did not accommodate to the allophonic contrast [ɛː] vs. [eː] as a realization of the long vowel -ä-. The results did not differ between the experimental groups that communicated with either the natural or the synthetic speech version of Mirabella. Testing the influence of the “Big Five” personality traits on the accommodating behavior revealed a tendency for neuroticism to influence the convergence of question intonation. On the level of individual speakers, we found considerable variation with respect to the degree and direction of accommodation. We conclude that phonetic accommodation on the level of local prosody and segmental pronunciation occurs in users of spoken dialog systems, which could be exploited in the context of computer-assisted language learning.

@article{Gessinger/etal:2021a,
title = {Phonetic accommodation in interaction with a virtual language learning tutor: A Wizard-of-Oz study},
author = {Iona Gessinger and Bernd M{\"o}bius and S{\'e}bastien Le Maguer and Eran Raveh and Ingmar Steiner},
url = {https://doi.org/10.1016/j.wocn.2021.101029},
doi = {https://doi.org/10.1016/j.wocn.2021.101029},
year = {2021},
date = {2021},
journal = {Journal of Phonetics},
pages = {101029},
volume = {86},
abstract = {We present a Wizard-of-Oz experiment examining phonetic accommodation of human interlocutors in the context of human-computer interaction. Forty-two native speakers of German engaged in dynamic spoken interaction with a simulated virtual tutor for learning the German language called Mirabella. Mirabella was controlled by the experimenter and used either natural or hidden Markov model-based synthetic speech to communicate with the participants. In the course of four tasks, the participants’ accommodating behavior with respect to wh-question realization and allophonic variation in German was tested. The participants converged to Mirabella with respect to modified wh-question intonation, i.e., rising F0 contour and nuclear pitch accent on the interrogative pronoun, and the allophonic contrast [ɪç] vs. [ɪk] occurring in the word ending -ig. They did not accommodate to the allophonic contrast [ɛː] vs. [eː] as a realization of the long vowel -{\"a}-. The results did not differ between the experimental groups that communicated with either the natural or the synthetic speech version of Mirabella. Testing the influence of the “Big Five” personality traits on the accommodating behavior revealed a tendency for neuroticism to influence the convergence of question intonation. On the level of individual speakers, we found considerable variation with respect to the degree and direction of accommodation. We conclude that phonetic accommodation on the level of local prosody and segmental pronunciation occurs in users of spoken dialog systems, which could be exploited in the context of computer-assisted language learning.},
pubstate = {published},
type = {article}
}

Copy BibTeX to Clipboard

Project:   C1

Proceedings for the First Workshop on Modelling Translation: Translatology in the Digital Age Proceeding

Bizzoni, Yuri; Teich, Elke; España i Bonet, Cristina; van Genabith, Josef;  (Ed.): Association for Computational Linguistics, online, 2021.

@proceeding{motra-2021-modelling,
title = {Proceedings for the First Workshop on Modelling Translation: Translatology in the Digital Age},
author = {},
editor = {Yuri Bizzoni and Elke Teich and Cristina Espa{\~n}a i Bonet and Josef van Genabith},
url = {https://aclanthology.org/2021.motra-1.0/},
year = {2021},
date = {2021},
publisher = {Association for Computational Linguistics},
address = {online},
pubstate = {published},
type = {proceeding}
}

Copy BibTeX to Clipboard

Project:   B7

Przybyl, Heike; Karakanta, Alina; Menzel, Katrin; Teich, Elke

Exploring linguistic variation in mediated discourse: translation vs. interpreting Book Chapter

Kajzer-Wietrzny, Marta; Bernardini, Silvia; Ferraresi, Adriano; Ivaska, Ilmari;  (Ed.): Empirical investigations into the forms of mediated discourse at the European Parliament, Language Science Press, Berlin, 2021.

@inbook{Przybyl2021exploring,
title = {Exploring linguistic variation in mediated discourse: translation vs. interpreting},
author = {Heike Przybyl and Alina Karakanta and Katrin Menzel and Elke Teich},
editor = {Marta Kajzer-Wietrzny and Silvia Bernardini and Adriano Ferraresi and Ilmari Ivaska},
year = {2021},
date = {2021},
booktitle = {Empirical investigations into the forms of mediated discourse at the European Parliament},
publisher = {Language Science Press},
address = {Berlin},
pubstate = {published},
type = {inbook}
}

Copy BibTeX to Clipboard

Project:   B7

Karakanta, Alina; Przybyl, Heike; Teich, Elke

Exploring variation in translation with probabilistic language models Incollection

Lavid-López, Julia; Maíz-Arévalo, Carmen; Zamorano-Mansilla, Juan Rafael;  (Ed.): Corpora in Translation and Contrastive Research in the Digital Age: Recent advances and explorations, 158, Benjamins, pp. 308-323, Amsterdam, 2021.

While some authors have suggested that translationese fingerprints are universal, others have shown that there is a fair amount of variation among translations due to source language shining through, translation type or translation mode. In our work, we attempt to gain empirical insights into variation in translation, focusing here on translation mode (translation vs. interpreting). Our goal is to discover features of translationese and interpretese that distinguish translated and interpreted output from comparable original text/speech as well as from each other at different linguistic levels. We use relative entropy (Kullback-Leibler Divergence) and visualization with word clouds. Our analysis shows differences in typical words between originals vs. non-originals as well as between translation modes both at lexical and grammatical levels.

@incollection{KarakantaEtAl2021,
title = {Exploring variation in translation with probabilistic language models},
author = {Alina Karakanta and Heike Przybyl and Elke Teich},
editor = {Julia Lavid-López and Carmen Ma{\'i}z-Ar{\'e}valo and Juan Rafael Zamorano-Mansilla},
url = {https://doi.org/10.1075/btl.158.12kar},
doi = {https://doi.org/10.1075/btl.158.12kar},
year = {2021},
date = {2021},
booktitle = {Corpora in Translation and Contrastive Research in the Digital Age: Recent advances and explorations},
pages = {308-323},
publisher = {Benjamins},
address = {Amsterdam},
abstract = {While some authors have suggested that translationese fingerprints are universal, others have shown that there is a fair amount of variation among translations due to source language shining through, translation type or translation mode. In our work, we attempt to gain empirical insights into variation in translation, focusing here on translation mode (translation vs. interpreting). Our goal is to discover features of translationese and interpretese that distinguish translated and interpreted output from comparable original text/speech as well as from each other at different linguistic levels. We use relative entropy (Kullback-Leibler Divergence) and visualization with word clouds. Our analysis shows differences in typical words between originals vs. non-originals as well as between translation modes both at lexical and grammatical levels.},
pubstate = {published},
type = {incollection}
}

Copy BibTeX to Clipboard

Project:   B7

Lapshinova-Koltunski, Ekaterina; Przybyl, Heike; Bizzoni, Yuri

Tracing variation in discourse connectives in translation and interpreting through neural semantic spaces Inproceedings

Proceedings of the 2nd Workshop on Computational Approaches to Discourse CODI, pp. 134-142, Punta Cana, Dominican Republic and Online, 2021.

In the present paper, we explore lexical contexts of discourse markers in translation and interpreting on the basis of word embeddings. Our special interest is on contextual variation of the same discourse markers in (written) translation vs. (simultaneous) interpreting. To explore this variation at the lexical level, we use a data-driven approach: we compare bilingual neural word embeddings trained on source-to- translation and source-tointerpreting aligned corpora. Our results show more variation of semantically related items in translation spaces vs. interpreting ones and a more consistent use of fewer connectives in interpreting. We also observe different trends with regard to the discourse relation types.

@inproceedings{LapshinovaEtAl2021codi,
title = {Tracing variation in discourse connectives in translation and interpreting through neural semantic spaces},
author = {Ekaterina Lapshinova-Koltunski and Heike Przybyl and Yuri Bizzoni},
url = {https://aclanthology.org/2021.codi-main.13.pdf},
year = {2021},
date = {2021},
booktitle = {Proceedings of the 2nd Workshop on Computational Approaches to Discourse CODI},
pages = {134-142},
address = {Punta Cana, Dominican Republic and Online},
abstract = {In the present paper, we explore lexical contexts of discourse markers in translation and interpreting on the basis of word embeddings. Our special interest is on contextual variation of the same discourse markers in (written) translation vs. (simultaneous) interpreting. To explore this variation at the lexical level, we use a data-driven approach: we compare bilingual neural word embeddings trained on source-to- translation and source-tointerpreting aligned corpora. Our results show more variation of semantically related items in translation spaces vs. interpreting ones and a more consistent use of fewer connectives in interpreting. We also observe different trends with regard to the discourse relation types.},
pubstate = {published},
type = {inproceedings}
}

Copy BibTeX to Clipboard

Project:   B7

Amponsah-Kaakyire, Kwabena; Pylypenko, Daria; España i Bonet, Cristina; van Genabith, Josef

Do not Rely on Relay Translations: Multilingual Parallel Direct Europarl Inproceedings

Proceedings of the Workshop on Modelling Translation: Translatology in the Digital Age (MoTra21), International Committee on Computational Linguistics, pp. 1-7, Iceland (Online), 2021.

Translationese data is a scarce and valuable resource. Traditionally, the proceedings of the European Parliament have been used for studying translationese phenomena since their metadata allows to distinguish between original and translated texts. However, translations are not always direct and we hypothesise that a pivot (also called ”relay”) language might alter the conclusions on translationese effects. In this work, we (i) isolate translations that have been done without an intermediate language in the Europarl proceedings from those that might have used a pivot language, and (ii) build comparable and parallel corpora with data aligned across multiple languages that therefore can be used for both machine translation and translation studies.

@inproceedings{AmposahEtal:MOTRA:2021,
title = {Do not Rely on Relay Translations: Multilingual Parallel Direct Europarl},
author = {Kwabena Amponsah-Kaakyire and Daria Pylypenko and Cristina Espa{\~n}a i Bonet and Josef van Genabith},
url = {https://aclanthology.org/2021.motra-1.1/},
year = {2021},
date = {2021},
booktitle = {Proceedings of the Workshop on Modelling Translation: Translatology in the Digital Age (MoTra21)},
pages = {1-7},
publisher = {International Committee on Computational Linguistics},
address = {Iceland (Online)},
abstract = {Translationese data is a scarce and valuable resource. Traditionally, the proceedings of the European Parliament have been used for studying translationese phenomena since their metadata allows to distinguish between original and translated texts. However, translations are not always direct and we hypothesise that a pivot (also called ”relay”) language might alter the conclusions on translationese effects. In this work, we (i) isolate translations that have been done without an intermediate language in the Europarl proceedings from those that might have used a pivot language, and (ii) build comparable and parallel corpora with data aligned across multiple languages that therefore can be used for both machine translation and translation studies.},
pubstate = {published},
type = {inproceedings}
}

Copy BibTeX to Clipboard

Project:   B6

Zhu, Dawei; Mogadala, Aditya; Klakow, Dietrich

Image manipulation with natural language using Two-sided Attentive Conditional Generative Adversarial Network Journal Article

Neural Networks, 136, pp. 207-217, 2021, ISSN 0893-6080.

Altering the content of an image with photo editing tools is a tedious task for an inexperienced user. Especially, when modifying the visual attributes of a specific object in an image without affecting other constituents such as background etc. To simplify the process of image manipulation and to provide more control to users, it is better to utilize a simpler interface like natural language. Therefore, in this paper, we address the challenge of manipulating images using natural language description. We propose the Two-sidEd Attentive conditional Generative Adversarial Network (TEA-cGAN) to generate semantically manipulated images while preserving other contents such as background intact. TEA-cGAN uses fine-grained attention both in the generator and discriminator of Generative Adversarial Network (GAN) based framework at different scales. Experimental results show that TEA-cGAN which generates 128×128 and 256×256 resolution images outperforms existing methods on CUB and Oxford-102 datasets both quantitatively and qualitatively.

@article{zhumogadala:2020,
title = {Image manipulation with natural language using Two-sided Attentive Conditional Generative Adversarial Network},
author = {Dawei Zhu and Aditya Mogadala and Dietrich Klakow},
url = {https://www.sciencedirect.com/science/article/pii/S0893608020303257},
doi = {https://doi.org/10.1016/j.neunet.2020.09.002},
year = {2021},
date = {2021},
journal = {Neural Networks},
pages = {207-217},
volume = {136},
abstract = {Altering the content of an image with photo editing tools is a tedious task for an inexperienced user. Especially, when modifying the visual attributes of a specific object in an image without affecting other constituents such as background etc. To simplify the process of image manipulation and to provide more control to users, it is better to utilize a simpler interface like natural language. Therefore, in this paper, we address the challenge of manipulating images using natural language description. We propose the Two-sidEd Attentive conditional Generative Adversarial Network (TEA-cGAN) to generate semantically manipulated images while preserving other contents such as background intact. TEA-cGAN uses fine-grained attention both in the generator and discriminator of Generative Adversarial Network (GAN) based framework at different scales. Experimental results show that TEA-cGAN which generates 128x128 and 256x256 resolution images outperforms existing methods on CUB and Oxford-102 datasets both quantitatively and qualitatively.},
pubstate = {published},
type = {article}
}

Copy BibTeX to Clipboard

Project:   B4

Mogadala, Aditya; Kalimuthu, Marimuthu; Klakow, Dietrich

Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods Journal Article

Journal of Artificial Intelligence Research, 71, pp. 1183-1317, 2021.

The interest in Artificial Intelligence (AI) and its applications has seen unprecedented growth in the last few years. This success can be partly attributed to the advancements made in the sub-fields of AI such as Machine Learning (ML), Computer Vision (CV), and Natural Language Processing (NLP). The largest of the growths in these fields has been made possible with deep learning, a sub-area of machine learning, which uses the principles of artificial neural networks. This has created significant interest in the integration of vision and language. The tasks are designed such that they perfectly embrace the ideas of deep learning. In this survey, we focus on ten prominent tasks that integrate language and vision by discussing their problem formulations, methods, existing datasets, evaluation measures, and compare the results obtained with corresponding state-of-the-art methods. Our efforts go beyond earlier surveys which are either task-specific or concentrate only on one type of visual content, i.e., image or video. Furthermore, we also provide some potential future directions in this field of research with an anticipation that this survey brings in innovative thoughts and ideas to address the existing challenges and build new applications.

@article{mogadala2021trends,
title = {Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods},
author = {Aditya Mogadala and Marimuthu Kalimuthu and Dietrich Klakow},
year = {2021},
date = {2021},
journal = {Journal of Artificial Intelligence Research},
pages = {1183-1317},
volume = {71},
abstract = {The interest in Artificial Intelligence (AI) and its applications has seen unprecedented growth in the last few years. This success can be partly attributed to the advancements made in the sub-fields of AI such as Machine Learning (ML), Computer Vision (CV), and Natural Language Processing (NLP). The largest of the growths in these fields has been made possible with deep learning, a sub-area of machine learning, which uses the principles of artificial neural networks. This has created significant interest in the integration of vision and language. The tasks are designed such that they perfectly embrace the ideas of deep learning. In this survey, we focus on ten prominent tasks that integrate language and vision by discussing their problem formulations, methods, existing datasets, evaluation measures, and compare the results obtained with corresponding state-of-the-art methods. Our efforts go beyond earlier surveys which are either task-specific or concentrate only on one type of visual content, i.e., image or video. Furthermore, we also provide some potential future directions in this field of research with an anticipation that this survey brings in innovative thoughts and ideas to address the existing challenges and build new applications.},
pubstate = {published},
type = {article}
}

Copy BibTeX to Clipboard

Project:   B4

Shi, Wei; Demberg, Vera

Entity Enhancement for Implicit Discourse Relation Classification in the Biomedical Domain Inproceedings

Proceedings of the Joint Conference of the 59th Annual Meeting of theAssociation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021), 2021.

Implicit discourse relation classification is a challenging task, in particular when the text domain is different from the standard Penn Discourse Treebank (PDTB; Prasad et al., 2008) training corpus domain (Wall Street Journal in 1990s). We here tackle the task of implicit discourse relation classification on the biomedical domain, for which the Biomedical Discourse Relation Bank (BioDRB; Prasad et al., 2011) is available. We show that entity information can be used to improve discourse relational argument representation. In a first step, we show that explicitly marked instances that are content-wise similar to the target relations can be used to achieve good performance in the cross-domain setting using a simple unsupervised voting pipeline. As a further step, we show that with the linked entity information from the first step, a transformer which is augmented with entity-related information (KBERT; Liu et al., 2020) sets the new state of the art performance on the dataset, outperforming the large pre-trained BioBERT (Lee et al., 2020) model by 2% points.

@inproceedings{shi2021entity,
title = {Entity Enhancement for Implicit Discourse Relation Classification in the Biomedical Domain},
author = {Wei Shi and Vera Demberg},
url = {https://aclanthology.org/2021.acl-short.116.pdf},
year = {2021},
date = {2021},
booktitle = {Proceedings of the Joint Conference of the 59th Annual Meeting of theAssociation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021)},
abstract = {Implicit discourse relation classification is a challenging task, in particular when the text domain is different from the standard Penn Discourse Treebank (PDTB; Prasad et al., 2008) training corpus domain (Wall Street Journal in 1990s). We here tackle the task of implicit discourse relation classification on the biomedical domain, for which the Biomedical Discourse Relation Bank (BioDRB; Prasad et al., 2011) is available. We show that entity information can be used to improve discourse relational argument representation. In a first step, we show that explicitly marked instances that are content-wise similar to the target relations can be used to achieve good performance in the cross-domain setting using a simple unsupervised voting pipeline. As a further step, we show that with the linked entity information from the first step, a transformer which is augmented with entity-related information (KBERT; Liu et al., 2020) sets the new state of the art performance on the dataset, outperforming the large pre-trained BioBERT (Lee et al., 2020) model by 2% points.},
pubstate = {published},
type = {inproceedings}
}

Copy BibTeX to Clipboard

Project:   B2

Marchal, Marian; Scholman, Merel; Demberg, Vera

Semi-automatic discourse annotation in a low-resource language: Developing a connective lexicon for Nigerian Pidgin Inproceedings

Proceedings of the Second Workshop on Computational Approaches to Discourse (CODI 2021), 2021.

@inproceedings{marchal2021,
title = {Semi-automatic discourse annotation in a low-resource language: Developing a connective lexicon for Nigerian Pidgin},
author = {Marian Marchal and Merel Scholman and Vera Demberg},
year = {2021},
date = {2021},
booktitle = {Proceedings of the Second Workshop on Computational Approaches to Discourse (CODI 2021)},
pubstate = {published},
type = {inproceedings}
}

Copy BibTeX to Clipboard

Project:   B2

Meßmer, Julia; Bader, Regine; Mecklinger, Axel

The more you know: Schema congruency supports associative encoding of novel compound words: Evidence from event-related potentials Journal Article

Brain and Cognition, 155, pp. 105813, 2021.

We aimed to investigate the neurocognitive mechanisms of event congruency with prior (schema) knowledge for the learning of novel compound words. Event-related potentials (ERPs) were recorded during an incidental learning task, in which novel noun-noun compounds were presented in a semantically congruent context, enabling schema-supported processing, or in a neutral context. As expected, associative memory performance was better for compounds preceded by a congruent context. Although the N400 was attenuated in the congruent condition, subsequent memory effects (SMEs) in the N400 time interval did not differ across conditions, suggesting that the processes reflected in the N400 cannot account for the memory advantage in the congruent condition. However, a parietal SME was obtained for compounds preceded by a congruent context, only, which we interpret as reflecting the schema-supported formation of a conceptual compound representation. A late frontal SME was obtained in both conditions, presumably reflecting the more general inter-item associative encoding of compound constituents.

@article{Messmer2021,
title = {The more you know: Schema congruency supports associative encoding of novel compound words: Evidence from event-related potentials},
author = {Julia Me{\ss}mer and Regine Bader and Axel Mecklinger},
year = {2021},
date = {2021},
journal = {Brain and Cognition},
pages = {105813},
volume = {155},
abstract = {We aimed to investigate the neurocognitive mechanisms of event congruency with prior (schema) knowledge for the learning of novel compound words. Event-related potentials (ERPs) were recorded during an incidental learning task, in which novel noun-noun compounds were presented in a semantically congruent context, enabling schema-supported processing, or in a neutral context. As expected, associative memory performance was better for compounds preceded by a congruent context. Although the N400 was attenuated in the congruent condition, subsequent memory effects (SMEs) in the N400 time interval did not differ across conditions, suggesting that the processes reflected in the N400 cannot account for the memory advantage in the congruent condition. However, a parietal SME was obtained for compounds preceded by a congruent context, only, which we interpret as reflecting the schema-supported formation of a conceptual compound representation. A late frontal SME was obtained in both conditions, presumably reflecting the more general inter-item associative encoding of compound constituents.},
pubstate = {published},
type = {article}
}

Copy BibTeX to Clipboard

Project:   A6

Sommerfeld, Linda; Staudte, Maria; Mani, Nivedita; Kray, Jutta

Children and adults integrate complex visual contexts in language prediction Miscellaneous

Architectures and Mechanisms for Language Processing AMLAP Annual Meeting, 2021.

@miscellaneous{sommerfeld2021children,
title = {Children and adults integrate complex visual contexts in language prediction},
author = {Linda Sommerfeld and Maria Staudte and Nivedita Mani and Jutta Kray},
url = {https://amlap2021.github.io/program/119.pdf},
year = {2021},
date = {2021},
booktitle = {Architectures and Mechanisms for Language Processing AMLAP Annual Meeting},
pubstate = {published},
type = {miscellaneous}
}

Copy BibTeX to Clipboard

Project:   A5

van Os, Marjolein; Kray, Jutta; Demberg, Vera

Mishearing as a Side Effect of Rational Language Comprehension in Noise Journal Article

Frontiers in Psychology, 12, pp. 3488, 2021, ISSN 1664-1078.

Language comprehension in noise can sometimes lead to mishearing, due to the noise disrupting the speech signal. Some of the difficulties in dealing with the noisy signal can be alleviated by drawing on the context – indeed, top-down predictability has shown to facilitate speech comprehension in noise. Previous studies have furthermore shown that strong reliance on the top-down predictions can lead to increased rates of mishearing, especially in older adults, which are attributed to general deficits in cognitive control in older adults. We here propose that the observed mishearing may be a simple consequence of rational language processing in noise. It should not be related to failure on the side of the older comprehenders, but instead would be predicted by rational processing accounts. To test this hypothesis, we extend earlier studies by running an online listening experiment with younger and older adults, carefully controlling the target and direct competitor in our stimuli. We show that mishearing is directly related to the perceptibility of the signal. We furthermore add an analysis of wrong responses, which shows that results are at odds with the idea that participants overly strongly rely on context in this task, as most false answers are indeed close to the speech signal, and not to the semantics of the context.

@article{vanOs2021FrontPsych,
title = {Mishearing as a Side Effect of Rational Language Comprehension in Noise},
author = {Marjolein van Os and Jutta Kray and Vera Demberg},
url = {https://www.frontiersin.org/article/10.3389/fpsyg.2021.679278},
doi = {https://doi.org/10.3389/fpsyg.2021.679278},
year = {2021},
date = {2021},
journal = {Frontiers in Psychology},
pages = {3488},
volume = {12},
abstract = {Language comprehension in noise can sometimes lead to mishearing, due to the noise disrupting the speech signal. Some of the difficulties in dealing with the noisy signal can be alleviated by drawing on the context – indeed, top-down predictability has shown to facilitate speech comprehension in noise. Previous studies have furthermore shown that strong reliance on the top-down predictions can lead to increased rates of mishearing, especially in older adults, which are attributed to general deficits in cognitive control in older adults. We here propose that the observed mishearing may be a simple consequence of rational language processing in noise. It should not be related to failure on the side of the older comprehenders, but instead would be predicted by rational processing accounts. To test this hypothesis, we extend earlier studies by running an online listening experiment with younger and older adults, carefully controlling the target and direct competitor in our stimuli. We show that mishearing is directly related to the perceptibility of the signal. We furthermore add an analysis of wrong responses, which shows that results are at odds with the idea that participants overly strongly rely on context in this task, as most false answers are indeed close to the speech signal, and not to the semantics of the context.},
pubstate = {published},
type = {article}
}

Copy BibTeX to Clipboard

Project:   A4

Donatelli, Lucia; Schmidt, Theresa; Biswas, Debanjali; Köhn, Arne; Zhai, Fangzhou; Koller, Alexander

Aligning Actions Across Recipe Graphs Inproceedings

Proceedings of EMNLP, pp. 6930–6942, 2021.

Recipe texts are an idiosyncratic form of instructional language that pose unique challenges for automatic understanding. One challenge is that a cooking step in one recipe can be explained in another recipe in different words, at a different level of abstraction, or not at all. Previous work has annotated correspondences between recipe instructions at the sentence level, often glossing over important correspondences between cooking steps across recipes. We present a novel and fully-parsed English recipe corpus, ARA (Aligned Recipe Actions), which annotates correspondences between individual actions across similar recipes with the goal of capturing information implicit for accurate recipe understanding. We represent this information in the form of recipe graphs, and we train a neural model for predicting correspondences on ARA. We find that substantial gains in accuracy can be obtained by taking fine-grained structural information about the recipes into account.

@inproceedings{donatelli21:align,
title = {Aligning Actions Across Recipe Graphs},
author = {Lucia Donatelli and Theresa Schmidt and Debanjali Biswas and Arne K{\"o}hn and Fangzhou Zhai and Alexander Koller},
url = {https://aclanthology.org/2021.emnlp-main.554},
year = {2021},
date = {2021},
booktitle = {Proceedings of EMNLP},
pages = {6930–6942},
abstract = {Recipe texts are an idiosyncratic form of instructional language that pose unique challenges for automatic understanding. One challenge is that a cooking step in one recipe can be explained in another recipe in different words, at a different level of abstraction, or not at all. Previous work has annotated correspondences between recipe instructions at the sentence level, often glossing over important correspondences between cooking steps across recipes. We present a novel and fully-parsed English recipe corpus, ARA (Aligned Recipe Actions), which annotates correspondences between individual actions across similar recipes with the goal of capturing information implicit for accurate recipe understanding. We represent this information in the form of recipe graphs, and we train a neural model for predicting correspondences on ARA. We find that substantial gains in accuracy can be obtained by taking fine-grained structural information about the recipes into account.},
pubstate = {published},
type = {inproceedings}
}

Copy BibTeX to Clipboard

Projects:   A3 A7

Zhai, Fangzhou; Skrjanec, Iza; Koller, Alexander

Script Parsing with Hierarchical Sequence Modelling Inproceedings

Proceedings of *SEM 2021: 10th Joint Conf. on Lexical and Computational Semantics, pp. 195-201, 2021.

Scripts capture commonsense knowledge about everyday activities and their participants. Script knowledge proved useful in a number of NLP tasks, such as referent prediction, discourse classification, and story generation. A crucial step for the exploitation of script knowledge is script parsing, the task of tagging a text with the events and participants from a certain activity. This task is challenging: it requires information both about the ways events and participants are usually uttered in surface language as well as the order in which they occur in the world. We show how to do accurate script parsing with a hierarchical sequence model and transfer learning. Our model improves the state of the art of event parsing by over 16 points F-score and, for the first time, accurately tags script participants.

@inproceedings{zhaiSkrjanecKoller2021,
title = {Script Parsing with Hierarchical Sequence Modelling},
author = {Fangzhou Zhai and Iza Skrjanec and Alexander Koller},
url = {https://aclanthology.org/2021.starsem-1.18},
doi = {https://doi.org/10.18653/v1/2021.starsem-1.18},
year = {2021},
date = {2021},
booktitle = {Proceedings of *SEM 2021: 10th Joint Conf. on Lexical and Computational Semantics},
pages = {195-201},
abstract = {Scripts capture commonsense knowledge about everyday activities and their participants. Script knowledge proved useful in a number of NLP tasks, such as referent prediction, discourse classification, and story generation. A crucial step for the exploitation of script knowledge is script parsing, the task of tagging a text with the events and participants from a certain activity. This task is challenging: it requires information both about the ways events and participants are usually uttered in surface language as well as the order in which they occur in the world. We show how to do accurate script parsing with a hierarchical sequence model and transfer learning. Our model improves the state of the art of event parsing by over 16 points F-score and, for the first time, accurately tags script participants.},
pubstate = {published},
type = {inproceedings}
}

Copy BibTeX to Clipboard

Project:   A3

Tourtouri, Elli; Delogu, Francesca; Crocker, Matthew W.

Rational Redundancy in Referring Expressions: Evidence from Event-related Potentials Journal Article

Cognitive Science, 45, Wiley, pp. e13071, 2021.

In referential communication, Grice’s Maxim of Quantity is thought to imply that utterances conveying unnecessary information should incur comprehension difficulties. There is, however, considerable evidence that speakers frequently encode redundant information in their referring expressions, raising the question as to whether such overspecifications hinder listeners’ processing. Evidence from previous work is inconclusive, and mostly comes from offline studies. In this article, we present two event-related potential (ERP) experiments, investigating the real-time comprehension of referring expressions that contain redundant adjectives in complex visual contexts. Our findings provide support for both Gricean and bounded-rational accounts. We argue that these seemingly incompatible results can be reconciled if common ground is taken into account. We propose a bounded-rational account of overspecification, according to which even redundant words can be beneficial to comprehension to the extent that they facilitate the reduction of listeners’ uncertainty regarding the target referent.

@article{Tourtouri2021rational,
title = {Rational Redundancy in Referring Expressions: Evidence from Event-related Potentials},
author = {Elli Tourtouri and Francesca Delogu and Matthew W. Crocker},
url = {https://doi.org/10.1111/cogs.13071},
doi = {https://doi.org/10.1111/cogs.13071},
year = {2021},
date = {2021-12-12},
journal = {Cognitive Science},
pages = {e13071},
publisher = {Wiley},
volume = {45},
number = {12},
abstract = {In referential communication, Grice's Maxim of Quantity is thought to imply that utterances conveying unnecessary information should incur comprehension difficulties. There is, however, considerable evidence that speakers frequently encode redundant information in their referring expressions, raising the question as to whether such overspecifications hinder listeners’ processing. Evidence from previous work is inconclusive, and mostly comes from offline studies. In this article, we present two event-related potential (ERP) experiments, investigating the real-time comprehension of referring expressions that contain redundant adjectives in complex visual contexts. Our findings provide support for both Gricean and bounded-rational accounts. We argue that these seemingly incompatible results can be reconciled if common ground is taken into account. We propose a bounded-rational account of overspecification, according to which even redundant words can be beneficial to comprehension to the extent that they facilitate the reduction of listeners’ uncertainty regarding the target referent.},
pubstate = {published},
type = {article}
}

Copy BibTeX to Clipboard

Project:   C3

Successfully