Ancestor-to-Creole Transfer is Not a Walk in the Park

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningpeer review

4 Citationer (Scopus)
12 Downloads (Pure)

Abstract

We aim to learn language models for Creole languages for which large volumes of data are not readily available, and therefore explore the potential transfer from ancestor languages (the ‘Ancestry Transfer Hypothesis’). We find that standard transfer methods do not facilitate ancestry transfer. Surprisingly, different from other non-Creole languages, a very distinct two-phase pattern emerges for Creoles: As our training losses plateau, and language models begin to overfit on their source languages, perplexity on the Creoles drop. We explore if this compression phase can lead to practically useful language models (the ‘Ancestry Bottleneck Hypothesis’), but also falsify this. Moreover, we show that Creoles even exhibit this two-phase pattern even when training on random, unrelated languages. Thus Creoles seem to be typological outliers and we speculate whether there is a link between the two observations.
OriginalsprogEngelsk
TitelProceedings of the Third Workshop on Insights from Negative Results in NLP
ForlagAssociation for Computational Linguistics (ACL)
Publikationsdato2022
Sider68-74
DOI
StatusUdgivet - 2022
BegivenhedThird Workshop on Insights from Negative Results in NLP - Dublin, Ireland
Varighed: 1 maj 20221 maj 2022

Konference

KonferenceThird Workshop on Insights from Negative Results in NLP
ByDublin, Ireland
Periode01/05/202201/05/2022

Citationsformater