Abstract
Model-agnostic meta-learning (MAML) hasbeen recently put forth as a strategy to learnresource-poor languages in a sample-efficientfashion. Nevertheless, the properties of theselanguages are often not well represented bythose available during training. Hence, weargue that the i.i.d. assumption ingrained inMAML makes it ill-suited for cross-lingualNLP. In fact, under a decision-theoretic framework, MAML can be interpreted as minimising the expected risk across training languages(with a uniform prior), which is known asBayes criterion. To increase its robustness tooutlier languages, we create two variants ofMAML based on alternative criteria: MinimaxMAML reduces the maximum risk across languages, while Neyman–Pearson MAML constrains the risk in each language to a maximum threshold. Both criteria constitute fullydifferentiable two-player games. In light ofthis, we propose a new adaptive optimiser solving for a local approximation to their Nashequilibrium. We evaluate both model variants on two popular NLP tasks, part-of-speechtagging and question answering. We reportgains for their average and minimum performance across low-resource languages in zeroand few-shot settings, compared to joint multisource transfer and vanilla MAML. The codefor our experiments is available at https://github.com/rahular/robust-maml.
Originalsprog | Engelsk |
---|---|
Titel | Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 |
Forlag | Association for Computational Linguistics |
Publikationsdato | 2021 |
Sider | 1245-1260 |
DOI | |
Status | Udgivet - 2021 |
Begivenhed | Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 - Virtual, Online Varighed: 1 aug. 2021 → 6 aug. 2021 |
Konference
Konference | Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 |
---|---|
By | Virtual, Online |
Periode | 01/08/2021 → 06/08/2021 |