Abstract
Multilingual large language models (LLMs) seem to generalize somewhat across languages. We hypothesize this is a result of implicit vector space alignment. Evaluating such alignment, we see that larger models exhibit very high-quality linear alignments between corresponding concepts in different languages. Our experiments show that multilingual LLMs suffer from two familiar weaknesses: generalization works best for languages with similar typology, and for abstract concepts. For some models, e.g., the Llama-2 family of models, prompt-based embeddings align better than word embeddings, but the projections are less linear – an observation that holds across almost all model families, indicating that some of the implicitly learned alignments are broken somewhat by prompt-based methods.
| Originalsprog | Engelsk |
|---|---|
| Titel | Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing |
| Forlag | Association for Computational Linguistics (ACL) |
| Publikationsdato | 2024 |
| Sider | 5511-5526 |
| DOI | |
| Status | Udgivet - 2024 |
| Begivenhed | 2024 Conference on Empirical Methods in Natural Language Processing - EMNLP, Miami, USA Varighed: 12 nov. 2024 → 16 nov. 2024 |
Konference
| Konference | 2024 Conference on Empirical Methods in Natural Language Processing |
|---|---|
| Lokation | EMNLP |
| Land/Område | USA |
| By | Miami |
| Periode | 12/11/2024 → 16/11/2024 |
Citationsformater
- APA
- Standard
- Harvard
- Vancouver
- Author
- BIBTEX
- RIS