Spring til hovednavigation Spring til søgning Spring til hovedindhold

Concept Space Alignment in Multilingual LLMs

Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › peer review

46 Downloads (Pure)

Abstract

Multilingual large language models (LLMs) seem to generalize somewhat across languages. We hypothesize this is a result of implicit vector space alignment. Evaluating such alignment, we see that larger models exhibit very high-quality linear alignments between corresponding concepts in different languages. Our experiments show that multilingual LLMs suffer from two familiar weaknesses: generalization works best for languages with similar typology, and for abstract concepts. For some models, e.g., the Llama-2 family of models, prompt-based embeddings align better than word embeddings, but the projections are less linear – an observation that holds across almost all model families, indicating that some of the implicitly learned alignments are broken somewhat by prompt-based methods.
OriginalsprogEngelsk
TitelProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
ForlagAssociation for Computational Linguistics (ACL)
Publikationsdato2024
Sider5511-5526
DOI
StatusUdgivet - 2024
Begivenhed2024 Conference on Empirical Methods in Natural Language Processing - EMNLP, Miami, USA
Varighed: 12 nov. 2024 → 16 nov. 2024

Konference

Konference2024 Conference on Empirical Methods in Natural Language Processing
LokationEMNLP
Land/OmrådeUSA
ByMiami
Periode12/11/2024 → 16/11/2024

Citationsformater