Highlights from Machine Translation and Multilinguality in June 2023
Here are the preprints that I found the most interesting in June 2023.
Exploring the Relationship between Alignment and Cross-lingual Transfer in Multilingual Transformers
Folks from LORIA (a French research institute) and Posos (a French company)
study the relationship between cross-lingual representation alignment and
cross-lingual transfer. Here, alignment means what I would call language
neutrality, i.e., that similar sentences should receive similar representation
across languages. (Not alignment as the new word for finetuning language models
to follow instructions, nor the stuff we need to do to prevent AI apocalypse.)
They measure the transfer quality as a relative performance drop across
languages (i.e., the performance in the target language is X points worse than
the source language on task Y). The alignment quality is evaluated using top-1
nearest neighbor