Lessons learned from analyzing values in multilingual encoders and what it means for LLMs
This post is a hindsight on two studies on multilingual sentence embeddings
we published a year ago and comments on what I think people analyzing LLMs
today should take away from them.
In late 2022, we (which mainly was the work of Kathy Hämmerl from Munich and
Björn Diesenroth and Patrick Schramowski from Darmstadt) finished a paper called
Speaking Multiple Languages Affects the Moral Bias of Language
Models (later published in
Findings of ACL 2023) where we tried to compare moral biases in sentence
embeddings across languages. Then, I had a short paper Is a Prestigious Job the
same as a Prestigious Country?
in Findings of EMNLP 2023 using the same methodology to assess what biases
multilingual sentence embeddings contain regarding countries and jobs.
The folks from Darmstadt