Machine Translation Weekly 67: Where the language neurality of mBERT reside?
If someone told me ten years ago when I was a freshly graduated bachelor of
computer science that there would models that would produce multilingual
sentence representation allowing zero-shot model transfer, I would have hardly
believed such a prediction. If they added that the models would be total black
boxes and we would not know why it worked, I would think they were insane.
After all, one of the goals of the mathematization of stuff in science is to
make things clear and predictable, often at the expense of some inevitable
reduction. Why would anyone use math and computer science for creating black
box models? Yet, here we are.
The paper I am going to comment on tries to partially uncover this
black-boxness. The title of the paper is