Issue #101 – Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation
02 Oct20
Issue #101 – Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation
Author: Dr. Chao-Hong Liu, Machine Translation Scientist @ Iconic
Introduction
Multilingual Neural Machine Translation (NMT), which enables zero-shot MT, is a significant development since the start of NMT. On the one hand, we have evidence that models trained with multiple languages can outperform those trained on a bilingual basis. On the other hand, multilingual NMT also enables us to train models of a language pair where there is no direct parallel corpus available in the training set. This is an important advancement, but we still need more evidence on whether it can compete with other approaches, e.g. pivot MT. In this post, we review the method proposed by Siddhant et al. (2020), which leverages monolingual data with self-supervision for multilingual NMT. The results confirmed the two benefits as mentioned above, and it also provided a viable way to add a new language without any parallel data or back translation.
Adapting MASS for Multilingual Models
There are many approaches dedicated to improving the performance of multilingual NMT. For example, Chaudhary et al. (2019) used multilingual sentence embeddings to