Issue #19 – Adaptive Neural MT

29 Nov18

Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic

Neural Machine Translation is known to be particularly poor at translating out-of-domain data. That is, an engine trained on generic data will be much worse at translating medical documents than an engine trained on medical data. It is much more sensitive to such differences than, say, Statistical MT. This problem is partially solved by domain adaptation techniques, which we covered in Issue #9 of this series. However, what if we are in a multi-domain scenario and we do not know the nature of the input in advance?

In this post, we take a look at a technique proposed by Amin Farajian et al. (2017), which consists of adapting the model on-the-fly for each source sentence with similar sentence pairs retrieved from the training corpus. This technique actually also works very well as micro-adaptation in a closed-domain scenario.

Adaptive adaptation

A standard domain adaptation method for Neural MT models consists of training a model on a large amount of generic data and resuming the training on in-domain data, usually available in a much smaller amount. Farajian et
To finish reading, please visit source site