Issue #102 – Nearest Neighbour Machine Translation
08 Oct20
Issue #102 – Nearest Neighbour Machine Translation
Author: Dr. Patrik Lambert, Senior Machine Translation Scientist @ Iconic
Introduction
Taking into account context information in neural MT is an active area of research, with applications in document-level translation, domain adaptation and multilingual neural MT. Today we take a look at a method which combines predictions from a neural MT model and from a nearest neighbour classifier, retrieved from similar contexts in a datastore of cached examples. This approach, called “Nearest Neighbor Machine Translation”, was proposed by Khandelwal et al. (2020).
Description of the Method
In nearest neighbour MT, the translation is generated word-by-word. For each word, the most similar contexts are found in the datastore, and a distribution over the corresponding target tokens is computed. This distribution is then interpolated with the output distribution from a previously trained neural MT model. This process is described in the following figure.
The translation contexts are taken from parallel training data and are composed, for each target word of each example, of the complete source