Highlights from Machine Translation and Multilinguality in November 2024
Mitigating Metric Bias in Minimum Bayes Risk Decoding
Minimum Bayes Risk Decoding tries to get the most typical output from a language or machine translation model rather than the most probable one. The main idea is that the probability scores do not consider how semantically similar sentences are. Therefore, the most probable sequence might not be the most typical from a meaning perspective. The weak point is that we have to decide what metric to use to estimate the similarity, which typically leads to overfitting towards the metric of choice. Those metrics are typically COMET, BLEURT, or MetriX (which are slow to compute) for machine translation. This paper from Google shows that if they use an ensemble of metrics (so, we make the slow decoding even slower), this