June 4, 2020 Estimators of Quality, NLP Leave a comment

Issue #84 – Are Neural Machine Translation Systems Good Estimators of Quality?

04 Jun20 Issue #84 – Are Neural Machine Translation Systems Good Estimators of Quality? Author: Prof. Lucia Specia, Professor of Natural Language Processing, Imperial College London (also to ADAPT/Dublin City University and University of Sheffield) This week, we are delighted to have a guest post from Prof. Lucia Specia of Imperial College London, and laterally the University of Sheffield and our own alma mater, Dublin City University. Prof. Specia is one of the world’s preeminent experts on the topic of […]

May 14, 2020 Constrained Decoding, NLP Leave a comment

Issue #82 – Constrained Decoding using Levenshtein Transformer

14 May20 Issue #82 – Constrained Decoding using Levenshtein Transformer Author: Raj Patel, Machine Translation Scientist @ Iconic Introduction In constrained decoding, we force in-domain terminology to appear in the final translation. We have previously discussed constrained decoding in earlier blog posts (#7, #9, #79). In this blog post, we will discuss a simple and effective algorithm for incorporating lexical constraints in Neural Machine Translation (NMT) proposed by Susanto et al. (2020) and try to understand how it is better than […]

May 7, 2020 Human-Machine Parity, NLP Leave a comment

Issue #81 – Evaluating Human-Machine Parity in Language Translation: part 2

07 May20 Issue #81 – Evaluating Human-Machine Parity in Language Translation: part 2 Author: Dr. Sheila Castilho, Post-Doctoral Researcher @ ADAPT Research Centre This is the second in a 2-part post addressing machine translation quality evaluation – an overarching topic regardless of the underlying algorithms. Following our own summary last week, this week we are delighted to have one of the paper’s authors, Dr. Sheila Castilho, give her take on the paper, their motivations for writing it, and where we […]

March 12, 2020 Mixed Multi-Head Self-Attention for Neural MT, NLP Leave a comment

Issue #73 – Mixed Multi-Head Self-Attention for Neural MT

12 Mar20 Issue #73 – Mixed Multi-Head Self-Attention for Neural MT Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic Self-attention is a key component of the Transformer, a state-of-the-art neural machine translation architecture. In the Transformer, self-attention is divided into multiple heads to allow the system to independently attend to information from different representation subspaces. Recently it has been shown that some redundancy occurs in the multiple heads. In this post, we take a look at approaches which ensure […]

February 7, 2020 BERT in Neural MT, NLP Leave a comment

Issue #68 – Incorporating BERT in Neural MT

07 Feb20 Issue #68 – Incorporating BERT in Neural MT Author: Raj Patel, Machine Translation Scientist @ Iconic BERT (Bidirectional Encoder Representations from Transformers) has shown impressive results in various Natural Language Processing (NLP) tasks. However, how to effectively apply BERT in Neural MT has not been fully explored. In general, BERT is used as fine-tuning for downstream NLP tasks. For Neural MT, a pre-trained BERT model is used to initialise the encoder in an encoder-decoder architecture. In this post we […]

January 23, 2020 Low-Resource Languages, NLP Leave a comment

Issue #66 – Neural Machine Translation Strategies for Low-Resource Languages

23 Jan20 Issue #66 – Neural Machine Translation Strategies for Low-Resource Languages This week we are pleased to welcome the newest member to our scientific team, Dr. Chao-Hong Liu. In this, his first post with us, he’ll give his views on two specific MT strategies, namely, pivot MT and zero-shot MT. While we have covered these topics in previous ‘Neural MT Weekly’ blog posts (Issue #54, Issue #40), these are topics that Chao-Hong has recently worked on prior to joining […]

December 13, 2019 Byte-level subwords, NLP Leave a comment

Issue #64 – Neural Machine Translation with Byte-Level Subwords

13 Dec19 Issue #64 – Neural Machine Translation with Byte-Level Subwords Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic In order to limit vocabulary, most neural machine translation engines are based on subwords. In some settings, character-based systems are even better (see issue #60). However, rare characters in noisy data or character-based languages can unnecessarily take up vocabulary slots and limit its compactness. In this post we take a look at an alternative, proposed by Wang et al. (2019), […]

November 28, 2019 Domain Differential Adaptation, NLP Leave a comment

Issue #62 – Domain Differential Adaptation for Neural MT

28 Nov19 Issue #62 – Domain Differential Adaptation for Neural MT Author: Raj Patel, Machine Translation Scientist @ Iconic Neural MT models are data hungry and domain sensitive, and it is nearly impossible to obtain a good amount ( >1M segments) of training data for every domain we are interested in. One common strategy is to align the statistics of the source and target domain, but the drawback of this approach is that the statistics of the different domains are inherently […]

November 14, 2019 Character-based Neural MT with Transformers, NLP Leave a comment

Issue #60 – Character-based Neural Machine Translation with Transformers

14 Nov19 Issue #60 – Character-based Neural Machine Translation with Transformers Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic We saw in issue #12 of this blog how character-based recurrent neural networks (RNNs) could outperform (sub)word-based models if the network is deep enough. However, character sequences are much longer than subword ones, which is not easy to deal with in RNNs. In this post, we discuss how the Transformer architecture changes the situation for character-based models. We take a […]

September 19, 2019 ACL 2019, NLP Leave a comment

Issue #52 – A Selection from ACL 2019

19 Sep19 Issue #52 – A Selection from ACL 2019 Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic The Conference of the Association for Computational Linguistics (ACL) took place this summer, and over the past few months we have reviewed a number of preprints (see Issues 28, 41 and 43) which were published at ACL. In this post, we take a look at three more papers presented at the conference, that we found particularly interesting, in the context of […]

« 1 … 904 905 906 907 908 … 910 »