Byte-level subwords, NLP Leave a comment

Issue #64 – Neural Machine Translation with Byte-Level Subwords

13 Dec19 Issue #64 – Neural Machine Translation with Byte-Level Subwords Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic In order to limit vocabulary, most neural machine translation engines are based on subwords. In some settings, character-based systems are even better (see issue #60). However, rare characters in noisy data or character-based languages can unnecessarily take up vocabulary slots and limit its compactness. In this post we take a look at an alternative, proposed by Wang et al. (2019), […]

Read more
Domain Differential Adaptation, NLP Leave a comment

Issue #62 – Domain Differential Adaptation for Neural MT

28 Nov19 Issue #62 – Domain Differential Adaptation for Neural MT Author: Raj Patel, Machine Translation Scientist @ Iconic Neural MT models are data hungry and domain sensitive, and it is nearly impossible to obtain a good amount ( >1M segments) of training data for every domain we are interested in. One common strategy is to align the statistics of the source and target domain, but the drawback of this approach is that the statistics of the different domains are inherently […]

Read more
Character-based Neural MT with Transformers, NLP Leave a comment

Issue #60 – Character-based Neural Machine Translation with Transformers

14 Nov19 Issue #60 – Character-based Neural Machine Translation with Transformers Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic We saw in issue #12 of this blog how character-based recurrent neural networks (RNNs) could outperform (sub)word-based models if the network is deep enough. However, character sequences are much longer than subword ones, which is not easy to deal with in  RNNs. In this post, we discuss how the Transformer architecture changes the situation for character-based models. We take a […]

Read more
ACL 2019, NLP Leave a comment

Issue #52 – A Selection from ACL 2019

19 Sep19 Issue #52 – A Selection from ACL 2019 Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic The Conference of the Association for Computational Linguistics (ACL) took place this summer, and over the past few months we have reviewed a number of preprints (see Issues 28, 41 and 43) which were published at ACL. In this post, we take a look at three more papers presented at the conference, that we found particularly interesting, in the context of […]

Read more
Multilingual e-disclosure, NLP Leave a comment

Issue #48 – It’s all French Belgian Fries to me… or The Art of Multilingual e-Disclosure (Part II)

01 Aug19 Issue #48 – It’s all French Belgian Fries to me… or The Art of Multilingual e-Disclosure (Part II) Author: Jérôme Torres Lozano, Director of Professional Services, Inventus This is the second of a two-part guest post from Jérôme Torres Lozano, the Director of Professional Services at Inventus, who shares his perspective on The Art of Multilingual e-Disclosure. In Part I,  we learned about the challenges of languages in e-disclosure.  In this post he will discuss language identification and translation options available […]

Read more
Multilingual e-disclosure, NLP Leave a comment

Issue #47 – It’s all French Belgian Fries to me, or The Art of Multilingual e-Disclosure (Part I)

25 Jul19 Issue #47 – It’s all French Belgian Fries to me, or The Art of Multilingual e-Disclosure (Part I) Author: Jérôme Torres Lozano, Director of Professional Services, Inventus Over the next two weeks, we’re taking a slightly different approach on the blog. In today’s article, the first of two parts, we will hear from Jérôme Torres-Lozano of Inventus, a user of Iconic’s Neural MT solutions for e-discovery. He gives us an entertaining look at his experiences on the challenges of language, […]

Read more
augmenting self-attention with persistent memory, NLP Leave a comment

Issue #46 – Augmenting Self-attention with Persistent Memory

18 Jul19 Issue #46 – Augmenting Self-attention with Persistent Memory Author: Dr. Rohit Gupta, Sr. Machine Translation Scientist @ Iconic In Issue #32 we introduced the Transformer model as the new state-of-the-art in Neural Machine Translation. Subsequently, in Issue #41 we looked at some approaches that were aiming to improve upon it. In this post, we take a look at significant change in the Transformer model, proposed by Sukhbaatar et al. (2019), which further improves its performance. Each Transformer layer consists of two types […]

Read more
Deep Transformer Models, NLP Leave a comment

Issue #41 – Deep Transformer Models for Neural MT

13 Jun19 Issue #41 – Deep Transformer Models for Neural MT Author: Dr. Patrik Lambert, Machine Translation Scientist @ Iconic The Transformer is a state-of-the-art Neural MT model, as we covered previously in Issue #32. So what happens when something works well with neural networks? We try to go wider and deeper! There are two research directions that look promising to enhance the Transformer model: building wider networks by increasing the size of word representation and attention vectors, or building […]

Read more
Neural MT, NLP Leave a comment

Issue #40 – Consistency by Agreement in Zero-shot Neural MT

06 Jun19 Issue #40 – Consistency by Agreement in Zero-shot Neural MT Author: Raj Patel, Machine Translation Scientist @ Iconic In two of our earlier posts (Issues #6 and #37), we discussed the zero-shot approach to Neural MT – learning to translate from source to target without seeing even a single example of the language pair directly. In Neural MT, the zero-shot training is achieved using multilingual architecture (Johnson et al. 2017) – a single NMT engine that can translate between […]

Read more
Context-aware NMT, NLP Leave a comment

Issue #39 – Context-aware Neural Machine Translation

30 May19 Issue #39 – Context-aware Neural Machine Translation Author: Dr. Rohit Gupta, Sr. Machine Translation Scientist @ Iconic Back in Issue #15, we looked at the topic of document-level translation and the idea of looking at more context than just the sentence when machine translating. In this post, we will have a look more generally at the role of context in machine translation as relates to specific types of linguistic phenomena and issues related to them. We review the work […]

Read more
1 … 890 891 892 893 894 … 896