Microsoft Turing Universal Language Representation model, T-ULRv2, tops XTREME leaderboard
Today, we are happy to announce that Turing multilingual language model (T-ULRv2) is the state of the art at the top of the Google XTREME public leaderboard. Created by the Microsoft Turing team in collaboration with Microsoft Research, the model beat the previous best from Alibaba (VECO) by 3.5 points in average score. To achieve this, in addition to the pretrained model, we leveraged “StableTune,” a novel multilingual fine-tuning technique based on stability training. Other models on the leaderboard include XLM-R, mBERT, XLM and more. One of the previous best submissions is also from Microsoft using FILTER.
Universal Language Representation
The Microsoft Turing team has long believed that language representation should be universal. In this paper, published in 2018,