The NLP Cypher | 07.04.21

Hey Welcome back! Want to wish everyone in the US a happy 4th of July🎆🎇! Also, want to quickly mention that the NLP Index has doubled in size (since its inception) with now housing over 6,000 repos, pretty cool!!! 😎 And as always, it gets updated weekly. But first, this week we asked 100 NLP developers: Name one thing Microsoft got for paying $7.5 billi for GitHub, and $1 billi to OpenAI? SURVEY SAYS:

7.5B + 1B = GitHub CoPilot 👍

If you want to hear GitHub’s take on their new code generating assistant read here:

Also… it turns out CoPilot is a serial killer 👀, but at least the code is readable. 💪

Hey did you know Microsoft has a stash of models tucked away in their repository spanning NLU, document understanding, cross-lingual and more? If these models interest you, follow this page:

UniLM (v1@NeurIPS'19 | v2@ICML'20 | v3@ACL'21): unified pre-training for language understanding and generation

InfoXLM (v1@NAACL'21 | v2@ACL'21): multilingual/cross-lingual pre-trained models for language understanding and generation

DeltaLM (NEW): encoder-decoder pre-training for language generation and translation by augmenting pretrained multilingual encoders

MiniLM (v1@NeurIPS'20 | v2@ACL'21): small and fast pre-trained models for language understanding and generation

AdaLM (v1@ACL'21): domain, language, and task adaptation of pre-trained models

LayoutLM (

 

To finish reading, please visit source site