Implementing Transformers in NLP Under 5 Lines Of Codes
Introduction
Today, we will see a gentle introduction to the transformers library for executing state-of-the-art models for complex NLP tasks.
Applying state-of-the-art Natural Language Processing models has never been more straightforward. Hugging Face has revealed a compelling library called transformers that allow us to perform and use a broad class of state-of-the-art NLP models in a specific way.
Today we are operating to install and use the transformers library for diverse tasks such as:
- Text Classification
- Question-Answering
- Masked Language Modeling
- Text Generation
- Named Entity Recognition
- Text Summarization
So before we start evaluating each of the implementations for the varying tasks, let’s fix the transformers library. In my case, I am operating on macOS; when attempting to install instantly with pip, I got an error which I did by previously connecting the Rust compiler as follows:
$ curl --proto '=https' --tlsv1.2