As discussed the previous session, transformer models are usually quite large in the order of billion parameters. Therefor, creating and training them from scratch makes little sense. Since new variants of transformer models are developed every day, the team at Hugging Face has democratized the usage of such state-of-the-art models with the help of Transformer APIs.
With such an API, it becomes easy for anyone to download any pre-trained models, load datasets, and tokenize the input as per the model requirements. Using pre-trained models can significantly reduce your compute costs and carbon footprint and save you the time and resources required to train a model from scratch.
Here are the features of the Transformer API:
- Ease of use: You can easily download, load, train, and save any state-of-the-art NLP model with few lines of code.
- Flexibility: The API allows you to change the language/framework (TensorFlow or PyTorch) for the model in which it is developed/used because all models are simple PyTorch nn.Module or TensorFlow tf.keras.Model classes.
- Simplicity: Transformers have a layered API that allows the programmer to engage with the library at various levels of abstraction.
By the end of this session, you should be able to:
In this session, you will learn how to utilize the different features provided by the Transformer API. These features will help you perform different NLP tasks, and among them, the easiest to use is the pipeline() function. Once you have a good understanding of this function, you will learn about the tokenizer() function to understand how a transformer preprocesses numerical inputs to output predictions. You will also learn how to process multiple sentences and pass them to the model for predictions simultaneously. Finally, you will apply what you learned to fine-tune a BERT model to perform the task of Quora question-pair similarity.
- Apply the pipeline() function to perform NLP tasks such as text generation and classification.
- Configure both tokenizer and transformer models to perform an NLP task.
- Fine-tune a transformer model for a custom use-case of Quora question-pair similarity.
Guidelines for in-module questions
The in-video and in-content questions for this module are not graded.
People you will hear from in this session
Subject Matter Expert
Ankush Chopra
Sr Manager, Tredence
Ankush has 10+ years of hands-on Data Science experience delivering data and ML solutions for Fortune 500 clients across the Telecom, Retail, Healthcare, and Finance industries. He currently works as a Senior Manager(AI-Centre of Excellence) in Tredence.
To understand the feature of the Transformers API, let’s go through the next segment.