What Is Attention?
Attention is becoming increasingly popular in machine learning, but what makes it such an attractive concept? What is the relationship between attention applied in artificial neural networks and its biological counterpart? What components would one expect to form an attention-based system in machine learning?
In this tutorial, you will discover an overview of attention and its application in machine learning.
After completing this tutorial, you will know:
- A brief overview of how attention can manifest itself in the human brain
- The components that make up an attention-based system and how these are inspired by biological attention
Kick-start your project with my book Building Transformer Models with Attention. It provides self-study tutorials with working code to guide you into