Graph Attention Networks: Self-Attention for GNNs
Graph Attention Networks (GATs) are one of the most popular types of Graph Neural Networks.
Instead of calculating static weights based on node degrees like Graph Convolutional Networks (GCNs), they assign dynamic weights to node features through a process called self-attention. The main idea behind GATs is that some neighbors are more important than others, regardless of their node degrees.
Node 4 is more important than node 3, which is more important than node 2
In this article, we will see how to calculate these attention scores and implement an efficient GAT in PyTorch Geometric (PyG). You can run the code of this tutorial with the following Google Colab notebook.