An all-MLP replacement for Transformers in Pytorch
gMLP – Pytorch Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch Install $ pip install g-mlp-pytorch Usage For masked language modelling import torch from g_mlp_pytorch import gMLP model = gMLP( num_tokens = 20000, dim = 512, depth = 6, seq_len = 256 ) x = torch.randint(0, 20000, (1, 256)) logits = model(x) # (1, 256, 20000) For image classification import torch from g_mlp_pytorch import gMLPVision model = gMLPVision( image_size = 256, patch_size = 16, num_classes = 1000, dim […]
Read more