Implementation of Uformer, Attention-based Unet, in Pytorch

Uformer – Pytorch

Implementation of Uformer, Attention-based Unet, in Pytorch. It will only offer the concat-cross-skip connection.

This repository will be geared towards use in a project for learning protein structures. Specifically, it will include the ability to condition on time steps (needed for DDPM), as well as 2d relative positional encoding using rotary embeddings (instead of the bias on the attention matrix in the paper).

Install

$ pip install uformer-pytorch

Usage

import torch

from uformer_pytorch import Uformer

model = Uformer(

    dim = 64,           # initial dimensions after input projection, which increases by 2x each stage

    stages = 4,         # number of stages

    num_blocks = 2,     # number of transformer blocks per stage

    window_size = 16,   # set window size (along one side) for
 
 

 
To finish reading, please visit source site