Cyclemoid implementation for PyTorch

This is an implementation of the cyclemoid activation function for PyTorch.

The cyclemoid function achieved state-of-the-art results in a recent benchmark with other popular activation functions as shown below:

Note that this is a figure from the paper submitted on April 1st, 2022. An arxiv preprint will be uploaded soon.

Installation

You can install the cyclemoid-pytorch package via

pip install cyclemoid_pytorch

Usage

This package implements a CycleMoid class and a cyclemoid function. You can use these are drop-in replacements for any activation in PyTorch. For example