Implementation of ConvMixer for Patches Are All You Need?
This repository contains an implementation of ConvMixer for the ICLR 2022 submission “Patches Are All You Need?” by Asher Trockman and Zico Kolter.
🔎 New: Check out this repository for training ConvMixers on CIFAR-10.
Code overview
The most important code is in convmixer.py. We trained ConvMixers using the timm framework, which we copied from here.
Update: ConvMixer is now integrated into the timm framework itself. You can see the PR here.
Inside pytorch-image-models, we have made the following modifications. (Though one could look at the diff, we think it is convenient to summarize them here.)
- Added ConvMixers
- added
timm/models/convmixer.py - modified
timm/models/__init__.py
- added
- Added “OneCycle” LR Schedule
- added
timm/scheduler/onecycle_lr.py - modified
timm/scheduler/scheduler.py - modified
timm/scheduler/scheduler_factory.py - modified
timm/scheduler/__init__.py - modified
train.py(added two lines to support this LR
- added