PyTorch library for training Submanifold Sparse Convolutional Networks
SparseConvNet
Submanifold Sparse Convolutional Networks
This is the PyTorch library for training Submanifold Sparse Convolutional Networks.
Spatial sparsity
This library brings Spatially-sparse convolutional networks to PyTorch. Moreover, it introduces Submanifold Sparse Convolutions, that can be used to build computationally efficient sparse VGG/ResNet/DenseNet-style networks.
With regular 3×3 convolutions, the set of active (non-zero) sites grows rapidly:
With Submanifold Sparse Convolutions, the set of active sites is unchanged. Active sites look at their active neighbors (green); non-active sites (red) have no computational overhead:
Stacking Submanifold Sparse Convolutions to build VGG and ResNet type ConvNets, information can flow along lines or surfaces of active points.
Disconnected components don’t communicate at first, although they will merge due to the effect of strided operations, either pooling or