Doc|Model|Dataset|Paper MWPToolkit is a PyTorch-based toolkit for Math Word Problem (MWP) solving. It is a comprehensive framework for research purpose that integrates popular MWP benchmark datasets and typical deep learning-based MWP algorithms. Our framework has the following architecture. You could utilize our toolkit to evaluate the build-in datasets, apply it to process your raw data copies or develop your own models. Figure: The Overall Framework of MWP Toolkit News Characteristics Unification and Modularization. We decouple solvers    

Hivemind: decentralized deep learning in PyTorch Hivemind is a PyTorch library for decentralized deep learning across the Internet. Its intended usage is training one large model on hundreds of computers from different universities, companies, and volunteers. Key Features Distributed training without a master node: Distributed Hash Table allows connecting computers in a decentralizednetwork. Fault-tolerant backpropagation: forward and backward passes succeed even if some nodes are unresponsive or take toolong to respond. Decentralized parameter averaging: iteratively aggregate updates from multiple workers […]

FairScale FairScale is a PyTorch extension library for high performance and large scale training.This library extends basic PyTorch capabilities while adding new SOTA scaling techniques.FairScale makes available the latest distributed training techniques in the form of composablemodules and easy to use APIs. These APIs are a fundamental part of a researcher’s toolbox asthey attempt to scale models with limited resources. FairScale was designed with the following values in mind: Usability –  Users should be able to understand and use FairScale […]

I implement the Decoupled Neural Interfaces using Synthetic Gradients in pytorch. The paper uses synthetic gradient to decouple the layers among the network, which is pretty interesting since we won’t suffer from update lock anymore. I test my model in mnist and almost the same performance, compared to the model updated with backpropagation. Requirement pytorch python 3.5 torchvision seaborn (optional) matplotlib (optional) TODO use multi-threading on gpu to analyze the speed What’s synthetic gradients? We ofter optimize NN by backpropogation, […]

A pytorch implementation of Paper “Improved Training of Wasserstein GANs”. Python, NumPy, SciPy, MatplotlibA recent NVIDIA GPU A latest master version of Pytorch [x] gan_toy.py : Toy datasets (8 Gaussians, 25 Gaussians, Swiss Roll).(Finished in 2017.5.8) [x] gan_language.py : Character-level language model (Discriminator is using nn.Conv1d. Generator is using nn.Conv1d. Finished in 2017.6.23. Finished in 2017.6.27.) [x] gan_mnist.py : MNIST (Running Results while Finished in 2017.6.26. Discriminator is using nn.Conv1d. Generator is using nn.Conv1d.) [ ] gan_64x64.py: 64×64 architectures(Looking forward […]

Molecular AutoEncoder in PyTorch Install $ git clone https://github.com/cxhernandez/molencoder.git && cd molencoder $ python setup.py install Download Dataset $ molencoder download –dataset chembl22 Train $ molencoder train –dataset data/chembl22.h5 Add –cuda flag to enable CUDA. Add –cont to continue training a model from a checkpoint file. Pre-Trained Model A pre-trained reference model is available in the ref/ directory. Currently, it performs with ~98% accuracy on the validation set after 100 epochs of training. However, if you succeed at training a […]

Implementations of polygamma, lgamma, and beta functions for PyTorch. It’s very hacky, but that’s usually ok for research use. To build, run: ./make.sh You’ll probably need to pass in the correct CUDA path to build.py, which is run inside make.sh, so modify it to instead call python build.py –cuda-path YOUR_CUDA_PATH Also, you’ll probably need to change the architecture version/CUDA compute capability inside make.sh, so replace sm_35 with whatever your GPU supports.Feel free to open an issue if you run into […]

Official repository for the paper Robust High-Resolution Video Matting with Temporal Guidance. RVM is specifically designed for robust human video matting. Unlike existing neural models that process frames as independent images, RVM uses a recurrent neural network to process videos with temporal memory. RVM can perform matting in real-time on any videos without additional inputs. It achieves 4K 76FPS and HD 104FPS on an Nvidia GTX 1080 Ti GPU. The project was developed at ByteDance Inc.

tinygrad For something in between a pytorch and a karpathy/micrograd This may not be the best deep learning framework, but it is a deep learning framework. Due to its extreme simplicity, it aims to be the easiest framework to add new accelerators to, with support for both inference and training. Support the simple basic ops, and you get SOTA vision extra/efficientnet.py and language extra/transformer.py models. We are working on support for the Apple Neural Engine. Eventually, we will build custom […]

(Generic) EfficientNets for PyTorch A ‘generic’ implementation of EfficientNet, MixNet, MobileNetV3, etc. that covers most of the compute/parameter efficient architectures derived from the MobileNet V1/V2 block sequence, including those found via automated neural architecture search. All models are implemented by GenEfficientNet or MobileNetV3 classes, with string based architecture definitions to configure the block layouts (idea from here) Models Implemented models include: I originally implemented and trained some these models with code here, this repository contains just the GenEfficientNet models, validation, […]