lightning-transformers Flexible interface for high-performance research using SOTA Transformers leveraging Pytorch Lightning, Transformers, and Hydra. Installation Option 1: from PyPI pip install lightning-transformers # instead of: `python train.py …`, run with: pl-transformers-train … Option 2: from source git clone https://github.com/PyTorchLightning/lightning-transformers.git cd lightning-transformers pip install . python train.py … # the `pl-transformers-train` endpoint is also available! Quick recipes Train bert-base-cased on the CARER emotion dataset using the Text Classification task. python train.py task=nlp/text_classification dataset=nlp/text_classification/emotion See the composed Hydra config used under-the-hood […]

Talking-Face_PC-AVS Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation (CVPR 2021) We propose Pose-Controllable Audio-Visual System (PC-AVS), which achieves free pose control when driving arbitrary talking faces with audios. Instead of learning pose motions from audios, we leverage another pose source video to compensate only for head motions. The key is to devise an implicit low-dimension pose code that is free of mouth shape or identity information. In this way, audio-visual representations are modularized into spaces of three key […]

CG-Net This codebase is created to build benchmarks for object detection in aerial images. It is modified from mmdetection. The master branch works with PyTorch 1.1 or higher. If you would like to use PyTorch 0.4.1, please checkout to the pytorch-0.4.1 branch. Results Visualization results for oriented object detection on the test set of DOTA. Comparison to the baseline on DOTA for oriented object detection with ResNet-101. The figures with blue boxes are the results of the baseline and pink […]

SimCSE We propose a simple contrastive learning framework that works with both unlabeled and labeled data. Unsupervised SimCSE simply takes an input sentence and predicts itself in a contrastive learning framework, with only standard dropout used as noise. Our supervised SimCSE incorporates annotated pairs from NLI datasets into contrastive learning by using entailment pairs as positives and contradiction pairs as hard negatives. The following figure is an illustration of our models. Use our models out of the box Our pre-trained […]

OpenPNM OpenPNM is a comprehensive framework for performing pore network simulations of porous materials. For more details about the package can be found in the on-line documentation To be notified of new releases click the Watch button on the Github repo page. To show your support for this project click the Star button, as we use this as a proxy for the size of our userbase when applying for funding. Installation and Requirements Preferred method The preferred way of installing […]

BossNAS This repository contains PyTorch code and pretrained models of our paper: BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search. Illustration of the fabric-like Hybrid CNN-transformer Search Space with flexible down-sampling positions. Our Results and Trained Models Here is a summary of our searched models: Model MAdds Steptime Top-1 (%) Top-5 (%) Url BossNet-T0 w/o SE 3.4B 101ms 80.5 95.0 checkpoint BossNet-T0 3.4B 115ms 80.8 95.2 checkpoint BossNet-T0^ 5.7B 147ms 81.6 95.6 same as above BossNet-T1 7.9B 156ms […]

MaskCam MaskCam is a prototype reference design for a Jetson Nano-based smart camera system that measures crowd face mask usage in real-time, with all AI computation performed at the edge. MaskCam detects and tracks people in its field of view and determines whether they are wearing a mask via an object detection, tracking, and voting algorithm. It uploads statistics (not videos) to the cloud, where a web GUI can be used to monitor face mask compliance in the field of […]

bodywork-core Bodywork deploys machine learning projects developed in Python, to Kubernetes. It helps you: serve models as microservices execute batch jobs run reproducible pipelines On demand, or on a schedule. It automates repetitive DevOps tasks and frees machine learning engineers to focus on what they do best – solving data problems with machine learning. Where does Bodywork Fit? Bodywork is aimed at teams who want to deploy machine learning projects in containers. It will deliver your project’s Python modules directly […]

Vector Neurons We introduce a general framework built on top of what we call Vector Neurons for creating SO(3) equivariant neural networks. Extending neurons from single scalars to 3D vectors, our vector neurons transport SO(3) actions to latent spaces and provide a framework for building equivariance in common neural operations including linear layers, non-linearities, pooling, and normalization. Created by Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacchi, and Leonidas Guibas. Overview vnn is the author’s implementation of Vector […]

Scene Graph Benchmark in PyTorch 1.4 This project aims at providing the necessary building blocks for easily creating detection and segmentation models using PyTorch 1.0. Highlights Upgrad to pytorch 1.4 (can also upgrade to 1.7) Multi-GPU training and inference Batched inference: can perform inference using multiple images per batch per GPU. Fast and flexible tsv dataset format Remove FasterRCNN detector dependency: during relation head training, can plugin bounding boxes from any detector. Provides pre-trained models for different scene graph detection […]