nsff_pl Neural Scene Flow Fields using pytorch-lightning. This repo reimplements the NSFF idea, but modifies several operations based on observation of NSFF results and discussions with the authors. For discussion details, please see the issues of the original repo. The code is based on my previous implementation. The main modifications are the followings: Remove the blending weight in static NeRF. I adopt the addition strategy in NeRF-W. Compose static dynamic also in image warping. Implementation details are in models/rendering.py. These […]

omd JAX code for the paper “Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation” Summary Model based reinforcement learning typically trains the dynamics and reward functions by minimizing the error of predictions.The error is only a proxy to maximizing the sum of rewards, the ultimate goal of the agent, leading to the objective mismatch.We propose an end-to-end algorithm called Optimal Model Design (OMD) that optimizes the returns directly for model learning.OMD leverages the implicit function theorem to optimize the model parameters […]

merlot MERLOT is a model for learning what we are calling “neural script knowledge” — representations about what is going on in videos, spanning multiple video frames with associated captions. What’s here We are releasing the following: Code for the MERLOT model (in model/, with data processing in data/ Code for running MERLOT over visual story ordering. We plan to release: Information about the videos used in this work Code for adapting the model to other tasks (not strictly needed, […]

StyleSpeech PyTorch Implementation of Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation. [x] StyleSpeech (naive branch) [x] Meta-StyleSpeech (main branch) Dependencies You can install the Python dependencies with pip3 install -r requirements.txt Inference You have to download pretrained models and put them in output/ckpt/LibriTTS/. For English single-speaker TTS, run python3 synthesize.py –text “YOUR_DESIRED_TEXT” –ref_audio path/to/reference_audio.wav –restore_step 200000 –mode single -p config/LibriTTS/preprocess.yaml -m config/LibriTTS/model.yaml -t config/LibriTTS/train.yaml The generated utterances will be put in output/result/. Your synthesized speech will have ref_audio‘s style. Batch Inference […]

BCES Python module for performing robust linear regression on (X,Y) data points where both X and Y have measurement errors. The fitting method is the bivariate correlated errors and intrinsic scatter (BCES) and follows the description given in Akritas & Bershady. 1996, ApJ. Some of the advantages of BCES regression compared to ordinary least squares fitting (quoted from Akritas & Bershady 1996): it allows for measurement errors on both variables it permits the measurement errors for the two variables to […]