STCN We present Space-Time Correspondence Networks (STCN) as the new, effective, and efficient framework to model space-time correspondences in the context of video object segmentation. STCN achieves SOTA results on multiple benchmarks while running fast at 20+ FPS without bells and whistles. Its speed is even higher with mixed precision. Despite its effectiveness, the network itself is very simple with lots of room for improvement. See the paper for technical details. A Gentle Introduction There are two main contributions: STCN […]

nsff_pl Neural Scene Flow Fields using pytorch-lightning. This repo reimplements the NSFF idea, but modifies several operations based on observation of NSFF results and discussions with the authors. For discussion details, please see the issues of the original repo. The code is based on my previous implementation. The main modifications are the followings: Remove the blending weight in static NeRF. I adopt the addition strategy in NeRF-W. Compose static dynamic also in image warping. Implementation details are in models/rendering.py. These […]

omd JAX code for the paper “Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation” Summary Model based reinforcement learning typically trains the dynamics and reward functions by minimizing the error of predictions.The error is only a proxy to maximizing the sum of rewards, the ultimate goal of the agent, leading to the objective mismatch.We propose an end-to-end algorithm called Optimal Model Design (OMD) that optimizes the returns directly for model learning.OMD leverages the implicit function theorem to optimize the model parameters […]