Beyond Image to Depth: Improving Depth Prediction using Echoes
beyond-image-to-depth We address the problem of estimating depth with multi modal audio visual data. Inspired by the ability of animals, such as bats and dolphins, to infer distance of objects with echolocation, we propose an end-to-end deep learning based pipeline utilizing RGB images, binaural echoes and estimated material properties of various objects within a scene for the task of depth estimation. Requirements The code is tesed with – Python 3.6 – PyTorch 1.6.0 – Numpy 1.19.5 Dataset Replica-VisualEchoes can be […]
Read more