ViLT Code for the ICML 2021 (long talk) paper: “ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision” Install pip install -r requirements.txt pip install -e . Download Pretrained Weights We provide five pretrained weights ViLT-B/32 Pretrained with MLM+ITM for 200k steps on GCC+SBU+COCO+VG (ViLT-B/32 200k) link ViLT-B/32 200k finetuned on VQAv2 link ViLT-B/32 200k finetuned on NLVR2 link ViLT-B/32 200k finetuned on COCO IR/TR link ViLT-B/32 200k finetuned on F30K IR/TR link Out-of-the-box MLM + Visualization Demo pip install gradio==1.6.4 […]

VRCNet Real-scanned point clouds are often incomplete due to viewpoint, occlusion, and noise. Existing point cloud completion methods tend to generate global shape skeletons and hence lack fine local details. Furthermore, they mostly learn a deterministic partial-to-complete mapping, but overlook structural relations in man-made objects. To tackle these challenges, this paper proposes a variational framework, Variational Relational point Completion network (VRCNet) with two appealing properties: 1) Probabilistic Modeling. In particular, we propose a dual-path architecture to enable principled probabilistic modeling […]

Do You Even Need Attention? A Stack of Feed-Forward Layers Does Surprisingly Well on ImageNet TL;DR We replace the attention layer in a vision transformer with a feed-forward layer and find that it still works quite well on ImageNet. Abstract The strong performance of vision transformers on image classification and other vision tasks is often attributed to the design of their multi-head attention layers. However, the extent to which attention is responsible for this strong performance remains unclear. In this […]