Intriguing Properties of Vision Transformers
Muzammal Naseer, Kanchana Ranasinghe, Salman Hameed Khan et al.
2021 · arXiv (Cornell University) · 432 citations
Vision transformers (ViT) have demonstrated impressive performance across various machine vision problems. These models are based on multi-head self-attention mechanisms that can flexibly attend to a sequence of image patches to encode contextual cues. An important question is how such flexibility in attending image-wide context conditioned on a given patch can facilitate handling nuisances in natural images e.g., severe occlusions, domain shifts, spatial permutations, adversarial and natural perturbations. We systematically study this question via an extensive set of experiments encompassing…
Explore this paper's citation graph on Constellation.