A Survey on Vision Transformer

Kai Han, Yunhe Wang, Hanting Chen et al.

2022 · IEEE Transactions on Pattern Analysis and Machine Intelligence · 4,652 citations

Transformer, first applied to the field of natural language processing, is a type of deep neural network mainly based on the self-attention mechanism. Thanks to its strong representation capabilities, researchers are looking at ways to apply transformer to computer vision tasks. In a variety of visual benchmarks, transformer-based models perform similar to or better than other types of networks such as convolutional and recurrent neural networks. Given its high performance and less need for vision-specific inductive bias, transformer is receiving more and more attention from the computer visi…

Read the paper →

Explore this paper's citation graph on Constellation.