File Download

There are no files associated with this item.

  Links for fulltext
     (May Require Subscription)
Supplementary

Conference Paper: Exploring Self-attention for Image Recognition

TitleExploring Self-attention for Image Recognition
Authors
Issue Date2020
Citation
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2020, p. 10073-10082 How to Cite?
AbstractRecent work has shown that self-attention can serve as a basic building block for image recognition models. We explore variations of self-attention and assess their effectiveness for image recognition. We consider two forms of self-attention. One is pairwise self-attention, which generalizes standard dot-product attention and is fundamentally a set operator. The other is patchwise self-attention, which is strictly more powerful than convolution. Our pairwise self-attention networks match or outperform their convolutional counterparts, and the patchwise models substantially outperform the convolutional baselines. We also conduct experiments that probe the robustness of learned representations and conclude that self-attention networks may have significant benefits in terms of robustness and generalization.
Persistent Identifierhttp://hdl.handle.net/10722/303697
ISSN
2023 SCImago Journal Rankings: 10.331

 

DC FieldValueLanguage
dc.contributor.authorZhao, Hengshuang-
dc.contributor.authorJia, Jiaya-
dc.contributor.authorKoltun, Vladlen-
dc.date.accessioned2021-09-15T08:25:50Z-
dc.date.available2021-09-15T08:25:50Z-
dc.date.issued2020-
dc.identifier.citationProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2020, p. 10073-10082-
dc.identifier.issn1063-6919-
dc.identifier.urihttp://hdl.handle.net/10722/303697-
dc.description.abstractRecent work has shown that self-attention can serve as a basic building block for image recognition models. We explore variations of self-attention and assess their effectiveness for image recognition. We consider two forms of self-attention. One is pairwise self-attention, which generalizes standard dot-product attention and is fundamentally a set operator. The other is patchwise self-attention, which is strictly more powerful than convolution. Our pairwise self-attention networks match or outperform their convolutional counterparts, and the patchwise models substantially outperform the convolutional baselines. We also conduct experiments that probe the robustness of learned representations and conclude that self-attention networks may have significant benefits in terms of robustness and generalization.-
dc.languageeng-
dc.relation.ispartofProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition-
dc.titleExploring Self-attention for Image Recognition-
dc.typeConference_Paper-
dc.description.naturelink_to_subscribed_fulltext-
dc.identifier.doi10.1109/CVPR42600.2020.01009-
dc.identifier.scopuseid_2-s2.0-85090597566-
dc.identifier.spage10073-
dc.identifier.epage10082-

Export via OAI-PMH Interface in XML Formats


OR


Export to Other Non-XML Formats