File Download

There are no files associated with this item.

  Links for fulltext
     (May Require Subscription)
Supplementary

Conference Paper: Improved techniques for training adaptive deep networks

TitleImproved techniques for training adaptive deep networks
Authors
Issue Date2019
Citation
Proceedings of the IEEE International Conference on Computer Vision, 2019, v. 2019-October, p. 1891-1900 How to Cite?
Abstract© 2019 IEEE. Adaptive inference is a promising technique to improve the computational efficiency of deep models at test time. In contrast to static models which use the same computation graph for all instances, adaptive networks can dynamically adjust their structure conditioned on each input. While existing research on adaptive inference mainly focuses on designing more advanced architectures, this paper investigates how to train such networks more effectively. Specifically, we consider a typical adaptive deep network with multiple intermediate classifiers. We present three techniques to improve its training efficacy from two aspects: 1) a Gradient Equilibrium algorithm to resolve the conflict of learning of different classifiers; 2) an Inline Subnetwork Collaboration approach and a One-for-all Knowledge Distillation algorithm to enhance the collaboration among classifiers. On multiple datasets (CIFAR-10, CIFAR-100 and ImageNet), we show that the proposed approach consistently leads to further improved efficiency on top of state-of-the-art adaptive deep networks.
Persistent Identifierhttp://hdl.handle.net/10722/281976
ISSN

 

DC FieldValueLanguage
dc.contributor.authorLi, Hao-
dc.contributor.authorZhang, Hong-
dc.contributor.authorQi, Xiaojuan-
dc.contributor.authorRuigang, Yang-
dc.contributor.authorHuang, Gao-
dc.date.accessioned2020-04-09T09:19:17Z-
dc.date.available2020-04-09T09:19:17Z-
dc.date.issued2019-
dc.identifier.citationProceedings of the IEEE International Conference on Computer Vision, 2019, v. 2019-October, p. 1891-1900-
dc.identifier.issn1550-5499-
dc.identifier.urihttp://hdl.handle.net/10722/281976-
dc.description.abstract© 2019 IEEE. Adaptive inference is a promising technique to improve the computational efficiency of deep models at test time. In contrast to static models which use the same computation graph for all instances, adaptive networks can dynamically adjust their structure conditioned on each input. While existing research on adaptive inference mainly focuses on designing more advanced architectures, this paper investigates how to train such networks more effectively. Specifically, we consider a typical adaptive deep network with multiple intermediate classifiers. We present three techniques to improve its training efficacy from two aspects: 1) a Gradient Equilibrium algorithm to resolve the conflict of learning of different classifiers; 2) an Inline Subnetwork Collaboration approach and a One-for-all Knowledge Distillation algorithm to enhance the collaboration among classifiers. On multiple datasets (CIFAR-10, CIFAR-100 and ImageNet), we show that the proposed approach consistently leads to further improved efficiency on top of state-of-the-art adaptive deep networks.-
dc.languageeng-
dc.relation.ispartofProceedings of the IEEE International Conference on Computer Vision-
dc.titleImproved techniques for training adaptive deep networks-
dc.typeConference_Paper-
dc.description.naturelink_to_subscribed_fulltext-
dc.identifier.doi10.1109/ICCV.2019.00198-
dc.identifier.scopuseid_2-s2.0-85081925431-
dc.identifier.volume2019-October-
dc.identifier.spage1891-
dc.identifier.epage1900-

Export via OAI-PMH Interface in XML Formats


OR


Export to Other Non-XML Formats