File Download
  Links for fulltext
     (May Require Subscription)
Supplementary

Conference Paper: Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow

TitleEvery Frame Counts: Joint Learning of Video Segmentation and Optical Flow
Authors
Issue Date2020
PublisherAAAI Press. The Journal's web site is located at https://aaai.org/Library/AAAI/aaai-library.php
Citation
Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI-20), New York, NY, USA, 7-12 February 2020, v. 34 n. 7, p. 10713-10720 How to Cite?
AbstractA major challenge for video semantic segmentation is the lack of labeled data. In most benchmark datasets, only one frame of a video clip is annotated, which makes most supervised methods fail to utilize information from the rest of the frames. To exploit the spatio-temporal information in videos, many previous works use pre-computed optical flows, which encode the temporal consistency to improve the video segmentation. However, the video segmentation and optical flow estimation are still considered as two separate tasks. In this paper, we propose a novel framework for joint video semantic segmentation and optical flow estimation. Semantic segmentation brings semantic information to handle occlusion for more robust optical flow estimation, while the non-occluded optical flow provides accurate pixel-level temporal correspondences to guarantee the temporal consistency of the segmentation. Moreover, our framework is able to utilize both labeled and unlabeled frames in the video through joint training, while no additional calculation is required in inference. Extensive experiments show that the proposed model makes the video semantic segmentation and optical flow estimation benefit from each other and outperforms existing methods under the same settings in both tasks.
DescriptionAAAI Technical Track: Vision
Persistent Identifierhttp://hdl.handle.net/10722/284159
ISSN

 

DC FieldValueLanguage
dc.contributor.authorDing, M-
dc.contributor.authorWang, Z-
dc.contributor.authorZhou, B-
dc.contributor.authorShi, J-
dc.contributor.authorLu, Z-
dc.contributor.authorLuo, P-
dc.date.accessioned2020-07-20T05:56:33Z-
dc.date.available2020-07-20T05:56:33Z-
dc.date.issued2020-
dc.identifier.citationProceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI-20), New York, NY, USA, 7-12 February 2020, v. 34 n. 7, p. 10713-10720-
dc.identifier.issn2159-5399-
dc.identifier.urihttp://hdl.handle.net/10722/284159-
dc.descriptionAAAI Technical Track: Vision-
dc.description.abstractA major challenge for video semantic segmentation is the lack of labeled data. In most benchmark datasets, only one frame of a video clip is annotated, which makes most supervised methods fail to utilize information from the rest of the frames. To exploit the spatio-temporal information in videos, many previous works use pre-computed optical flows, which encode the temporal consistency to improve the video segmentation. However, the video segmentation and optical flow estimation are still considered as two separate tasks. In this paper, we propose a novel framework for joint video semantic segmentation and optical flow estimation. Semantic segmentation brings semantic information to handle occlusion for more robust optical flow estimation, while the non-occluded optical flow provides accurate pixel-level temporal correspondences to guarantee the temporal consistency of the segmentation. Moreover, our framework is able to utilize both labeled and unlabeled frames in the video through joint training, while no additional calculation is required in inference. Extensive experiments show that the proposed model makes the video semantic segmentation and optical flow estimation benefit from each other and outperforms existing methods under the same settings in both tasks.-
dc.languageeng-
dc.publisherAAAI Press. The Journal's web site is located at https://aaai.org/Library/AAAI/aaai-library.php-
dc.relation.ispartofProceedings of the AAAI Conference on Artificial Intelligence-
dc.rightsCopyright (c) 2019 Association for the Advancement of Artificial Intelligence-
dc.titleEvery Frame Counts: Joint Learning of Video Segmentation and Optical Flow-
dc.typeConference_Paper-
dc.identifier.emailLuo, P: pluo@hku.hk-
dc.identifier.authorityLuo, P=rp02575-
dc.description.naturelink_to_OA_fulltext-
dc.identifier.doi10.1609/aaai.v34i07.6699-
dc.identifier.hkuros311019-
dc.identifier.volume34-
dc.identifier.issue7-
dc.identifier.spage10713-
dc.identifier.epage10720-
dc.publisher.placeUnited States-

Export via OAI-PMH Interface in XML Formats


OR


Export to Other Non-XML Formats