LIP: Learning Instance Propagation for Video Object Segmentation

Lyu, Ye; Vosselman, G.; Xia, Gui-Song; Yang, Michael Ying

doi:10.1109/iccvw.2019.00335

Cited by 3 publications

(2 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the aforementioned methods with either 2D or 3D convolutions have limited temporal receptive fields and therefore cannot adequately capture variable temporal dependencies. On the other hand, a few recent works attempt to explicitly model temporal relationships and demonstrate promising results in several tasks, to name a few, temporal relational reasoning [40]- [44], object detection and tracking [6]- [9], event recognition [45]- [47], video segmentation [48]- [50], dynamic texture recognition [51], and spatiotemporal learning [52], [53].…”

Section: Introductionmentioning

confidence: 99%

FuTH-Net: Fusing Temporal Relations and Holistic Features for Aerial Video Classification

Mou²,

Hua³

et al. 2022

IEEE Trans. Geosci. Remote Sensing

Self Cite

View full text Add to dashboard Cite

Unmanned aerial vehicles (UAVs) are now widely applied to data acquisition due to its low cost and fast mobility. With the increasing volume of aerial videos, the demand for automatically parsing these videos is surging. To achieve this, current researches mainly focus on extracting a holistic feature with convolutions along both spatial and temporal dimensions. However, these methods are limited by small temporal receptive fields and cannot adequately capture long-term temporal dependencies which are important for describing complicated dynamics. In this paper, we propose a novel deep neural network, termed FuTH-Net, to model not only holistic features, but also temporal relations for aerial video classification. Furthermore, the holistic features are refined by the multi-scale temporal relations in a novel fusion module for yielding more discriminative video representations. More specially, FuTH-Net employs a twopathway architecture: (1) a holistic representation pathway to learn a general feature of both frame appearances and shortterm temporal variations and (2) a temporal relation pathway to capture multi-scale temporal relations across arbitrary frames, providing long-term temporal dependencies. Afterwards, a novel fusion module is proposed to spatiotemporal integrate the two features learned from the two pathways. Our model is evaluated on two aerial video classification datasets, ERA and Drone-Action, and achieves the state-of-the-art results. This demonstrates its effectiveness and good generalization capacity across different recognition tasks (event classification and human action recognition). To facilitate further research, we release the code at https://gitlab.lrz.de/ai4eo/reasoning/futh-net.

show abstract

Section: Introductionmentioning

confidence: 99%

FuTH-Net: Fusing Temporal Relations and Holistic Features for Aerial Video Classification

Mou²,

Hua³

et al. 2022

IEEE Trans. Geosci. Remote Sensing

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 99%

FuTH-Net: Fusing Temporal Relations and Holistic Features for Aerial Video Classification

Pu¹,

Mou²,

Hua³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

This work has been accepted by IEEE TGRS for publication. Unmanned aerial vehicles (UAVs) are now widely applied to data acquisition due to its low cost and fast mobility. With the increasing volume of aerial videos, the demand for automatically parsing these videos is surging. To achieve this, current researches mainly focus on extracting a holistic feature with convolutions along both spatial and temporal dimensions. However, these methods are limited by small temporal receptive fields and cannot adequately capture long-term temporal dependencies which are important for describing complicated dynamics. In this paper, we propose a novel deep neural network, termed FuTH-Net, to model not only holistic features, but also temporal relations for aerial video classification. Furthermore, the holistic features are refined by the multi-scale temporal relations in a novel fusion module for yielding more discriminative video representations. More specially, FuTH-Net employs a twopathway architecture: (1) a holistic representation pathway to learn a general feature of both frame appearances and shortterm temporal variations and (2) a temporal relation pathway to capture multi-scale temporal relations across arbitrary frames, providing long-term temporal dependencies. Afterwards, a novel fusion module is proposed to spatiotemporal integrate the two features learned from the two pathways. Our model is evaluated on two aerial video classification datasets, ERA and Drone-Action, and achieves the state-of-the-art results. This demonstrates its effectiveness and good generalization capacity across different recognition tasks (event classification and human action recognition). To facilitate further research, we release the code at https://gitlab.lrz.de/ai4eo/reasoning/futh-net.

show abstract