Efficient Outdoor Video Semantic Segmentation Using Feedback-Based Fully Convolution Neural Network

Wong, Chi-Chong; Gan, Yanfen; Vong, Chi‐Man

doi:10.1109/tii.2019.2950031

Cited by 10 publications

(8 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unlike traditional CNNs that employ big convolution kernels to extract features, VGGNet utilizes several 3 * 3 small convolution kernels for feature extraction. Hence, VGGNet can extract richer features and reduce the calculation amount significantly [26][27][28].…”

Section: Optimization Of Vggnet Cnn Model Cat's Visual Cortex Theory ...mentioning

confidence: 99%

The Fusion Application of Deep Learning Biological Image Visualization Technology and Human-Computer Interaction Intelligent Robot in Dance Movements

Jin

Wen

Xie

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

The paper aims to apply the deep learning-based image visualization technology to extract, recognize, and analyze human skeleton movements and evaluate the effect of the deep learning-based human-computer interaction (HCI) system. Dance education is researched. Firstly, the Visual Geometry Group Network (VGGNet) is optimized using Convolutional Neural Network (CNN). Then, the VGGNet extracts the human skeleton movements in the OpenPose database. Secondly, the Long Short-Term Memory (LSTM) network is optimized and recognizes human skeleton movements. Finally, an HCI system for dance education is designed based on the extraction and recognition methods of human skeleton movements. Results demonstrate that the highest extraction accuracy is 96%, and the average recognition accuracy of different dance movements is stable. The effectiveness of the proposed model is verified. The recognition accuracy of the optimized F-Multiple LSTMs is increased to 88.9%, suitable for recognizing human skeleton movements. The dance education HCI system’s interactive accuracy built by deep learning-based visualization technology reaches 92%; the overall response time is distributed between 5.1 s and 5.9 s. Hence, the proposed model has excellent instantaneity. Therefore, the deep learning-based image visualization technology has enormous potential in human movement recognition, and combining deep learning and HCI plays a significant role.

show abstract

Section: Optimization Of Vggnet Cnn Model Cat's Visual Cortex Theory ...mentioning

confidence: 99%

The Fusion Application of Deep Learning Biological Image Visualization Technology and Human-Computer Interaction Intelligent Robot in Dance Movements

Jin

Wen

Xie

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…By using such a coarse segmentation, objects can be identified from intra-class semantic regions. Subsequently, various object-level feature representations can be extracted from an image by using Equation (1).…”

Section: Hierarchical Feature Alignment Schemementioning

confidence: 99%

“…Deep neural networks (DNNs) have achieved remarkable performance in computer vision, especially in semantic segmentation [1]. Semantic segmentation is to assign the prediction of each pixel in an image.…”

Section: Introductionmentioning

confidence: 99%

Synthetic to Realistic Imbalanced Domain Adaption for Urban Scene Perception

Hua

2022

IEEE Trans. Ind. Inf.

View full text Add to dashboard Cite

Deep neural networks (DNNs) technique has achieved impressive performance on semantic segmentation, while its training process requires a large amount of pixel-wise labelled data. Domain adaptation, as a promising solution, can break the restriction by training the model on synthetic data, and generalising it in real-world data. However, there is still a lack of attention paid to the imbalance problems on semantic segmentation adaptation, including the imbalance problem between i) source and target data, ii) different classes. To solve these problems, a progressive hierarchical feature alignment method is proposed in this paper. To alleviate the data imbalance problem, the network is progressively trained by the data from multisource domains, so as to obtain domain-invariant features. To address the class imbalance problem, the features are aligned hierarchically across domains. According to the experimental results, our method shows the competitive adapted segmentation performance on three benchmark datasets.

show abstract

“…In [ 7 ], a fully connected VGG16 neural network was proposed for real-time path finding intended to help visually impaired or blind people. In [ 8 ], a novel neural-network model called F 2 2CNN which integrates a feedback mechanism into deep FCNN was proposed for outdoor video semantic segmentation.…”

Section: Introductionmentioning

confidence: 99%

FASSVid: Fast and Accurate Semantic Segmentation for Video Sequences

Portillo-Portillo

Olivares-Mercado

Alvarado

et al. 2022

Entropy

View full text Add to dashboard Cite

Most of the methods for real-time semantic segmentation do not take into account temporal information when working with video sequences. This is counter-intuitive in real-world scenarios where the main application of such methods is, precisely, being able to process frame sequences as quickly and accurately as possible. In this paper, we address this problem by exploiting the temporal information provided by previous frames of the video stream. Our method leverages a previous input frame as well as the previous output of the network to enhance the prediction accuracy of the current input frame. We develop a module that obtains feature maps rich in change information. Additionally, we incorporate the previous output of the network into all the decoder stages as a way of increasing the attention given to relevant features. Finally, to properly train and evaluate our methods, we introduce CityscapesVid, a dataset specifically designed to benchmark semantic video segmentation networks. Our proposed network, entitled FASSVid improves the mIoU accuracy performance over a standard non-sequential baseline model. Moreover, FASSVid obtains state-of-the-art inference speed and competitive mIoU results compared to other state-of-the-art lightweight networks, with significantly lower number of computations. Specifically, we obtain 71% of mIoU in our CityscapesVid dataset, running at 114.9 FPS on a single NVIDIA GTX 1080Ti and 31 FPS on the NVIDIA Jetson Nano embedded board with images of size 1024×2048 and 512×1024, respectively.

show abstract

Efficient Outdoor Video Semantic Segmentation Using Feedback-Based Fully Convolution Neural Network

Cited by 10 publications

References 23 publications

The Fusion Application of Deep Learning Biological Image Visualization Technology and Human-Computer Interaction Intelligent Robot in Dance Movements

The Fusion Application of Deep Learning Biological Image Visualization Technology and Human-Computer Interaction Intelligent Robot in Dance Movements

Synthetic to Realistic Imbalanced Domain Adaption for Urban Scene Perception

FASSVid: Fast and Accurate Semantic Segmentation for Video Sequences

Contact Info

Product

Resources

About