Extreme Low-Resolution Activity Recognition Using a Super-Resolution-Oriented Generative Adversarial Network

Hou, Mingzheng; Song, Lixin; Zhou, Jiliu; Zhang, Yi; Feng, Ziliang

doi:10.3390/mi12060670

Cited by 10 publications

(8 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Action recognition of low-resolution videos. To recognize low-resolution videos, methods have been proposed to apply super-resolution to low-resolution videos to convert them in order to high-resolution [5,17], as in the case of images. In addition, some method deals with the low frame rate [29], which is smaller than a typical frame rate of 30 fps.…”

Section: Related Workmentioning

confidence: 99%

“…For image recognition (not action recognition), there are various studies on the effect of image quality, including object and face recognition at low resolution [4,31,36,40], investigations of the effect of recognition models on various image quality degradation such as blur, noise, contrast change, and JPEG compression [8,16]. For action recognition, several methods for low-resolution videos have been proposed [5,17]; however, the approaches modulate the quality degradation problem by image enhancement to ignore the effect of the video quality degradation.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the Performance Evaluation of Action Recognition Models on Transcoded Low Quality Videos

Otani¹,

Hashiguchi²,

Omi³

et al. 2022

Preprint

View full text Add to dashboard Cite

In the design of action recognition models, the quality of videos in the dataset is an important issue, however the trade-off between the quality and performance is often ignored. In general, action recognition models are trained and tested on high-quality videos, but in actual situations where action recognition models are deployed, sometimes it might not be assumed that the input videos are of high quality. In this study, we report qualitative evaluations of action recognition models for the quality degradation associated with transcoding by JPEG and H.264/AVC. Experimental results are shown for evaluating the performance of pre-trained models on the transcoded validation videos of Kinetics400. The models are also trained on the transcoded training videos. From these results, we quantitatively show the degree of degradation of the model performance with respect to the degradation of the video quality.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

On the Performance Evaluation of Action Recognition Models on Transcoded Low Quality Videos

Otani¹,

Hashiguchi²,

Omi³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…For image recognition (not action recognition), there are various studies on the effect of image quality, including object and face recognition at low resolution [19]- [22], investigations of the effect of recognition models on various image quality degradation such as blur, noise, contrast change, and JPEG compression [23], [24]. For action recognition, several methods for low-resolution videos have been proposed [25], [26]; however, the approaches modulate the quality degradation problem by image enhancement to ignore the effect of the video quality degradation. Therefore, in this study, we quantitatively evaluate the trade-off between the video quality and the performance of action recognition models on Kinetics400 [5], an action dataset commonly used for evaluating model performance.…”

Section: Introductionmentioning

confidence: 99%

Performance Evaluation of Action Recognition Models on Low Quality Videos

et al. 2022

View full text Add to dashboard Cite

In the design of action recognition models, the quality of videos is an important issue; however, the trade-off between the quality and performance is often ignored. In general, action recognition models are trained on high-quality videos, hence it is not known how the model performance degrades when tested on low-quality videos, and how much the quality of training videos affects the performance. The issue of video quality is important, however, it has not been studied so far. The goal of this study is to show the trade-off between the performance and the quality of training and test videos by quantitative performance evaluation of several action recognition models for transcoded videos in different qualities. First, we show how the video quality affects the performance of pre-trained models. We transcode the original validation videos of Kinetics400 by changing quality control parameters of JPEG (compression strength) and H.264/AVC (CRF). Then we use the transcoded videos to validate the pre-trained models. Second, we show how the models perform when trained on transcoded videos. We transcode the original training videos of Kinetics400 by changing the quality parameters of JPEG and H.264/AVC. Then we train the models on the transcoded training videos and validate them with the original and transcoded validation videos. Experimental results with JPEG transcoding show that there is no severe performance degradation (up to −1.5%) for compression strength smaller than 70 where no quality degradation is visually observed, and for larger than 80 the performance degrades linearly with respect to the quality index. Experiments with H.264/AVC transcoding show that there is no significant performance loss (up to −1%) with CRF30 while the total size of video files is reduced to 30%. In summary, the video quality doesn't have a large impact on the performance of action recognition models unless the quality degradation is severe and visible. This enables us to transcode the training and validation videos and reduce the file sizes to one-third of the original videos.

show abstract

“…Woo et al [25] proposed a convolutional block attention model(CBAM), which obtained satisfied result. Furthermore, Hou et al [26] adopts the alternative upscaled and downscaled layers in the generator with relativistic disciminator to capture the high-resolution image from extreme low-resolution image. Moreover, Zhang et al [27] presented a fast medical super resolution (FMISR) method, which contributes to the mini-network and uses the sub-pixel convolution layer.…”

Section: Introductionmentioning

confidence: 99%

Wavelet Frequency Separation Attention Network for Chest X-ray Image Super-Resolution

She

Liu

2021

Micromachines

View full text Add to dashboard Cite

Medical imaging is widely used in medical diagnosis. The low-resolution image caused by high hardware cost and poor imaging technology leads to the loss of relevant features and even fine texture. Obtaining high-quality medical images plays an important role in disease diagnosis. A surge of deep learning approaches has recently demonstrated high-quality reconstruction for medical image super-resolution. In this work, we propose a light-weight wavelet frequency separation attention network for medical image super-resolution (WFSAN). WFSAN is designed with separated-path for wavelet sub-bands to predict the wavelet coefficients, considering that image data characteristics are different in the wavelet domain and spatial domain. In addition, different activation functions are selected to fit the coefficients. Inputs comprise approximate sub-bands and detail sub-bands of low-resolution wavelet coefficients. In the separated-path network, detail sub-bands, which have more sparsity, are trained to enhance high frequency information. An attention extension ghost block is designed to generate the features more efficiently. All results obtained from fusing layers are contracted to reconstruct the approximate and detail wavelet coefficients of the high-resolution image. In the end, the super-resolution results are generated by inverse wavelet transform. Experimental results show that WFSAN has competitive performance against state-of-the-art lightweight medical imaging methods in terms of quality and quantitative metrics.

show abstract

Extreme Low-Resolution Activity Recognition Using a Super-Resolution-Oriented Generative Adversarial Network

Cited by 10 publications

References 29 publications

On the Performance Evaluation of Action Recognition Models on Transcoded Low Quality Videos

On the Performance Evaluation of Action Recognition Models on Transcoded Low Quality Videos

Performance Evaluation of Action Recognition Models on Low Quality Videos

Wavelet Frequency Separation Attention Network for Chest X-ray Image Super-Resolution

Contact Info

Product

Resources

About