Hierarchical Motion Estimation for Small Objects in Frame-Rate Up-Conversion

Thắng, Nguyễn Văn; Choi, Jung-Oh; Hong, Ju-Hyeon; Kim, Jin-Sung; Lee, Hyuk-Jae

doi:10.1109/access.2018.2875688

Cited by 8 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, FRUC has become a basic step to increase the frame rate of video in many IoT applications [7][8][9]. erefore, many works have been proposed to develop effective FRUC algorithms [10][11][12].…”

Section: Introductionmentioning

confidence: 99%

Machine-Type Video Communication Using Pretrained Network for Internet of Things

Hao

Sun

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

With the increasing demand for internet of things (IoT) applications, machine-type video communications have become an indispensable means of communication. It is changing the way we live and work. In machine-type video communications, the quality and delay of the video transmission should be guaranteed to satisfy the requirements of communication devices at the condition of limited resources. It is necessary to reduce the burden of transmitting video by losing frames at the video sender and then to increase the frame rate of transmitting video at the receiver. In this paper, based on the pretrained network, we proposed a frame rate up-conversion (FRUC) algorithm to guarantee low-latency video transmitting in machine-type video communications. At the IoT node, by periodically discarding the video frames, the video sequences are significantly compressed. At the IoT cloud, a pretrained network is used to extract the feature layers of the transmitted video frames, which is fused into the bidirectional matching to produce the motion vectors (MVs) of the losing frames, and according to the output MVs, the motion-compensated interpolation is implemented to recover the original frame rate of the video sequence. Experimental results show that the proposed FRUC algorithm effectively improve both objective and subjective qualities of the transmitted video sequences.

show abstract

Section: Introductionmentioning

confidence: 99%

Machine-Type Video Communication Using Pretrained Network for Internet of Things

Hao

Sun

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…ME is used to predict the Motion Vector Filed (MVF) between two adjacent frames, and MCI is used to interpolate the absent frame according to MVF output by ME. The visual quality of an interpolated frame depends heavily on the accuracy of ME algorithm used, so lots of works [4][5][6] have spared no effort to develop different ME algorithms. It is found that a high investment of computation promotes a high ME accuracy, so some approaches are needed to keep a good balance between ME accuracy and computational complexity.…”

Section: Motivation and Objectivementioning

confidence: 99%

A Low-Complex Frame Rate Up-Conversion with Edge-Preserved Filtering

et al. 2020

Electronics

View full text Add to dashboard Cite

The improvement of resolution of digital video requires a continuous increase of computation invested into Frame Rate Up-Conversion (FRUC). In this paper, we combine the advantages of Edge-Preserved Filtering (EPF) and Bidirectional Motion Estimation (BME) in an attempt to reduce the computational complexity. The inaccuracy of BME results from the existing similar structures in the texture regions, which can be avoided by using EPF to remove the texture details of video frames. EPF filters out by the high-frequency components, so each video frame can be subsampled before BME, at the same time, with the least accuracy degradation. EPF also preserves the edges, which prevents the deformation of object in the process of subsampling. Besides, we use predictive search to reduce the redundant search points according to the local smoothness of Motion Vector Field (MVF) to speed up BME. The experimental results show that the proposed FRUC algorithm brings good objective and subjective qualities of the interpolated frames with a low computational complexity.

show abstract

“…The image quality of an interpolated frame depends on the accuracy of the motion trajectories and the performance of the MCFI algorithm. A Block Matching Algorithm (BMA) is widely used for FRUC in Liquid Crystal Displays (LCDs) [33]- [36]. In [34] a hybrid adaptive non-selective block-based MEMC The associate editor coordinating the review of this manuscript and approving it for publication was Zhaoqing Pan .…”

Section: Introductionmentioning

confidence: 99%

“…In [34] a hybrid adaptive non-selective block-based MEMC The associate editor coordinating the review of this manuscript and approving it for publication was Zhaoqing Pan . approach is proposed for general cases in MCFI meanwhile a specific semi-global ME method for repetition-like patterns [35] and a hierarchical ME algorithm for small objects [36] are applied independently for challenging cases in MCFI. However, the motion vectors estimated by a BMA are not always the true trajectories of objects because the objective function of BMA is to minimize matching error but does not cover the motion constraints of objects.…”

Section: Introductionmentioning

confidence: 99%

A Stacked Deep MEMC Network for Frame Rate Up Conversion and its Application to HEVC

Thắng

Lee

2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

Optical flows and video frame interpolation are considered as a chicken-egg problem such that one problem affects the other and vice versa. This paper presents a stack of deep networks to estimate intermediate optical flows from the very first intermediate synthesized frame and later generate the very end interpolated frame by combining the very first one and two learned intermediate optical flows based warped frames. The primary benefit is that it glues two problems into a single comprehensive framework that learns altogether by using both an analysis-by-synthesis technique for optical flow estimation and Convolutional Neural Networks (CNN) kernels-based frame synthesis. The proposed network is the first attempt to merge two previous branches of previous approaches, optical flow-based synthesis and CNN kernels-based synthesis into a comprehensive network. Experiments are carried out with various challenging datasets, all showing that the proposed network outperforms the state-of-the-art methods with significant margins for video frame interpolation and the estimated optical flows are more accurate for challenging movements. Furthermore, the proposed Motion Estimation Motion Compensation (MEMC) network shows its outstanding enhancement of the quality of compressed videos. INDEX TERMS Frame rate up conversion, video frame interpolation, optical flow, HEVC, MEMC, CNN, convolutional neural networks.

show abstract

Hierarchical Motion Estimation for Small Objects in Frame-Rate Up-Conversion

Cited by 8 publications

References 21 publications

Machine-Type Video Communication Using Pretrained Network for Internet of Things

Machine-Type Video Communication Using Pretrained Network for Internet of Things

A Low-Complex Frame Rate Up-Conversion with Edge-Preserved Filtering

A Stacked Deep MEMC Network for Frame Rate Up Conversion and its Application to HEVC

Contact Info

Product

Resources

About