Jitter-Robust Video Retargeting With Kalman Filter And Attention Saliency Fusion Network

Nam, Hyunwoo; Park, Dubok; Jeon, Kang-Won

doi:10.1109/icip40778.2020.9191354

Cited by 6 publications

(5 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The proposed method was implemented using Python and PyTorch. In the publicly available source code we release 1 , we have also implemented a technique to assess the quality of the cropped version and opt to resort to padding in the case where the results of cropping are unsatisfactory due to the nature of the video's content, i.e. only a small percentage of the salient visual information can be covered by any crop window.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…The video aspect ratio transformation algorithms of the literature can be divided in three main categories: a) warping [1,2], b) cropping [3,4,5,6], and c) seam carving [7,8].…”

Section: Related Workmentioning

confidence: 99%

“…The videos can be found in the website of the DHF1k dataset 2 . Our crop window ground-truth annotations for each video of the Retar-getVid dataset are available online 1 . They are in the form of text files, where the i-th line contains the top-left and bottomright coordinates of the crop window for the i-th frame.…”

Section: Retargetvid Dataset Creationmentioning

confidence: 99%

See 2 more Smart Citations

A Fast Smart-Cropping Method and Dataset for Video Retargeting

Apostolidis

Mezaris

2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

In this paper a method that re-targets a video to a different aspect ratio using cropping is presented. We argue that cropping methods are more suitable for video aspect ratio transformation when the minimization of semantic distortions is a prerequisite. For our method, we utilize visual saliency to find the image regions of attention, and we employ a filteringthrough-clustering technique to select the main region of focus. We additionally introduce the first publicly available benchmark dataset for video cropping, annotated by 6 human subjects. Experimental evaluation on the introduced dataset shows the competitiveness of our method.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

“…The video aspect ratio transformation algorithms of the literature can be divided in three main categories: a) warping [1,2], b) cropping [3,4,5,6], and c) seam carving [7,8].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Fast Smart-Cropping Method and Dataset for Video Retargeting

Apostolidis

Mezaris

2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

“…The video aspect-ratio transformation algorithms of the literature can be divided in three main categories: i) warping [176,177], ii) cropping [178][179][180][181], and i) seam carving [182,183]. Warping methods, instead of resizing the entire video frame uniformly, determine scaling factors in a content-adaptive way: the frame is divided using a grid and important regions are left untouched, while scale factors are applied to other less-important areas.…”

Section: Transforming the Contentmentioning

confidence: 99%

Data-driven personalisation of television content: a survey

et al. 2022

View full text Add to dashboard Cite

show abstract

“…NCV [17] proposed by Wolf et al combines the gradient map, face detection, and foreground motion to produce importance map and then uses mesh deformation to realize video retargeting. Nam et al [24] proposed a video retargeting method based on Kalman filter and saliency fusion to reduce video content jitter, so as to enhance the robustness of video retargeting. Wang et al [25] proposed a multi-operator method based on improved seam carving to realize video retargeting.…”

Section: Introductionmentioning

confidence: 99%

Visual Attention and Motion Estimation-Based Video Retargeting for Medical Data Security

Liu

Kang

Hua

et al. 2022

Security and Communication Networks

View full text Add to dashboard Cite

Medical data security is an important guarantee for intelligent medical system. Medical video data can help doctors understand the patients’ condition. Medical video retargeting can greatly reduce the storage capacity of data on the premise of preserving the original content information as much as possible. The smaller volume of medical data can reduce the execution time of data encryption and threat detection algorithm and improve the performance of medical data security methods. The existing methods mainly focus on the temporal pixel relationship and foreground motion between adjacent frames, but these methods ignore the user’s attention to the video content and the impact of background movement on retargeting, resulting in serious deformation of important content and area. To solve the above problems, this paper proposes an innovative video retargeting method, which is based on visual attention and motion estimation. Firstly, the visual attention map is obtained from eye tracking data, by K-means clustering method and Euclidean distance factor equation. Secondly, the motion estimation map is generated from both the foreground and background displacements, which are calculated based on the feature points and salient object positions between adjacent frames. Then, the visual attention map, the motion estimation map, and gradient map are fused to the importance map. Finally, video retargeting is performed by mesh deformation based on the importance map. Experiment on open datasets shows that the proposed method can protect important area and has a better effect on salient object flutter suppression.

show abstract

Jitter-Robust Video Retargeting With Kalman Filter And Attention Saliency Fusion Network

Cited by 6 publications

References 18 publications

A Fast Smart-Cropping Method and Dataset for Video Retargeting

A Fast Smart-Cropping Method and Dataset for Video Retargeting

Data-driven personalisation of television content: a survey

Visual Attention and Motion Estimation-Based Video Retargeting for Medical Data Security

Contact Info

Product

Resources

About