Fully Convolutional Network with Multi-Step Reinforcement Learning for Image Processing

Furuta, Ryosuke; Inoue, Naoto; Yamasaki, Toshihiko

doi:10.1609/aaai.v33i01.33013598

Cited by 47 publications

(46 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep reinforcement learning (LR) has recently drawn significant attention in the field of deep learning. Deep LR has been applied to a variety of image processing and computer vision tasks: image super-resolution, image denoising, and image restoration [123][124][125]. A deep reinforcement learning framework can play a role in a GAN to generate high-quality output, whereby the generator can be utilized as an agent and the discriminator's results as the reward signal.…”

Section: Directions Of Future Researchmentioning

confidence: 99%

Deep Generative Adversarial Networks for Image-to-Image Translation: A Review

Alotaibi

2020

Symmetry

View full text Add to dashboard Cite

Many image processing, computer graphics, and computer vision problems can be treated as image-to-image translation tasks. Such translation entails learning to map one visual representation of a given input to another representation. Image-to-image translation with generative adversarial networks (GANs) has been intensively studied and applied to various tasks, such as multimodal image-to-image translation, super-resolution translation, object transfiguration-related translation, etc. However, image-to-image translation techniques suffer from some problems, such as mode collapse, instability, and a lack of diversity. This article provides a comprehensive overview of image-to-image translation based on GAN algorithms and its variants. It also discusses and analyzes current state-of-the-art image-to-image translation techniques that are based on multimodal and multidomain representations. Finally, open issues and future research directions utilizing reinforcement learning and three-dimensional (3D) modal translation are summarized and discussed.

show abstract

Section: Directions Of Future Researchmentioning

confidence: 99%

Deep Generative Adversarial Networks for Image-to-Image Translation: A Review

Alotaibi

2020

Symmetry

View full text Add to dashboard Cite

show abstract

“…Yu et al [32] proposed an image restoration method by selecting a toolchain from a toolbox. Furuta et al [5,6] proposed a fully convolutional network that allowed agents to perform pixel-wise manipulations for image denoising, image restoration, and color enhancement. Ganin et al [8] used an adversarially trained agent for synthesizing simple images of letters or digits using a non-differentiable renderer.…”

Section: Reinforcement Learning For Image Processingmentioning

confidence: 99%

Self-Play Reinforcement Learning for Fast Image Retargeting

Kajiura

Kosugi

Wang

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

In this study, we address image retargeting, which is a task that adjusts input images to arbitrary sizes. In one of the best-performing methods called MULTIOP, multiple retargeting operators were combined and retargeted images at each stage were generated to find the optimal sequence of operators that minimized the distance between original and retargeted images. The limitation of this method is in its tremendous processing time, which severely prohibits its practical use. Therefore, the purpose of this study is to find the optimal combination of operators within a reasonable processing time; we propose a method of predicting the optimal operator for each step using a reinforcement learning agent. The technical contributions of this study are as follows. Firstly, we propose a reward based on self-play, which will be insensitive to the large variance in the content-dependent distance measured in MULTIOP. Secondly, we propose to dynamically change the loss weight for each action to prevent the algorithm from falling into a local optimum and from choosing only the most frequently used operator in its training. Our experiments showed that we achieved multi-operator image retargeting with less processing time by three orders of magnitude and the same quality as the original multi-operator-based method, which was the best-performing algorithm in retargeting tasks. CCS CONCEPTS • Computing methodologies → Reinforcement learning; Image processing; Neural networks.

show abstract

“…[11] used a deep neural network to approximate the Q function and achieved great performance on many Atari games. Then, deep reinforcement learning has been applied into many different fields including computer vision [3,7,19].…”

Section: Reinforcement Learningmentioning

confidence: 99%

Weakly Supervised Video Summarization by Hierarchical Reinforcement Learning

Chen

Wang

et al. 2019

Proceedings of the ACM Multimedia Asia

Self Cite

View full text Add to dashboard Cite

Conventional video summarization approaches based on reinforcement learning have the problem that the reward can only be received after the whole summary is generated. Such kind of reward is sparse and it makes reinforcement learning hard to converge. Another problem is that labelling each frame is tedious and costly, which usually prohibits the construction of large-scale datasets. To solve these problems, we propose a weakly supervised hierarchical reinforcement learning framework, which decomposes the whole task into several subtasks to enhance the summarization quality. This framework consists of a manager network and a worker network. For each subtask, the manager is trained to set a subgoal only by a task-level binary label, which requires much fewer labels than conventional approaches. With the guide of the subgoal, the worker predicts the importance scores for video frames in the subtask by policy gradient according to both global reward and innovative defined sub-rewards to overcome the sparse problem. Experiments on two benchmark datasets show that our proposal has achieved the best performance, even better than supervised approaches. CCS CONCEPTS• Computing methodologies → Video summarization; • Theory of computation → Reinforcement learning.

show abstract

Fully Convolutional Network with Multi-Step Reinforcement Learning for Image Processing

Cited by 47 publications

References 28 publications

Deep Generative Adversarial Networks for Image-to-Image Translation: A Review

Deep Generative Adversarial Networks for Image-to-Image Translation: A Review

Self-Play Reinforcement Learning for Fast Image Retargeting

Weakly Supervised Video Summarization by Hierarchical Reinforcement Learning

Contact Info

Product

Resources

About