Bokeh Effects Based on Stereo Vision

Li, Dongwei; Nicolescu, Radu; Klette, Reinhard

doi:10.1007/978-3-319-23192-1_17

Cited by 10 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many works leverage the depth information obtained from stereo or monocular cameras to render bokeh effect. In the work of [7], the authors obtain a depth map with stereo matching and then apply the bokeh effect. The work of [8] produces a high resolution disparity map and combines it with a rendering algorithm to render bokeh effect.…”

Section: Related Workmentioning

confidence: 99%

Bokeh Effect Rendering with Vision Transformers

Nagasubramaniam¹,

Younes²

2022

Preprint

View full text Add to dashboard Cite

Bokeh effect is growing to be an important feature in photography, essentially to choose an object of interest to be in focus with the rest of the background being blurred. While naturally rendering this effect requires a DSLR with large diameter of aperture, with the current advancements in Deep Learning, this effect can also be produced in mobile cameras. Most of the existing methods use Convolutional Neural Networks while some relying on the depth map to render this effect. In this paper, we propose an end-to-end Vision Transformer model for Bokeh rendering of images from monocular camera. This architecture uses vision transformers as backbone, thus learning from the entire image rather than just the parts from the filters in a CNN. This property of retaining global information coupled with initial training of the model for image restoration before training to render the blur effect for the background, allows our method to produce clearer images and outperform the current state-of-the-art models on the EBB! Data set. The code to our proposed method can be found at: https://github.com/Soester10/ Bokeh-Rendering-with-Vision-Transformers.

show abstract

Section: Related Workmentioning

confidence: 99%

Bokeh Effect Rendering with Vision Transformers

Nagasubramaniam¹,

Younes²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Many works in Bokeh Effect Rendering leveraged depth information from images captured by two cameras. Liu et al [14] presented a bokeh simulation method by using depth estimation map through stereo matching. They have also designed a convenient bokeh control interface that uses a little user interaction for identifying the salient object and control the bokeh effect by setting a specific kernel.…”

Section: Related Workmentioning

confidence: 99%

“…One approach to solve these limitations is to computationally simulate the bokeh effect on the mobile devices with small camera. There are some works where additional depth information obtained from a dual pixel camera [19] or stereo camera (one main camera and one calibrated subcamera) [15,3,14] is incorporated in the model to simulate a more realistic bokeh image. However, these systems suffer from a variety of disadvantages like (a) addition of a stereo camera or depth sensor increases the size and the cost of the device and power consumption during usage (b) structured light depth sensors can be affected by poor resolution or high sensitivity to interference with ambient light (c) it performs badly when the target of interest is substantially distant from the camera, it is difficult to estimate good depth map for further regions leading because of the small stereo baseline of stereo camera (d) these approaches can't be used to enhance pictures already taken with monocular cameras as post processing since depth information is often not saved along with the picture.…”

Section: Introductionmentioning

confidence: 99%

Stacked Deep Multi-Scale Hierarchical Network for Fast Bokeh Effect Rendering from a Single Image

Dutta¹,

Das²,

Shah³

et al. 2021

Preprint

View full text Add to dashboard Cite

The Bokeh Effect is one of the most desirable effects in photography for rendering artistic and aesthetic photos. Usually, it requires a DSLR camera with different aperture and shutter settings and certain photography skills to generate this effect. In smartphones, computational methods and additional sensors are used to overcome the physical lens and sensor limitations to achieve such effect. Most of the existing methods utilized additional sensor's data or pretrained network for fine depth estimation of the scene and sometimes use portrait segmentation pretrained network module to segment salient objects in the image. Because of these reasons, networks have many parameters, become runtime intensive and unable to run in mid-range devices. In this paper, we used an end-to-end Deep Multi-Scale Hierarchical Network (DMSHN) model for direct Bokeh effect rendering of images captured from the monocular camera. To further improve the perceptual quality of such effect, a stacked model consisting of two DMSHN modules is also proposed. Our model does not rely on any pretrained network module for Monocular Depth Estimation or Saliency Detection, thus significantly reducing the size of model and run time. Stacked DMSHN achieves state-of-the-art results on a large scale EBB! dataset with around 6x less runtime compared to the current state-of-the-art model in processing HD quality images.

show abstract