WESPE: Weakly Supervised Photo Enhancer for Digital Cameras

Abstract

Low-end and compact mobile cameras demonstrate limited photo quality mainly due to space, hardware and budget constraints. In this work, we propose a deep learning solution that translates photos taken by cameras with limited capabilities into DSLR-quality photos automatically. We tackle this problem by introducing a weakly supervised photo enhancer (WESPE) -a novel image-to-image Generative Adversarial Network-based architecture. The proposed model is trained by under weak supervision: unlike previous works, there is no need for strong supervision in the form of a large annotated dataset of aligned original/enhanced photo pairs. The sole requirement is two distinct datasets: one from the source camera, and one composed of arbitrary high-quality images that can be generally crawled from the Internet -the visual content they exhibit may be unrelated. Hence, our solution is repeatable for any camera: collecting the data and training can be achieved in a couple of hours. In this work, we emphasize on extensive evaluation of obtained results. Besides standard objective metrics and subjective user study, we train a virtual rater in the form of a separate CNN that mimics human raters on Flickr data and use this network to get reference scores for both original and enhanced photos. Our experiments on the DPED, KITTI and Cityscapes datasets as well as pictures from several generations of smartphones demonstrate that WESPE produces comparable or improved qualitative results with state-of-the-art strongly supervised methods.

Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement

“…For the albedo, we use a fully convolutional network without downsampling or upsampling blocks. This results in a small receptive field for the network and better preserves the texture details while avoiding large structural changes [16,17]. As shown in Figure 6, allowing downsampling blocks in the Figure 4: To finish the backward cycle, the real image is first translated to the PBR domain.…”

AI Benchmark: All About Deep Learning on Smartphones in 2019

“…Nowadays, various deep learning models can be found in nearly any mobile device. Among the most popular tasks are different computer vision problems like image classification [38,82,23], image enhancement [27,28,32,30], image super-resolution [17,42,83], bokeh simulation [85], object tracking [87,25], optical character recognition [56], face detection and recognition [44,70], augmented reality [3,16], etc. Another important group of tasks running on mobile devices is related to various NLP (Natural Language Processing) problems, such as natural language translation [80,7], sentence completion [52,24], sentence sentiment analysis [77,72,33], voice assistants [18] and interactive chatbots [71].…”

SMIT: Stochastic Multi-Label Image-to-Image Translation

“…The ability of humans to easily imagine how a black haired person would look like if they were blond, or with a different type of eyeglasses, or to imagine a winter scene as summer is formulated as the image-to-image (I2I) translation problem in the computer vision community. Since the recent introduction of Generative Adversarial Networks (GANs) [19], a plethora of problems such as video analysis [51,7], super resolution [33,9], semantic synthesis [26,10], photo enhancement [24,25], photo editing [49,14], and most recently domain adaptation [21,43] have been addressed as I2I translation problems.…”

“…However, this approach is unpractical because the full representation of the cross-domain mapping is, in most cases, intractable. Existing techniques try to perform deterministic I2I translation with unpaired images to map from one domain into another (one-to-one) [55,4,37,25], or into multiple domains (one-to-many) [12,46,20]. Nevertheless, many problems are fundamentally stochastic as there are countless mappings from one domain to another e.g., a day↔night or cat↔dog translation.…”