Sensor-realistic Synthetic Data Engine for Multi-frame High Dynamic Range Photography

Hu, Jinhan; Choe, Gyeongmin; Nadir, Zeeshan; Nabil, Osama; Lee, Seokjun; Sheikh, H.R.; Yoo, Yong Keun; Polley, Michael O.

doi:10.1109/cvprw50498.2020.00266

Cited by 14 publications

(19 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prabhakar dataset [1], Sen dataset [37], and Tursun dataset [40]), and one synthetic dataset (Samsung dataset [36]). Among them, Kalantari, Prabhakar, and Samsung datasets are used for quantitative and qualitative evaluation, while Sen and Tursun datasets are used for qualitative comparison only as they do not provide ground truth 2 .…”

Section: Datasets the Experiments Are Conducted On Five Public Datase...mentioning

confidence: 99%

“…Prabhakar dataset [1] has 116 testing samples and it is used only for evaluation. The Samsung dataset [36] is a synthetic one, containing 100 samples. The dataset is created in a similar way to the Kalantari dataset, except that all the data are synthesized through a game engine.…”

Section: Datasets the Experiments Are Conducted On Five Public Datase...mentioning

confidence: 99%

“…In the experiments, the training and evaluation are divided into three parts: 1) For real images, the model is trained on the Kalantari [2]'s training samples and evaluated on the Kalantari [2] and Prabhakar [1] testing samples; 2) For synthetic images, the training and evaluation are on the Samsung dataset [36]; 3) For the RAW images, training and evaluation are on the RAW dataset. For those training samples with ground truth, during training, the images are randomly cropped into 256×256 small patches and then data augmentation (e.g., flip and rotate) is applied for effective training.…”

Section: Datasets the Experiments Are Conducted On Five Public Datase...mentioning

confidence: 99%

“…Quantitative Results. Table I shows the quantitative comparison between the state-of-the-art models and ours on three datasets: Kalantari [2], Prabhakar [1], and Samsung [36]. Our model outperforms most of them, especially on the Kalantari and Samsung datasets, where it achieves the best results on five out of the six evaluation metrics.…”

Section: A Comparison With State-of-the-artsmentioning

confidence: 99%

See 3 more Smart Citations

Wavelet-Based Network For High Dynamic Range Imaging

Dai¹,

Li²,

Cao³

et al. 2021

Preprint

View full text Add to dashboard Cite

High dynamic range (HDR) imaging from multiple low dynamic range (LDR) images has been suffering from ghosting artifacts caused by scene and objects motion. Existing methods, such as optical flow based and end-to-end deep learning based solutions, are error-prone either in detail restoration or ghosting artifacts removal. Comprehensive empirical evidence shows that ghosting artifacts caused by large foreground motion are mainly low-frequency signals and the details are mainly highfrequency signals. In this work, we propose a novel frequencyguided end-to-end deep neural network (FHDRNet) to conduct HDR fusion in the frequency domain, and Discrete Wavelet Transform (DWT) is used to decompose inputs into different frequency bands. The low-frequency signals are used to avoid specific ghosting artifacts, while the high-frequency signals are used for preserving details. Using a U-Net as the backbone, we propose two novel modules: merging module and frequencyguided upsampling module. The merging module applies the attention mechanism to the low-frequency components to deal with the ghost caused by large foreground motion. The frequencyguided upsampling module reconstructs details from multiple frequency-specific components with rich details. In addition, a new RAW dataset is created for training and evaluating multiframe HDR imaging algorithms in the RAW domain. Extensive experiments are conducted on public datasets and our RAW dataset, showing that the proposed FHDRNet achieves state-ofthe-art performance.

show abstract

Section: Datasets the Experiments Are Conducted On Five Public Datase...mentioning

confidence: 99%

Section: Datasets the Experiments Are Conducted On Five Public Datase...mentioning

confidence: 99%

Section: Datasets the Experiments Are Conducted On Five Public Datase...mentioning

confidence: 99%

Section: A Comparison With State-of-the-artsmentioning

confidence: 99%

See 2 more Smart Citations

Wavelet-Based Network For High Dynamic Range Imaging

Dai¹,

Li²,

Cao³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…There can be large-scale datasets of interiors with any number of furniture sets, or separate datasets with objects of the environment. On the other hand, there can be various samples of residential or non-residential environments [10,11] simulated under different lighting conditions [12,13]. There are also large-scale urban datasets, including modelled natural areas and landscapes [14,15], and it is shown that such datasets have a good effect on convolutional neural network (CNN) training.…”

Section: Introductionmentioning

confidence: 99%

Producing Synthetic Dataset for Human Fall Detection in AR/VR Environments

et al. 2021

View full text Add to dashboard Cite

Human poses and the behaviour estimation for different activities in (virtual reality/augmented reality) VR/AR could have numerous beneficial applications. Human fall monitoring is especially important for elderly people and for non-typical activities with VR/AR applications. There are a lot of different approaches to improving the fidelity of fall monitoring systems through the use of novel sensors and deep learning architectures; however, there is still a lack of detail and diverse datasets for training deep learning fall detectors using monocular images. The issues with synthetic data generation based on digital human simulation were implemented and examined using the Unreal Engine. The proposed pipeline provides automatic “playback” of various scenarios for digital human behaviour simulation, and the result of a proposed modular pipeline for synthetic data generation of digital human interaction with the 3D environments is demonstrated in this paper. We used the generated synthetic data to train the Mask R-CNN-based segmentation of the falling person interaction area. It is shown that, by training the model with simulation data, it is possible to recognize a falling person with an accuracy of 97.6% and classify the type of person’s interaction impact. The proposed approach also allows for covering a variety of scenarios that can have a positive effect at a deep learning training stage in other human action estimation tasks in an VR/AR environment.

show abstract