Tunable U-Net: Controlling Image-to-Image Outputs Using a Tunable Scalar Value

Abstract

Image-to-image conversion tasks are more accurate and sophisticated than ever thanks to advances in deep learning. However, since typical deep learning models are trained to perform only one task, multiple trained models are required to perform each task even if they are related to each other. For example, the popular image-to-image convolutional neural network, U-Net, is normally trained for a single task. Based on U-Net, this study proposes a model that outputs variable results using only one trained model. The proposed method produces a continuously changing output by setting an external parameter. We confirm the robustness of our proposed model by evaluating it on binarization and background blurring. According to these evaluations, we confirmed that the proposed model can generate well-predicted outputs by using un-trained tuning parameters as well as the outputs by using trained tuning parameters. Furthermore, the proposed model can generate extrapolated outputs outside the learning range.INDEX TERMS Image-to-image conversion, multiple tasks, U-Net, image binarization, background blur

3d mask simulation and lithographic imaging using physics-informed neural networks

“…Subsequently, we explored the strategies to tune the network with incident and azimuthal angles. Following the successful experience of [32], we extended U-Net by the implementation of the tuning network (Figure 3) with trainable weights. Our tuning network brings the tunable scalar parameter to the main U-Net model controlling the second convolutional layer so that the physical loss may simultaneously be minimized for different illumination directions.…”

3D EUV mask simulator based on physics-informed neural networks: effects of polarization and illumination

“…However, the illumination parameters such as illumination angle or exposure wavelength are inherently scalar floating-point numbers, and their representation in an image-based format is not immediately straightforward. The authors [33] explored strategies to continuously change the network output by external scalar parameters. Building upon this successful approach, we extended the U-Net scheme by implementing a tuning network with trainable weights.…”