Controllable Image Synthesis With Attribute-Decomposed GAN

Guo, Pengju; Men, Yifang; Mao, Y.; Jiang, Yuning; Ma, Wei‐Ying; Lian, Zhouhui

doi:10.1109/tpami.2022.3161985

Cited by 8 publications

(3 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…3D image synthesis can be used to generate realistic face images from 3D face scans or facial landmarks, which can be used for identity verification or access control. For example, Face ID on iPhone uses 3D image synthesis to project infrared dots on the user's face and match them with the stored 3D face model [207], [208].…”

Section: Applicationmentioning

confidence: 99%

Enhancing image resolution of confocal fluorescence microscopy with deep learning

Huang

Yao

et al. 2023

PhotoniX

View full text Add to dashboard Cite

Super-resolution optical imaging is crucial to the study of cellular processes. Current super-resolution fluorescence microscopy is restricted by the need of special fluorophores or sophisticated optical systems, or long acquisition and computational times. In this work, we present a deep-learning-based super-resolution technique of confocal microscopy. We devise a two-channel attention network (TCAN), which takes advantage of both spatial representations and frequency contents to learn a more precise mapping from low-resolution images to high-resolution ones. This scheme is robust against changes in the pixel size and the imaging setup, enabling the optimal model to generalize to different fluorescence microscopy modalities unseen in the training set. Our algorithm is validated on diverse biological structures and dual-color confocal images of actin-microtubules, improving the resolution from ~ 230 nm to ~ 110 nm. Last but not least, we demonstrate live-cell super-resolution imaging by revealing the detailed structures and dynamic instability of microtubules.

show abstract

Section: Applicationmentioning

confidence: 99%

Enhancing image resolution of confocal fluorescence microscopy with deep learning

Huang

Yao

et al. 2023

PhotoniX

View full text Add to dashboard Cite

show abstract

“…However, these methodologies primarily focus on attribute editing for individual objects, encountering challenges when there are multiple objects within a real-world scene due to complex spatial layouts among them. For example, face editing (Ju et al 2023;Pu et al 2022) involves editing attributes such as age, expression, and skin color. Pose transfer (Men et al 2020) allows editing of attributes like posture, clothing, and texture.…”

Section: Intoductionmentioning

confidence: 99%

Multi-Region Text-Driven Manipulation of Diffusion Imagery

Li,

Zhou,

Sun

et al. 2024

AAAI

View full text Add to dashboard Cite

Text-guided image manipulation has attracted significant attention recently. Prevailing techniques concentrate on image attribute editing for individual objects, however, encountering challenges when it comes to multi-object editing. The main reason is the lack of consistency constraints on the spatial layout. This work presents a multi-region guided image manipulation framework, enabling manipulation through region-level textual prompts. With MultiDiffusion as a baseline, we are dedicated to the automatic generation of a rational multi-object spatial distribution, where disparate regions are fused as a unified entity. To mitigate interference from regional fusion, we employ an off-the-shelf model (CLIP) to impose region-aware spatial guidance on multi-object manipulation. Moreover, when applied to the StableDiffusion, the presence of quality-related yet object-agnostic lengthy words hampers the manipulation. To ensure focus on meaningful object-specific words for efficient guidance and generation, we introduce a keyword selection method. Furthermore, we demonstrate a downstream application of our method for multi-region inversion, which is tailored for manipulating multiple objects in real images. Our approach, compatible with variants of Stable Diffusion models, is readily applicable for manipulating diverse objects in extensive images with high-quality generation, showing superb image control capabilities. Code is available at https://github.com/liyiming09/multi-region-guided-diffusion.

show abstract

“…However, with a background in materials science, the ML methods are so outdated that they cannot be satisfied with many application scenarios. For example, most ANN models are based on feedforward neural networks, while different types of neural networks such as time series networks [ 104 , 105 , 106 ], recurrent neural networks [ 107 , 108 , 109 ], and adversarial networks [ 110 , 111 , 112 ] have appeared in the field of computer science. Later, the use of various new neural networks mixed in materials science will become a future trend.…”

Section: Outlook and Conclusionmentioning

confidence: 99%

A Review of Performance Prediction Based on Machine Learning in Materials Science

Liu

Huang

et al. 2022

Nanomaterials

View full text Add to dashboard Cite

With increasing demand in many areas, materials are constantly evolving. However, they still have numerous practical constraints. The rational design and discovery of new materials can create a huge technological and social impact. However, such rational design and discovery require a holistic, multi-stage design process, including the design of the material composition, material structure, material properties as well as process design and engineering. Such a complex exploration using traditional scientific methods is not only blind but also a huge waste of time and resources. Machine learning (ML), which is used across data to find correlations in material properties and understand the chemical properties of materials, is being considered a new way to explore the materials field. This paper reviews some of the major recent advances and applications of ML in the field of properties prediction of materials and discusses the key challenges and opportunities in this cross-cutting area.

show abstract

Controllable Image Synthesis With Attribute-Decomposed GAN

Cited by 8 publications

References 50 publications

Enhancing image resolution of confocal fluorescence microscopy with deep learning

Enhancing image resolution of confocal fluorescence microscopy with deep learning

Multi-Region Text-Driven Manipulation of Diffusion Imagery

A Review of Performance Prediction Based on Machine Learning in Materials Science

Contact Info

Product

Resources

About