SemanticAdv: Generating Adversarial Examples via Attribute-Conditioned Image Editing

Qiu, Haonan; Xiao, Chaowei; Yang, Lei; Yan, Xinchen; Lee, Honglak; Li, Bo

doi:10.1007/978-3-030-58568-6_2

Cited by 101 publications

(67 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bhattad et al [2] leverage pre-trained colourisation and texturetransfer models to adversarially change the colours and textures of an image. A number of publications in some way exploit generative models with disentangled latent spaces, be it by using Fader Networks [24], using a dataset with labelled attributes to train a conditional generator [36], or using a StyleGAN and partitioning the latent space according to whether or not it should influence the label [15]. Selecting the features to perturb like this allows for precise control over these features, but like hand-crafted perturbations, result in narrow kinds of changes to images.…”

Section: Related Workmentioning

confidence: 99%

Exposing previously undetectable faults in deep neural networks

Dunn

Pouget

Kroening

et al. 2021

Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Existing methods for testing DNNs solve the oracle problem by constraining the raw features (e.g. image pixel values) to be within a small distance of a dataset example for which the desired DNN output is known. But this limits the kinds of faults these approaches are able to detect. In this paper, we introduce a novel DNN testing method that is able to find faults in DNNs that other methods cannot. The crux is that, by leveraging generative machine learning, we can generate fresh test inputs that vary in their high-level features (for images, these include object shape, location, texture, and colour). We demonstrate that our approach is capable of detecting deliberately injected faults as well as new faults in state-of-the-art DNNs, and that in both cases, existing methods are unable to find these faults. CCS CONCEPTS• Software and its engineering → Software testing and debugging; • Computing methodologies → Neural networks.

show abstract

Section: Related Workmentioning

confidence: 99%

Exposing previously undetectable faults in deep neural networks

Dunn

Pouget

Kroening

et al. 2021

Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

show abstract

“…The street view image can describe the fine-grained physical environment elements, including urban infrastructure, human-made cityscapes, and natural landscapes. Street view images will provide sufficient façade images to ensure reliable results for the GANs model training [30]. However, the street view database does not directly give the orthographic projection of building façade pictures.…”

Section: Street View Imagesmentioning

confidence: 99%

Automatic Object Removal With Obstructed Façades Completion Using Semantic Segmentation and Generative Adversarial Inpainting

2021

View full text Add to dashboard Cite

Automatic object removal with obstructed façades completion in the urban environment is essential for many applications such as scene restoration, environmental impact assessment, and urban mapping. However, the previous object removal typically requires a user to manually create a mask around unwanted objects and obtain background façade information in advance, which would be labor-intensive when implementing multitasking projects. Moreover, accurately detecting objects to be removed in the cityscape and inpainting the static obstructed building façade to obtain plausible images are the main challenges for this objective. To overcome these difficulties, this study addresses the object removal with the façade inpainting problem from the following two aspects. First, we proposed an image-based cityscape elimination method for automatic object removal and façade inpainting by applying semantic segmentation to detect several classes, including pedestrians, riders, vegetation, and cars, as well as using generative adversarial networks (GANs) for filling detected regions by background textures and patching information from street-level imagery. Second, we proposed a workflow to filter unoccluded building façades from street view images automatically and tailored a dataset for the GAN-based image inpainting model with original and mask images. Furthermore, several full-reference image quality assessment (IQA) metrics are introduced to evaluate the generated image quality. Validation results demonstrated the feasibility and effectiveness of our proposed method, and the synthetic image is visually realistic and semantically consistent.INDEX TERMS Generative adversarial networks, semantic segmentation, automatic object removal, façade inpainting, street view images.

show abstract

“…This already led to an incessant attacker-defender race in the fast moving field of security for machine learning and adversarial examples [53][54][55][56]. In recent years, researchers have among others developed different attack schemes on how to evade cybersecurity AI [57], e-mail protection, verification tools [58], forensic classifiers [59] and person detectors [60], how to elicit algorithmic biases [13,61], how to fool medical AI [62][63][64][65], law enforcement tools [66] as well as autonomous vehicles [67,68], how to perform denial-of-service and other adversarial attacks on commercial AI services [69][70][71], how to cause energy-intense and unnecessarily prolonged processing time [72] and how to poison AI systems postdeployment [73].…”

Section: Rda For Ai Risk Instantiations Ia and Ib-examplesmentioning

confidence: 99%

Transdisciplinary AI Observatory—Retrospective Analyses and Future-Oriented Contradistinctions

2021

View full text Add to dashboard Cite

In the last years, artificial intelligence (AI) safety gained international recognition in the light of heterogeneous safety-critical and ethical issues that risk overshadowing the broad beneficial impacts of AI. In this context, the implementation of AI observatory endeavors represents one key research direction. This paper motivates the need for an inherently transdisciplinary AI observatory approach integrating diverse retrospective and counterfactual views. We delineate aims and limitations while providing hands-on-advice utilizing concrete practical examples. Distinguishing between unintentionally and intentionally triggered AI risks with diverse socio-psycho-technological impacts, we exemplify a retrospective descriptive analysis followed by a retrospective counterfactual risk analysis. Building on these AI observatory tools, we present near-term transdisciplinary guidelines for AI safety. As further contribution, we discuss differentiated and tailored long-term directions through the lens of two disparate modern AI safety paradigms. For simplicity, we refer to these two different paradigms with the terms artificial stupidity (AS) and eternal creativity (EC) respectively. While both AS and EC acknowledge the need for a hybrid cognitive-affective approach to AI safety and overlap with regard to many short-term considerations, they differ fundamentally in the nature of multiple envisaged long-term solution patterns. By compiling relevant underlying contradistinctions, we aim to provide future-oriented incentives for constructive dialectics in practical and theoretical AI safety research.

show abstract

SemanticAdv: Generating Adversarial Examples via Attribute-Conditioned Image Editing

Cited by 101 publications

References 52 publications

Exposing previously undetectable faults in deep neural networks

Exposing previously undetectable faults in deep neural networks

Automatic Object Removal With Obstructed Façades Completion Using Semantic Segmentation and Generative Adversarial Inpainting

Transdisciplinary AI Observatory—Retrospective Analyses and Future-Oriented Contradistinctions

Contact Info

Product

Resources

About