“…Recent research [10,11,23,41] visually explains how higher level concepts can be constructed by neural connections. In the context of adversarial attack, some methods [10,11,31] identify where in a network the activation pathways of a benign and attacked input instance diverge, and how those diverging activations arrive at an incorrect prediction through connections among neurons. Inspired by these techniques, we summarize and visualize how neuron groups interact through connections among them, providing a new way to interpret concept cascading across layers.…”