“…After overfitting all the training points, it is very likely the neural network maps representations of all training points with the same label type to an identical or very similar representations on the topmost layer (i.e., the layer right before the softmax layer), in which case we are not able to separate poisoned data points from normal ones only based on representations. 1 Secondly, though it is relatively easy for neural representations to capture the abnormality for simple and conspicuous triggers such as word insertion in NLP (Dai et al, 2019;Kurita et al, 2020b;Gan et al, 2021;Chen et al, 2021b) or pixel attack in vision (Gu et al, 2017;, it is not necessarily true or theoretically valid that subtle, hidden and complicated triggers (e.g., syntactic trigger to paraphrase a natural language Qi et al (2021a) or triggers being input dependent Nguyen & Tran (2020)) can be captured by intermediate representations, and if they are truly captured, where and how.…”