Adversarial attack and defense technologies in natural language processing: A survey

Qiu, Shilin; Liu, Qihe; Zhou, Shijie; Huang, Wen Nan

doi:10.1016/j.neucom.2022.04.020

Cited by 40 publications

(8 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Research on adversarial examples, whereby input data is intentionally perturbed to understand the system's resilience to human error or data tampering, has shown that even small modifications to the original text, such as swapping, inserting or deleting single character, can determine whether a sentence is interpreted as a positive or negative statement. Similar unreliability can be obtained by replacing a word with a synonym or by inserting, replacing or deleting a whole sentence (Qiu et al, 2022).…”

Section: Unreliabilitymentioning

confidence: 85%

The future of standardised assessment: Validity and trust in algorithms for assessment and scoring

Aloisi¹

2023

Euro J of Education

View full text Add to dashboard Cite

This article considers the challenges of using artificial intelligence (AI) and machine learning (ML) to assist high‐stakes standardised assessment. It focuses on the detrimental effect that even state‐of‐the‐art AI and ML systems could have on the validity of national exams of secondary education, and how lower validity would negatively affect trust in the system. To reach this conclusion, three unresolved issues in AI (unreliability, low explainability and bias) are addressed, to show how each of them would compromise the interpretations and uses of exam results (i.e., exam validity). Furthermore, the article relates validity to trust, and specifically to the ABI+ model of trust. Evidence gathered as part of exam validation supports each of the four trust‐enabling components of the ABI+ model (ability, benevolence, integrity and predictability). It is argued, therefore, that the three AI barriers to exam validity limit the extent to which an AI‐assisted exam system could be trusted. The article suggests that addressing the issues of AI unreliability, low explainability and bias should be sufficient to put AI‐assisted exams on par with traditional ones, but might not go as far as fully reassure the public. To achieve this, it is argued that changes to the quality assurance mechanisms of the exam system will be required. This may involve, for example, integrating principled AI frameworks in assessment policy and regulation.

show abstract

Section: Unreliabilitymentioning

confidence: 85%

The future of standardised assessment: Validity and trust in algorithms for assessment and scoring

Aloisi¹

2023

Euro J of Education

View full text Add to dashboard Cite

show abstract

“…Another approach is to detect adversarial noise that might cause wrong predictions [92], [93]. While these methods focus on images, there are other approaches that focus on generating and detecting adversarial attacks for natural language processing [94] or cybersecurity [95], which can be used to test and improve the robustness of these models.…”

Section: Testing Correctness Publicationsmentioning

confidence: 99%

A Survey on Verification and Validation, Testing and Evaluations of Neurosymbolic Artificial Intelligence

Renkhoff,

Feng,

Meier-Doernberg

et al. 2024

IEEE Trans. Artif. Intell.

View full text Add to dashboard Cite

Neurosymbolic artificial intelligence (AI) is an emerging branch of AI that combines the strengths of symbolic AI and sub-symbolic AI. Symbolic AI is based on the idea that intelligence can be represented using semantically meaningful symbolic rules and representations, while deep learning (DL), or sometimes called sub-symbolic AI, is based on the idea that intelligence emerges from the collective behavior of artificial neurons that are connected to each other. A major drawback of DL is that it acts as a "black box", meaning that predictions are difficult to explain, making the testing & evaluation (T&E) and validation & verification (V&V) processes of a system that uses sub-symbolic AI a challenge. Since neurosymbolic AI combines the advantages of both symbolic and sub-symbolic AI, this survey explores how neurosymbolic applications can ease the V&V process. This survey considers two taxonomies of neurosymbolic AI, evaluates them, and analyzes which algorithms are commonly used as the symbolic and sub-symbolic components in current applications. Additionally, an overview of current techniques for the T&E and V&V processes of these components is provided. Furthermore, it is investigated how the symbolic part is used for T&E and V&V purposes in current neurosymbolic applications. Our research shows that neurosymbolic AI has great potential to ease the T&E and V&V processes of sub-symbolic AI by leveraging the possibilities of symbolic AI. Additionally, the applicability of current T&E and V&V methods to neurosymbolic AI is assessed, and how different neurosymbolic architectures can impact these methods is explored. It is found that current T&E and V&V techniques are partly sufficient to test, evaluate, verify, or validate the symbolic and sub-symbolic part of neurosymbolic applications independently, while some of them use approaches where current T&E and V&V methods are not applicable by default, and adjustments or even new approaches are needed. Our research shows that there is great potential in using symbolic AI to test, evaluate, verify, or validate the predictions of a sub-symbolic model, making neurosymbolic AI an interesting research direction for safe, secure, and trustworthy AI.

show abstract

“…For example, Sun et al [144] However, to increase the reliability of these models in real-world applications, especially in critical domains like medicine, it is essential to systematically study the robustness of these models in various scenarios. Adversarial robustness refers to the model's ability to maintain good performance even in the case of deliberately crafted instances [464], [465]. These instances are called adversarial instances and are carefully designed by making subtle changes in the original inputs to deceive the model.…”

Section: Robustness Of Gllmsmentioning

confidence: 99%

A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4

Kalyan

2023

SSRN Journal

View full text Add to dashboard Cite

Adversarial attack and defense technologies in natural language processing: A survey

Cited by 40 publications

References 76 publications

The future of standardised assessment: Validity and trust in algorithms for assessment and scoring

The future of standardised assessment: Validity and trust in algorithms for assessment and scoring

A Survey on Verification and Validation, Testing and Evaluations of Neurosymbolic Artificial Intelligence

A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4

Contact Info

Product

Resources

About