DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature

Mitchell, Eric; Lee, Yoonho; Khazatsky, Alexander; Manning, Christopher D.; Finn, Chelsea

doi:10.48550/arxiv.2301.11305

Cited by 52 publications

(100 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…These works evaluate the expected per-token log probability of texts and perform thresholding to detect AI-generated texts. Mitchell et al [2023] observe that AI-generated passages tend to lie in negative curvature of log probability of texts. They propose DetectGPT, a zero-shot LLM text detection method, to leverage this observation.…”

Section: Introductionmentioning

confidence: 90%

“…Several detection works study this problem as a binary classification problem [OpenAI, 2019, Jawahar et al, 2020, Mitchell et al, 2023, Bakhtin et al, 2019, Fagni et al, 2020. For example, OpenAI fine-tunes RoBERTa-based GPT-2 detector models to distinguish between non-AI generated and GPT-2 generated texts [OpenAI, 2019].…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, through both empirical and theoretical analysis, we show that state-of-the-art AI-text detectors are not reliable in practical scenarios. We first study empirical attacks on soft watermarking [Kirchenbauer et al, 2023], and a wide range of zero-shot [Mitchell et al, 2023] and neural networkbased detectors [OpenAI, 2019]. We show that a paraphrasing attack, where a lightweight neural network-based paraphraser is applied to the output text of the AI-generative model, can evade various types of detectors.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Can AI-Generated Text be Reliably Detected?

Sadasivan¹,

Kumar²,

Balasubramanian³

et al. 2023

Preprint

View full text Add to dashboard Cite

The rapid progress of Large Language Models (LLMs) has made them capable of performing astonishingly well on various tasks including document completion and question answering. The unregulated use of these models, however, can potentially lead to malicious consequences such as plagiarism, generating fake news, spamming, etc. Therefore, reliable detection of AI-generated text can be critical to ensure the responsible use of LLMs. Recent works attempt to tackle this problem either using certain model signatures present in the generated text outputs or by applying watermarking techniques that imprint specific patterns onto them. In this paper, both empirically and theoretically, we show that these detectors are not reliable in practical scenarios. Empirically, we show that paraphrasing attacks, where a light paraphraser is applied on top of the generative text model, can break a whole range of detectors, including the ones using the watermarking schemes as well as neural network-based detectors and zero-shot classifiers. We then provide a theoretical impossibility result indicating that for a sufficiently good language model, even the best-possible detector can only perform marginally better than a random classifier. Finally, we show that even LLMs protected by watermarking schemes can be vulnerable against spoofing attacks where adversarial humans can infer hidden watermarking signatures and add them to their generated text to be detected as text generated by the LLMs, potentially causing reputational damages to their developers. We believe these results can open an honest conversation in the community regarding the ethical and reliable use of AI-generated text.Preprint. Under review.

show abstract

Section: Introductionmentioning

confidence: 90%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Can AI-Generated Text be Reliably Detected?

Sadasivan¹,

Kumar²,

Balasubramanian³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…In particular, there is a lot of concern regarding the possibility that chatbots can be used to write scientific material 8 . Such concerns have resulted in the development of mitigation strategies, such as AI-generated text detectors 40 . Similar algorithms aimed at detecting AI-generated text are being developed and implemented by major publishers 38 .…”

Section: Further Developmentsmentioning

confidence: 99%

ChatGPT in glioma patient adjuvant therapy decision making: ready to assume the role of a doctor in the tumour board?

Haemmerli

Sveikata

Nouri

et al. 2023

Preprint

View full text Add to dashboard Cite

Introduction ChatGPT, a novel AI-based chatbot, sparked a lot of interest in the scientific community. Complex central CNS tumour cases require multidisciplinary expert recommendations that incorporate multimodal disease information. Thus, the potential of ChatGPT to integrate comprehensive treatment information may be of tremendous benefit for CNS tumour decision-making. We evaluated the ChatGPT recommendations for glioma management by a panel of CNS tumour experts. Methods We randomly selected 10 patients with primary CNS gliomas discussed at our institution's Tumour Board. Patients' clinical status, surgical, imaging, and immuno-pathology-related information was provided to ChatGPT and seven CNS tumour experts. The chatbot was asked to give the most likely diagnosis, the adjuvant treatment choice, and the regimen while considering the patient's functional status. The experts rated the AI-based recommendations from 0 (complete disagreement) to 10 (complete agreement). An intraclass correlation agreement (ICC) was used to measure the inter-rater agreement. Results Eight patients (80%) met the criteria for glioblastoma and two (20%) were low-grade gliomas. The experts rated the quality of ChatGPT recommendations as poor for diagnosis (median 3, IQR 1-7.8, ICC 0.9, 95% CI 0.7-1.0), good for treatment recommendation (7, IQR 6-8, ICC 0.8, 95% CI 0.4-0.9), good for therapy regimen (7, IQR 4-8, ICC 0.8, 95% CI 0.5-0.9), moderate for functional status consideration (6, IQR 1-7, ICC 0.7, 95% CI 0.3-0.9), and moderate for overall agreement with the recommendations (5, IQR 3-7, ICC 0.7, 95% CI 0.3-0.9). No differences were observed between the glioblastomas and low-grade glioma ratings. Conclusions ChatGPT performed poorly in classifying glioma types but was good for adjuvant treatment recommendations as evaluated by CNS Tumour Board experts. Even though the ChatGPT lacks the precision to replace expert opinion, it may become a promising tool to supplement experts, especially in low-resource settings.

show abstract

“…Bài báo này sẽ phân tích và tổng hợp nội dung từ nghiên cứu của nhóm các nhà khoa học trường Đại học Stanford: Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, Chelsea Finn với bài nghiên cứu mang tiêu đề: DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature [4]. Trong nghiên cứu này, DetectGPT được ra đời để nhận diện văn bản sinh ra bởi máy bằng cách vận dụng các đặc tính riêng biệt của các văn bản sinh ra bởi mô hình ngôn ngữ.…”

Section: Introduction To Detectgpt -A Model To Detect Machine-generat...unclassified

Giới thiệu về DetectGPT – Công cụ phát hiện văn bản do máy tính tạo ra

Hoàng¹

2023

JSTT

View full text Add to dashboard Cite

Với sự bùng nổ của những mô hình ngôn ngữ lớn - Large language models (LLMs), những tiến bộ gần đây trong việc tạo ngôn ngữ tự nhiên bằng trí tuệ nhân tạo đang tạo ra một làn sóng ý kiến trên các trang mạng xã hội và cộng đồng trí tuệ nhân tạo. Đặc biệt, với sự ra mắt gần đây của ChatGPT, giới trẻ, cụ thể là các sinh viên đã bắt đầu cập nhập và ứng dụng mô hình này trong lĩnh vực học tập. Với ChatGPT hoặc các mô hình ngôn ngữ lớn tương tự, sinh viên hoàn toàn có thể hoàn thành các bài tập tiểu luận với rất ít công sức và sự chú tâm. Điều này cũng rấy lên những dư ngại trong quá trình dạy và học của các trường đại học. Từ đó, một nhóm các nhà nghiên cứu tại trường đại học Stanford đã đề xuất một mô hình, có tên gọi DetectGPT, với mục đích phát hiện các văn bản được tạo ra bởi ChatGPT. Bài báo sẽ phân tích mô hình hệ thống DetectGPT dưới góc nhìn thuật toán và so sánh kết quả của DetectGPT với các mô hình đã tồn tại. Rồi từ đó nêu lên được đặc trưng nổi trội của mô hình DetectGPT.

show abstract

DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature

Cited by 52 publications

References 0 publications

Can AI-Generated Text be Reliably Detected?

Can AI-Generated Text be Reliably Detected?

ChatGPT in glioma patient adjuvant therapy decision making: ready to assume the role of a doctor in the tumour board?

Giới thiệu về DetectGPT – Công cụ phát hiện văn bản do máy tính tạo ra

Contact Info

Product

Resources

About