Do People Trust ChatGPT to Write Accurate Statements?

Buchanan, Joy; Hickman, W. Braddock

doi:10.1257/rct.11546

Search citation statements

Order By: Relevance

Paper Sections

Select...

Introduction1

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2023

Publication Types

Select...

Article1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work supports the notion that certain creative tasks can be outsourced to LLMs, while some of the more routine work in research production still cannot be entirely entrusted to ChatGPT (Buchanan & Hickman, 2023; Cowen & Tabarrok, 2023). Our contribution does not only concern the usefulness of ChatGPT for the particular task of writing a literature review.…”

Section: Introductionsupporting

confidence: 74%

ChatGPT Hallucinates Non-existent Citations: Evidence from Economics

Buchanan,

Hill,

Shapoval

2023

The American Economist

View full text Add to dashboard Cite

In this study, we generate prompts derived from every topic within the Journal of Economic Literature to assess the abilities of both GPT-3.5 and GPT-4 versions of the ChatGPT large language model (LLM) to write about economic concepts. ChatGPT demonstrates considerable competency in offering general summaries but also cites non-existent references. More than 30% of the citations provided by the GPT-3.5 version do not exist and this rate is only slightly reduced for the GPT-4 version. Additionally, our findings suggest that the reliability of the model decreases as the prompts become more specific. We provide quantitative evidence for errors in ChatGPT output to demonstrate the importance of LLM verification.JEL Codes: B4; O33; I2

show abstract

Section: Introductionsupporting

confidence: 74%