A Manager and an AI Walk into a Bar: Does ChatGPT Make Biased Decisions Like We Do?

Chen, Yang; Andiappan, Meena; Jenkin, Tracy A.; Овчінніков, Антон

doi:10.2139/ssrn.4380365

Cited by 11 publications

(11 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the one hand, LLMs replicated results from tasks related to personality traits (Caron & Srivastava, 2022), framing effects (Chen et al, 2023), as well as political attitudes and party preferences (Argyle et al, 2023). For example, Caron and Srivastava (2022) surveyed Reddit users about their “Big Five” personalities and trained LLMs with these user‐specific contextual data.…”

Section: Using Llms To Mimic Human Behaviormentioning

confidence: 62%

“…Although efforts to substitute human respondents with LLMs are relatively new, several studies have already conducted comparisons of human and silicon samples. These comparisons stem from various domains (e.g., human-computer-interaction, general psychology, social psychology) and consider a wide range of tasks and settings (e.g., cognitive reflection task, Hagendorff et al, 2023; On the one hand, LLMs replicated results from tasks related to personality traits (Caron & Srivastava, 2022), framing effects (Chen et al, 2023), as well as political attitudes and party preferences (Argyle et al, 2023). For example, Caron and Srivastava (2022) surveyed Reddit users about their "Big Five" personalities and trained LLMs with these user-specific contextual data.…”

Section: Using Llms To Mimic Human Behaviormentioning

confidence: 99%

“…On the other hand, researchers were unable to replicate effects known to characterize consumer behavior in many cases, such as the endowment effect, mental accounting, or the sunk cost fallacy (Chen et al, 2023). For example, when replicating Kahneman and Tversky's (1979) classic prospect theory experiment to identify risk preferences regarding gains versus losses, Chen et al (2023) found that ChatGPT mostly focuses on maximizing the expected payoffs, rather than, as humans do, acting risk‐averse for gains and risk‐seeking for losses. Similarly, Park et al (2023), using GPT‐3.5, re‐ran 14 studies from Many Labs 2, a large‐scale replication project of major findings from psychological research (Klein et al, 2018).…”

Section: Using Llms To Mimic Human Behaviormentioning

confidence: 99%

See 2 more Smart Citations

Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelines

Sarstedt,

Adler,

Rau

et al. 2024

Psychology and Marketing

View full text Add to dashboard Cite

Should consumer researchers employ silicon samples and artificially generated data based on large language models, such as GPT, to mimic human respondents' behavior? In this paper, we review recent research that has compared result patterns from silicon and human samples, finding that results vary considerably across different domains. Based on these results, we present specific recommendations for silicon sample use in consumer and marketing research. We argue that silicon samples hold particular promise in upstream parts of the research process such as qualitative pretesting and pilot studies, where researchers collect external information to safeguard follow‐up design choices. We also provide a critical assessment and recommendations for using silicon samples in main studies. Finally, we discuss ethical issues of silicon sample use and present future research avenues.

show abstract

Section: Using Llms To Mimic Human Behaviormentioning

confidence: 62%

Section: Using Llms To Mimic Human Behaviormentioning

confidence: 99%

Section: Using Llms To Mimic Human Behaviormentioning

confidence: 99%

See 1 more Smart Citation

Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelines

Sarstedt,

Adler,

Rau

et al. 2024

Psychology and Marketing

View full text Add to dashboard Cite

show abstract

“…In addition, a study (Chen et al, 2023) specifically investigated behavioral biases relevant to operations management. ChatGPT exhibits human-like biases in complex, ambiguous, and implicit problems, such as conjunction bias, probability weighting, framing effects, salience of anticipated regret, and reference dependence.…”

Section: Current Research On the Integration Of Chatgpt In Toc Tpmentioning

confidence: 99%

Leveraging ChatGPT for Enhanced Logical Analysis in the Theory of Constraints Thinking Process

Aljaž

2024

Organizacija

View full text Add to dashboard Cite

Background/Purpose Artificial intelligence (AI) has traditionally been used for quantitative analysis using explicit data. However, much of the information critical to decision making remains undocumented and is not stored in a structured way. This study explores the integration of AI, specifically ChatGPT, into Theory of Constraints (TOC) Thinking Process (TP) tools. Method In this study, we applied ChatGPT to a real-world IT project management case using a variety of research methods, including international literature analysis, observation, and personal experience. The use of the TOC TP allowed us to understand the decision-making process of ChatGPT and to systematically explore its advantages and limitations in creating logical trees of TOC TP. Results ChatGPT significantly enhanced efficiency and depth in TOC TP data collection and analysis, effectively addressing logical leaps for more coherent structures. It also promoted deeper analytical thinking and aided root cause identification. The integration of ChatGPT into the TOC TP process led to faster decision-making, reduced bias, and clearer analysis. Challenges of ChatGPT including the need for human oversight, specific TOC TP training, and ethical considerations were noted. Conclusion This study provides an initial investigation into the use of ChatGPT in TOC TP tools. The results suggest that ChatGPT has the potential to be a valuable tool for organizations seeking to improve their decision making and performance. However, further research is needed to validate these findings and explore the full potential of AI in TOC TP.

show abstract

“…Universal self-consistency for LLM generation (Chen Y. et al, 2023). This approach utilizes multiple reasoning paths sampled from LLMs and then select the most consistent answer among multiple candidates.…”

Section: Model Examples In the Corresponding Domain Observationsmentioning

confidence: 99%

Dual-process theories of thought as potential architectures for developing neuro-symbolic AI models

Gronchi,

Perini

2024

Front. Cognit.

View full text Add to dashboard Cite

show abstract

A Manager and an AI Walk into a Bar: Does ChatGPT Make Biased Decisions Like We Do?

Cited by 11 publications

References 41 publications

Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelines

Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelines

Leveraging ChatGPT for Enhanced Logical Analysis in the Theory of Constraints Thinking Process

Dual-process theories of thought as potential architectures for developing neuro-symbolic AI models

Contact Info

Product

Resources

About