Authorship Identity and Spatiality: Social Influences on Text Production

Alvero, AJ; Luqueño, Leslie; Pearman, Francis; Antonio, Anthony Lising

doi:10.31235/osf.io/pt6b2

Cited by 2 publications

(2 citation statements)

References 22 publications

(36 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also tried some minimal variations around this prompting strategy, with similar results (see Appendix for details). The fact that other papers find similar patterns, either a low variance in the responses of LLMs (Alvero et al, 2024; (Abdulhai et al, 2023), only reinforces our confidence in this finding.…”

Section: Limitationssupporting

confidence: 77%

“…Other articles also notice patterns in the models that cannot be linked to any subsample of the population, although mostly incidentally. Bisbee et al (2023) remarked that the responses provided by ChatGPT had a much smaller variance than those of surveys taken by humans, which means that the model had a more narrow range of opinions than the subjects it was trying to imitate (see also Alvero et al, 2024). In this paper, we propose formal tests of these three theses.…”

Section: Can Llms Imitate (At Least Some) Humans?mentioning

confidence: 91%

See 1 more Smart Citation

Machine Bias. Generative Large Language Models Have a View of Their Own

Boelaert,

Coavoux,

Ollion

et al. 2024

Preprint

View full text Add to dashboard Cite

Generative AI is increasingly presented as a potential substitute for humans in many areas. This is also true in research. Large language models (LLMs) are often said to be able to replace human subjects, be they agents in simulated models, economic Mactors, subjects for experimental psychology, survey respondents, or potential consumers. Yet, there is no scientific consensus on how closely these in-silico clones represent their human counterparts. Another body of research points out the models inaccuracies, which they link to a bias. According to this skeptic view, LLMs are said to take the views of certain social groups, or of those which predominate in certain countries. Through a targeted experiment on survey questionnaires, we demonstrate that these critics are right to be wary of generative AI, but probably not for the right reasons. Our results i) confirm that to date, the models cannot replace humans for opinion or attitudinal research; and ii) that they also display a strong bias. Yet we also show that this bias is iii) both specific (it has a very low variance) and not easily related to any given social group. We devise a simple strategy to test for ‘social biases’ in models, from which we conclude that the social bias perspective is not adequate. We call machine bias the large, dominant part of the error in prediction, attributing it to the nature of current large language models rather than to their training data only

show abstract

Section: Limitationssupporting

confidence: 77%

Section: Can Llms Imitate (At Least Some) Humans?mentioning

confidence: 91%