Man versus Machine: Comparing manual with LIWC coding of perceptual and contextual details for verbal lie detection

Schütte, Michael; Bogaard, Glynis; Giolla, Erik Mac; Warmelink, Lara; Kleinberg, Bennett; verschuere, Bruno

doi:10.31234/osf.io/cth58

Cited by 5 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, when researchers use LIWC to automatically code RM features and RM score, RM scores of verbal data are output in percentages. When LIWC and human coding were measured on the same metric (as count scores) in one study with four large verbal databases, neither LIWC nor human coding were superior to the other (Schutte et al, 2021). That finding is qualified in that LIWC scoring differed greatly between Dutch, English, and Swedish statements, and methods of manual coding was found to differ greatly between human coders in the four studies/datasets.…”

Section: Reality Monitoringmentioning

confidence: 97%

‘Sleepy Joe’ and ‘Donald, King of Whoppers’: Reality Monitoring and Verbal Deception in the 2020 U.S. Presidential Election Debates

et al. 2022

View full text Add to dashboard Cite

The 2020 U.S. Presidential election was a campaign that could be characterized as ‘one of the nastiest presidential campaigns in recent memory,’ partly because the general election debates were highly contentious and featured frequent interruptions and several insults and invectives between candidates. This research compared the language used in the debates to fact-checked truths and lies using a Reality Monitoring (RM) deception detection algorithm in Linguistic Inquiry and Word Count (LIWC) to investigate the veracity of real-life high-stakes verbal messages in the political context. We found that overall RM scores were lower and not significantly different between debate language and fact-checked lies, and RM scores were significantly higher in fact-checked truth statements, indicating that most debate language uttered was deceptive. This result supports the finding that the RM algorithm in LIWC distinguishes truth from lies and debate language in the context of politics. The 60.7% classification rate in this study may reflect a problem with the relatively short word counts of fact-checked lie and truth statements, but most probably reflects individual candidates’ deviations in RM features used in their statements. Each individual has a style that they use in communication—‘the way people talk and write have been recognized as stamps of individual identity.’ Even with a corpus of many statements from the same individual candidates, they probably regularly amplify certain features of RM and diminish other features of RM in their truthful and deceptive messages. This is a fruitful area of research that could be explored in future studies.

show abstract

Section: Reality Monitoringmentioning

confidence: 97%

‘Sleepy Joe’ and ‘Donald, King of Whoppers’: Reality Monitoring and Verbal Deception in the 2020 U.S. Presidential Election Debates

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Third, we used the Linguistic Inquiry and Word Count software (LIWC, Pennebaker et al, 2015) to count the number of perceptual and contextual details (Schutte et al, 2021).…”

Section: Statementsmentioning

confidence: 99%

Does (Biasing) Nonverbal Information Deteriorate the Accuracy of the Take-the-Best Heuristic for Deception detection?

Lob,

Chakir,

van Munster van Heuven

et al. 2023

Preprint

View full text Add to dashboard Cite

Does having access to nonverbal information facilitate or rather deteriorate deception detection accuracy? In study1, participants (N = 109) judged statements presented as video versus text on deception. In Study2, participants (N = 88) judging video statements with or without strongly biasing nonverbal behavior (i.e., truth tellers diverting their gaze, liars maintaining gaze). In both studies, we also manipulated the judgment method and replicated that focusing solely on the level of detail in the message outperformed intuitive judgments. The mere presence of nonverbal behavior did not deteriorate performance (Study1), but strongly biasing nonverbal behavior did (Study2). Our findings challenge the lay notion that access to nonverbal behavior could benefit deception detection, and shows it can even harm deception detection.

show abstract

Verbal lie detection using Large Language Models

Loconte,

Russo,

Capuozzo

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Human accuracy in detecting deception with intuitive judgments has been proven to not go above the chance level. Therefore, several automatized verbal lie detection techniques employing Machine Learning and Transformer models have been developed to reach higher levels of accuracy. This study is the first to explore the performance of a Large Language Model, FLAN-T5 (small and base sizes), in a lie-detection classification task in three English-language datasets encompassing personal opinions, autobiographical memories, and future intentions. After performing stylometric analysis to describe linguistic differences in the three datasets, we tested the small- and base-sized FLAN-T5 in three Scenarios using 10-fold cross-validation: one with train and test set coming from the same single dataset, one with train set coming from two datasets and the test set coming from the third remaining dataset, one with train and test set coming from all the three datasets. We reached state-of-the-art results in Scenarios 1 and 3, outperforming previous benchmarks. The results revealed also that model performance depended on model size, with larger models exhibiting higher performance. Furthermore, stylometric analysis was performed to carry out explainability analysis, finding that linguistic features associated with the Cognitive Load framework may influence the model’s predictions.

show abstract

Man versus Machine: Comparing manual with LIWC coding of perceptual and contextual details for verbal lie detection

Cited by 5 publications

References 21 publications

‘Sleepy Joe’ and ‘Donald, King of Whoppers’: Reality Monitoring and Verbal Deception in the 2020 U.S. Presidential Election Debates

‘Sleepy Joe’ and ‘Donald, King of Whoppers’: Reality Monitoring and Verbal Deception in the 2020 U.S. Presidential Election Debates

Does (Biasing) Nonverbal Information Deteriorate the Accuracy of the Take-the-Best Heuristic for Deception detection?

Verbal lie detection using Large Language Models

Contact Info

Product

Resources

About