Rapid, precise, and reliable measurement of delay discounting using a Bayesian learning algorithm

Abstract

Machine learning has the potential to facilitate the development of computational methods that improve the measurement of cognitive and mental functioning. in three populations (college students, patients with a substance use disorder, and Amazon Mechanical turk workers), we evaluated one such method, Bayesian adaptive design optimization (ADo), in the area of delay discounting by comparing its test-retest reliability, precision, and efficiency with that of a conventional staircase method. In all three populations tested, the results showed that ADO led to 0.95 or higher test-retest reliability of the discounting rate within 10-20 trials (under 1-2 min of testing), captured approximately 10% more variance in test-retest reliability, was 3-5 times more precise, and was 3-8 times more efficient than the staircase method. The ADO methodology provides efficient and precise protocols for measuring individual differences in delay discounting. Delay discounting, one dimension of impulsivity 1 , assesses how individuals make trade-offs between small but immediately available rewards versus large but delayed rewards. Delay discounting is broadly linked to normative cognitive and behavioral processes, such as financial decision making 2 , social decision making 3 , and personality 4 , among others. Also, individual differences in delay discounting are associated with several cognitive capacities, including working memory 5 , intelligence 6 , and top-down regulation of impulse control mediated by the prefrontal cortex 7,8. Delay discounting is a strong candidate endophenotype for a wide range of maladaptive behaviors, including addictive disorders 9,10 and health risk behaviors for a review, see 11. Studies of test-retest reliability of delay discounting demonstrate reliability both for adolescents 12 and adults 13 , and genetics studies indicate that delay discounting may be a heritable trait 9. As such, delay discounting has received attention in the developing field of precision medicine in mental health as a potentially rapid and reliable (bio)marker of individual differences relevant for treatment outcomes 14-16. The construct validity of delay discounting has been demonstrated in numerous studies. For example, the delay discounting task is widely used to assess (altered) temporal impulsivity of various psychiatric disorders, including patients with substance use disorders e.g., 11 , schizophrenia 13,14 , and bipolar disorder 14. Therefore, improved assessment of delay discounting may be beneficial to many fields, including psychology, neuroscience, medicine 15 , and economics. To link decision making tasks to mental functioning is a formidable challenge that requires simultaneously achieving multiple measurement goals. We focus on three aspects of measurement: reliability, precision, and efficiency. Reliable measurement of latent neurocognitive constructs or biological processes, such as impulsivity, reward sensitivity, or learning rate, is difficult. Recent advancements in neuroscience and computational psychiatry 16...

Anxiety Modulates Preference for Immediate Rewards Among Trait-Impulsive Individuals: A Hierarchical Bayesian Analysis

“…We used a DDT (Ahn et al, 2020) that uses a version of Bayesian active-learning, adaptive-design optimization (ADO) to improve task efficiency and the precision of parameter estimation (see Myung, Cavagnaro, & Pitt, 2013). Trial by trial, ADO selects dollar–day pairs that are expected to improve parameter estimation the most.…”

“…Participant-level parameters (discounting rate, k , and choice sensitivity, c ) are updated between trials using Bayesian updating, and delays and monetary values are then selected using a grid search over potential dollar-day pairs such that participants’ choices minimize uncertainty in parameter estimates. This DDT version makes it possible to collect data 3 to 8 times more rapidly and 3 to 5 times more precisely than traditional staircase approaches (Ahn et al, 2020). Although each participant’s parameters were estimated as the participant progressed through the task, modeling was conducted on raw choice data to facilitate hierarchical modeling.…”

“…The student and SUD groups both underwent 42 trials per session, whereas the MTurk group underwent 20 trials per session. We used fewer trials for MTurk participants because analyses of data from the other groups, who were tested first, showed that additional trials rarely improved parameter estimation (test–retest reliability of delay-discounting estimates exceeded r = .95 after 20 or fewer ADO trials) and to minimize off-task behavior (Ahn et al, 2020).…”

A Model Guided Approach to Evoke Homogeneous Behavior During Temporal Reward and Loss Discounting

“…For instance, investigating the underlying neurobiological substrates by comparing differences between immediate and delayed choices may fall short of statistical power given highly unbalanced trial types and the high variability in discounting strength across individuals [e.g., (27)(28)(29)(30)(31)(32); for an overview see (21)]. At times, participants even have to be excluded from analyses due to not discounting at all [e.g., (31,(33)(34)(35)(36)(37)(38)(39)(40)(41)(42)(43)(44)(45)].…”

Unifying Evidence on Delay Discounting: Open Task, Analysis Tutorial, and Normative Data from an Italian Sample

“…Different procedures have been used over the years to capture delay discounting in human behavior. From “fill-in-the-blank” or “matching” questions [ 43 ] to binary choices over a fixed-set of options [ 18 ] and binary choices with an underlying staircase design optimization procedure [ 40 , 44 ]. All such tasks have proven to reliably detect to a certain degree the typical (or atypical, when dysregulated) decay in subjective value for monetary and other rewards [ 20 , 38 , 45 , 46 , 47 ].…”