Memory illusions and category malleability: False recognition for goal-derived reorganizations of common categories

Soro, Jerônimo C.; Ferreira, Mário B.; Carneiro, Paula; Moreira, Sérgio

doi:10.3758/s13421-020-01026-4

Cited by 2 publications

(2 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Taking into account false alarms means we now have two measures-hits and false alarmswhen we wish to have only one: how strong the memory was. Despite this, some researchers simply report hit and false alarm rates separately, not attempting to unify them into a single coherent measure of memory (e.g., Bainbridge, Isola & Oliva, 2013;Bjork & Bjork, 2003;Castella, Pina, Baques, & Allen, 2020;Chan & McDermott, 2007;De Brigard, Brady, Ruzic, & Schacter, 2017;Gardiner & Java, 1991;Jimenez, Mendez, Agra, & Ortiz-Tudela, 2020;Khader, Ranganath, Seemuller, & Rosler, 2007;Otero, Weekes, & Hutton, 2011;;;Smith & Hunt, 2020;Soro, Ferreira, Carneiro, & Moreira, 2020;;Yin, O'Neill, Brady, & De Brigard, 2019). In many cases, this effectively results in the inferences being made based on hit rates only, treating false alarms as a nuisance variable, or a totally distinct process, rather than using them to ask which people have or which conditions lead to the strongest memories.…”

Section: The Need For Counterfactual Reasoning In Measuring Memorymentioning

confidence: 99%

Measuring memory is harder than you think: How to avoid problematic measurement practices in memory research

Tf¹,

Mm²,

Williams³

et al. 2021

Preprint

View full text Add to dashboard Cite

There is a crisis of measurement in memory research, with major implications for theory and practice. This crisis arises because of a critical complication present when measuring memory using the recognition memory task that dominates the study of working memory and long-term memory (“did you see this item? yes/no” or “did this item change? yes/no”). Such tasks give two measures of performance, the “hit rate” (how often you say you previously saw an item you actually did previously see) and the “false alarm rate” (how often you say you saw something you never saw). Yet what researchers want is one single, integrated measure of memory performance. Integrating the hit and false alarm rate into a single measure, however, requires a complex problem of counterfactual reasoning that depends on the (unknowable) distribution of underlying memory signals: when faced with two people differing in both hit rate and false alarm rate, the question of who had the better memory is really “who would have had more hits if they each had the same number of false alarms”. As a result of this difficulty, different literatures in memory research (e.g., visual working memory, eyewitness identification, picture memory, etc) have settled on a variety of distinct metrics to combine hit rates and false alarm rates (e.g., A’, corrected hit rate, percent correct, d’, diagnosticity ratios, K values, etc.). These metrics make different, contradictory assumptions about the distribution of latent memory signals, and all of their assumptions are frequently incorrect. Despite a large literature on how to properly measure memory performance, spanning decades, real-life decisions are often made using these metrics, even when they subsequently turn out to be wrong when memory is studied with better measures. We suggest that in order for the psychology and neuroscience of memory to become a cumulative, theory-driven science, more attention must be given to measurement issues. We make a concrete suggestion: the default memory task should change from old/new (“did you see this item’?”) to forced-choice (“which of these two items did you see?”). In situations where old/new variants are preferred (e.g., eyewitness identification; theoretical investigations of the nature of memory decisions), receiver operating characteristic (ROC) analysis should always be performed.

show abstract

Section: The Need For Counterfactual Reasoning In Measuring Memorymentioning

confidence: 99%

Measuring memory is harder than you think: How to avoid problematic measurement practices in memory research

Tf¹,

Mm²,

Williams³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Taking into account false alarms means we now have two measures-hits and false alarmswhen we wish to have only one measure, which tells us "how strong" the memory was. One way to deal with this is to report hit and false alarm rates separately, not attempting to unify them into a single coherent measure of memory (e.g., Bainbridge, Isola & Oliva, 2013;Bjork & Bjork, 2003;Castella, Pina, Baques, & Allen, 2020;Chan & McDermott, 2007;De Brigard, Brady, Ruzic, & Schacter, 2017;Gardiner & Java, 1991;Jimenez, Mendez, Agra, & Ortiz-Tudela, 2020;Khader, Ranganath, Seemuller, & Rosler, 2007;Otero, Weekes, & Hutton, 2011;Smith & Hunt, 2020;Soro, Ferreira, Carneiro, & Moreira, 2020;Yin, O'Neill, Brady, & De Brigard, 2019). In many cases, this effectively results in inferences being made based on hit rates only, whereas false alarms are treated as a nuisance variable or as a measure of a completely distinct process, rather than a variable that can provide insight into which people have or which conditions lead to the strongest memories.…”

Section: The Need For Counterfactual Reasoning In Measuring Recogniti...mentioning

confidence: 99%

Measuring memory is harder than you think: How to avoid problematic measurement practices in memory research

et al. 2022

View full text Add to dashboard Cite

We argue that critical areas of memory research rely on problematic measurement practices and provide concrete suggestions to improve the situation. In particular, we highlight the prevalence of memory studies that use tasks (like the "old/new" task: "have you seen this item before? yes/no") where quantifying performance is deeply dependent on counterfactual reasoning that depends on the (unknowable) distribution of underlying memory signals. As a result of this difficulty, different literatures in memory research (e.g., visual working memory, eyewitness identification, picture memory, etc) have settled on a variety of fundamentally different metrics to get performance measures from such tasks (e.g., A′, corrected hit rate, percent correct, d′, diagnosticity ratios, K values, etc.), even though these metrics make different, contradictory assumptions about the distribution of latent memory signals, and even though all of their assumptions are frequently incorrect. We suggest that in order for the psychology and neuroscience of memory to become a more cumulative, theory-driven science, more attention must be given to measurement issues. We make a concrete suggestion: the default memory task for those simply interested in performance should change from old/new ("did you see this item'?") to two-alternative forced-choice ("which of these two items did you see?"). In situations where old/new variants are preferred (e.g., eyewitness identification; theoretical investigations of the nature of memory signals), receiver operating characteristic (ROC) analysis should be performed rather than a binary old/new task.

show abstract

Memory illusions and category malleability: False recognition for goal-derived reorganizations of common categories

Cited by 2 publications

References 39 publications

Measuring memory is harder than you think: How to avoid problematic measurement practices in memory research

Measuring memory is harder than you think: How to avoid problematic measurement practices in memory research

Measuring memory is harder than you think: How to avoid problematic measurement practices in memory research

Contact Info

Product

Resources

About