“The Questions Shape the Answers”: Assessing the Quality of Published Survey Instruments in Health Professions Education Research

Artino, Anthony R.; Phillips, Andrew; Utrankar, Amol; Ta, Andrew Q.; Durning, Steven J.

doi:10.1097/acm.0000000000002002

Cited by 42 publications

(18 citation statements)

References 24 publications

(1 reference statement)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Is it something new that happens to the participant? A questionnaire, or even a question, is an event ( Schwarz, 1999 ). When something new happens, the participant may wonder, “What’s this about?” Almost all verbal measures are events, even essays that will later be coded for something the participant doesn’t know about.…”

Section: Solutionsmentioning

confidence: 99%

“…Researchers have to make assumptions about the meaning of the behavior in using these measures. But the multiple meanings of verbal measures, and their sensitivity to small contextual factors have been known for decades: there is no reason to believe that a rating scale provides a pure and unambiguous measure of the mental states corresponding to the words on the scale, and there are dozens of well-documented reasons to believe that it does not ( Schwarz, 1999 ; Kagan, 2007 ). Filling out a scale is an event that interrupts the participant’s experience, and the response to the scale is not just a reflection of the participant’s experience before the scale appeared, but also a response to what the participant thinks about the scale itself, and this response may also change the experience so that even if the experimenter succeeded in creating a particular mental state, it may no longer exist after the measurement.…”

Section: Solutionsmentioning

confidence: 99%

See 1 more Smart Citation

Are Manipulation Checks Necessary?

2018

View full text Add to dashboard Cite

Researchers are concerned about whether manipulations have the intended effects. Many journals and reviewers view manipulation checks favorably, and they are widely reported in prestigious journals. However, the prototypical manipulation check is a verbal (rather than behavioral) measure that always appears at the same point in the procedure (rather than its order being varied to assess order effects). Embedding such manipulation checks within an experiment comes with problems. While we conceptualize manipulation checks as measures, they can also act as interventions which initiate new processes that would otherwise not occur. The default assumption that manipulation checks do not affect experimental conclusions is unwarranted. They may amplify, undo, or interact with the effects of a manipulation. Further, the use of manipulation checks in mediational analyses does not rule out confounding variables, as any unmeasured variables that correlate with the manipulation check may still drive the relationship. Alternatives such as non-verbal and behavioral measures as manipulation checks and pilot testing are less problematic. Reviewers should view manipulation checks more critically, and authors should explore alternative methods to ensure the effectiveness of manipulations.

show abstract

Section: Solutionsmentioning

confidence: 99%

Section: Solutionsmentioning

confidence: 99%

Are Manipulation Checks Necessary?

2018

View full text Add to dashboard Cite

show abstract

“…Quite a lot of evidence exists, and in my view it does not support the idea that emotions are necessarily conscious, nor that verbal reports are the best way to assess emotion. Considerable experimental evidence indicates that some features of human emotion and motivation cannot actually be accessed well via introspection or described in subjective reports ( Nisbett and Wilson, 1978 ; Wilson and Schooler, 1991 ; Schwarz, 1999 ; Dijksterhuis et al, 2006 ; Schooler and Mauss, 2010 ; Greenwald and Banaji, 2017 ; Torre and Lieberman, 2018 ).…”

Section: Subjective Feelings Versus Objective Features Of Motivation/mentioning

confidence: 99%

Evolving Concepts of Emotion and Motivation

2018

View full text Add to dashboard Cite

This review takes a historical perspective on concepts in the psychology of motivation and emotion, and surveys recent developments, debates and applications. Old debates over emotion have recently risen again. For example, are emotions necessarily subjective feelings? Do animals have emotions? I review evidence that emotions exist as core psychological processes, which have objectively detectable features, and which can occur either with subjective feelings or without them. Evidence is offered also that studies of emotion in animals can give new insights into human emotions. Beyond emotion, motivation concepts have changed over decades too, and debates still continue. Motivation was once thought in terms of aversive drives, and reward was thought of in terms of drive reduction. Motivation-as-drive concepts were largely replaced by motivation-as-incentive concepts, yet aversive drive concepts still occasionally surface in reward neuroscience today. Among incentive concepts, incentive salience is a core motivation process, mediated by brain mesocorticolimbic systems (dopamine-related systems) and sometimes called ‘wanting’ (in quotation marks), to distinguish it from cognitive forms of desire (wanting without quotation marks). Incentive salience as ‘wanting’ is separable also from pleasure ‘liking’ for the same reward, which has important implications for several human clinical disorders. Ordinarily, incentive salience adds motivational urgency to cognitive desires, but ‘wanting’ and cognitive desires can dissociate in some conditions. Excessive incentive salience can cause addictions, in which excessive ‘wanting’ can diverge from cognitive desires. Conversely, lack of incentive salience may cause motivational forms of anhedonia in depression or schizophrenia, whereas a negatively-valenced form of ‘fearful salience’ may contribute to paranoia. Finally, negative ‘fear’ and ‘disgust’ have both partial overlap but also important neural differences.

show abstract

“…Severity and chronicity of illness as well as cognitive functioning are associated with worse functional outcomes, and have been specifically reported to predict reduced social and occupational engagement (Gitlin & Miklowitz, 2017). However, many of the currently used symptom inventories administered in research and clinical settings are subject to recall and state-dependent biases (Artino, Phillips, Utrankar, Ta, & Durning, 2018). Further, chaotic lifestyles associated with polarized mood can make it unreasonable to expect some BD patients to return to clinical settings regularly.…”

Section: Unique Methodology To Investigate Outcomes In Bdmentioning

confidence: 99%

298. Peripheral inflammation, Physical Activity and Cognition in Bipolar Disorder

Dev

Sutherland

Wing

et al. 2017

Biological Psychiatry

View full text Add to dashboard Cite

for their integral roles in supporting this project. I would also like to thank the countless number of individuals in my life that have provided their unconditional support. To my parents and brother-thank you for teaching me to soar while always giving me a safe place to land. Thank you to my "framily" for their daily words of encouragement, laughter, and support. Finally, thank you to my incredible JDP cohort and numerous colleagues, who have all taught me to be a better researcher, clinician, and person. My graduate training and this dissertation were made possible by the following funding:

show abstract

“The Questions Shape the Answers”: Assessing the Quality of Published Survey Instruments in Health Professions Education Research

Cited by 42 publications

References 24 publications

Are Manipulation Checks Necessary?

Are Manipulation Checks Necessary?

Evolving Concepts of Emotion and Motivation

298. Peripheral inflammation, Physical Activity and Cognition in Bipolar Disorder

Contact Info

Product

Resources

About