Psychometric Properties of the Outcome Rating Scale: An Item Response Theory Analysis

Toland, Michael D.; Li, Caihong; Kodet, Jonathan; Reese, Robert J.

doi:10.1080/07481756.2020.1745647

Cited by 8 publications

(11 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Its clinical utility and usefulness as a tool to facilitate treatment were confirmed by Toland et al. (2020, p. 14), and a recent meta‐analysis reported its suitability for clients that might fit with a ‘fast change’ model in counselling (Østergård et al., 2020). Lambert et al.…”

Section: Methodsmentioning

confidence: 90%

See 1 more Smart Citation

Conducting practice‐based research with a counsellor in private practice

Manthei

Tuck

2021

Couns and Psychother Res

View full text Add to dashboard Cite

show abstract

Section: Methodsmentioning

confidence: 90%

“…It takes less than two minutes to complete, has a high completion rate, is simple to explain and administer and is not tied to any particular theoretical orientation (Anker et al., 2009). Since there is a high correlation among the four items, only the total ORS is used as a measure of well‐being (Miller et al., 2003; Toland et al., 2020).…”

Section: Methodsmentioning

confidence: 99%

Conducting practice‐based research with a counsellor in private practice

Manthei

Tuck

2021

Couns and Psychother Res

View full text Add to dashboard Cite

show abstract

“…The accuracy and precision with which respondents make a setting at the location where they intend to make it is higher if intermediate marks are provided along the length of the VAS line. It is noteworthy that use of VAS items in psychometric testing has not included intermediate marks (see, e.g., Simms, Zelazny, Williams, & Bernstein, 2019;Toland, Li, Kodet, & Reese, 2021), perhaps because use of such marks was explicitly and unfoundedly discouraged from the very beginning (Freyd, 1923) and subsequently. Thus, on describing the use of VAS in health research, McDowell (2006, p. 580) emphasized that "to produce a smooth response distribution, the VAS generally does not include numbers along the scale.…”

Section: Discussionmentioning

confidence: 99%

“…A researcher's ability to measure with exquisite precision the location of marks made on a line segment should never be misconstrued as a precise expression of the respondents' numerical translation of their subjective ratings. Yet, in practice, the location of marks made on a VAS line are always physically measured with precision and interpreted as reflecting the actual magnitude that subjects intended to indicate, although sometimes data are subsequently polychotomized for convenience (e.g., Flynn et al, 2004;Toland et al, 2021;Hyland, Shevlin, McBride, Murphy, Karatzias, Bentall, Martinez, & Vallières, 2020;van Laerhoven et al, 2004).…”

Section: Discussionmentioning

confidence: 99%

Accuracy and precision of responses to visual analog scales: Inter- and intra-individual variability

García‐Pérez

Alcalá-Quintana

2022

Behav Res

View full text Add to dashboard Cite

Visual analog scales (VASs) are gaining popularity for collecting responses in computer administration of psychometric tests and surveys. The VAS format consists of a line marked at its endpoints with the minimum and maximum positions that it covers for respondents to place a mark at their selected location. Creating the line with intermediate marks along its length was discouraged, but no empirical evidence has ever been produced to show that their absence does any good. We report a study that asked respondents to place marks at pre-selected locations on a 100-unit VAS line, first when it only had numerical labels (0 and 100) at its endpoints and then when intermediate locations (from 0 to 100 in steps of 20) were also labeled. The results show that settings are more accurate and more precise when the VAS line has intermediate tick marks: The average absolute error decreased from 3.02 units without intermediate marks to 0.82 units with them. Provision of intermediate tick marks also reduced substantially inter- and intra-individual variability in accuracy and precision: The standard deviation of absolute error decreased from 0.87 units without tick marks to 0.25 units with them and the standard deviation of signed distance to target decreased from 1.16 units without tick marks to 0.24 units with them. These results prompt the recommendation that the design of VASs includes intermediate tick marks along the length of the line.

show abstract

“…Then, even if the category labels for Likert response options did not universally and per se represent steps of equal size on the underlying judgments, item content seems to modulate the relation in ways that need to be assessed on an item-by-item basis. Toland et al (2021) took a different route and actually investigated the issue on an item-by-item basis. They administered a four-item test in VAS format and subsequently discretized the responses into K = 10, 5, or 4 categories.…”

Section: Comparison With Earlier Attempts To Investigate the Equal-in...mentioning

confidence: 99%

Are the Steps on Likert Scales Equidistant? Responses on Visual Analog Scales Allow Estimating Their Distances

García‐Pérez

2023

Educational and Psychological Measurement

View full text Add to dashboard Cite

A recurring question regarding Likert items is whether the discrete steps that this response format allows represent constant increments along the underlying continuum. This question appears unsolvable because Likert responses carry no direct information to this effect. Yet, any item administered in Likert format can identically be administered with a continuous response format such as a visual analog scale (VAS) in which respondents mark a position along a continuous line. Then, the operating characteristics of the item would manifest under both VAS and Likert formats, although perhaps differently as captured by the continuous response model (CRM) and the graded response model (GRM) in item response theory. This article shows that CRM and GRM item parameters hold a formal relation that is mediated by the form in which the continuous dimension is partitioned into intervals to render the discrete Likert responses. Then, CRM and GRM characterizations of the items in a test administered with VAS and Likert formats allow estimating the boundaries of the partition that renders Likert responses for each item and, thus, the distance between consecutive steps. The validity of this approach is first documented via simulation studies. Subsequently, the same approach is used on public data from three personality scales with 12, eight, and six items, respectively. The results indicate the expected correspondence between VAS and Likert responses and reveal unequal distances between successive pairs of Likert steps that also vary greatly across items. Implications for the scoring of Likert items are discussed.

show abstract

Psychometric Properties of the Outcome Rating Scale: An Item Response Theory Analysis

Cited by 8 publications

References 26 publications

Conducting practice‐based research with a counsellor in private practice

Conducting practice‐based research with a counsellor in private practice

Accuracy and precision of responses to visual analog scales: Inter- and intra-individual variability

Are the Steps on Likert Scales Equidistant? Responses on Visual Analog Scales Allow Estimating Their Distances

Contact Info

Product

Resources

About