Incorporating the Testlet Concept in Test Score Analyses<sup>1</sup>

Lee, Guemin; Brennan, Robert L.; Frisbie, David A.

doi:10.1111/j.1745-3992.2000.tb00041.x

Cited by 30 publications

(23 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Some items were retained because of the importance of their content and scale location; others were removed from this version. In the future, we may develop a series of testlet-type items [43, 44], in which the amounts of weight are reworked as response choices in one item. To remove items purely on a statistical basis without assessing the impact on the content validity and coverage of the scale is, in our view, undesirable.…”

Section: Discussionmentioning

confidence: 99%

Replenishing a computerized adaptive test of patient-reported daily activity functioning

Haley

Jette

et al. 2009

Qual Life Res

View full text Add to dashboard Cite

Purpose Computerized adaptive testing (CAT) item banks may need to be updated, but before new items can be added, they must be linked to the previous CAT. The purpose of this study was to evaluate 41 pretest items prior to including them into an operational CAT. Methods We recruited 6,882 patients with spine, lower extremity, upper extremity, and nonorthopedic impairments who received outpatient rehabilitation in one of 147 clinics across 13 states of the USA. Forty-one new Daily Activity (DA) items were administered along with the Activity Measure for Post-Acute Care Daily Activity CAT (DA-CAT-1) in five separate waves. We compared the scoring consistency with the full item bank, test information function (TIF), person standard errors (SEs), and content range of the DA-CAT-1 to the new CAT (DA-CAT-2) with the pretest items by real data simulations. Results We retained 29 of the 41 pretest items. Scores from the DA-CAT-2 were more consistent (ICC = 0.90 versus 0.96) than DA-CAT-1 when compared with the full item bank. TIF and person SEs were improved for persons with higher levels of DA functioning, and ceiling effects were reduced from 16.1% to 6.1%. Conclusions Item response theory and online calibration methods were valuable in improving the DA-CAT.

show abstract

Section: Discussionmentioning

confidence: 99%

Replenishing a computerized adaptive test of patient-reported daily activity functioning

Haley

Jette

et al. 2009

Qual Life Res

View full text Add to dashboard Cite

show abstract

“…To reduce the number of items, the concepts derived from the Phase I process were employed as an alternative to the individual instrument items. Scores from each item within a concept were added together to become an item for the factor analysis, i.e., if a concept had 4 items, all of the items were added together to form the “concept as an item.” This aggregated measurement unit is referred to as a “testlet” (Lee, Brennan, & Frisbie, 2000) or “parcel” (Hagtvet & Nasser, 2004); (Kishton & Widaman, 1994); (Little, Cunningham, Shahar, & Widaman, 2002) and (Hall, Snell, & Singer Foust, 1999). Parceling is considered an acceptable option when the parcels are founded on conceptual grounds.…”

Section: Methodsmentioning

confidence: 99%

Barriers to Cognitive Behavioral Therapy Homework Completion Scale–Depression Version: Development and Psychometric Evaluation

Callan¹,

Dunbar‐Jacob²,

Sereika³

et al. 2012

International Journal of Cognitive Therapy

View full text Add to dashboard Cite

We conducted a two-phase study to develop and evaluate the psychometric properties of an instrument to identify barriers to Cognitive Behavioral Therapy (CBT) homework completion in a depressed sample. In Phase I, we developed an item pool by interviewing 20 depressed patients and 20 CBT therapists. In Phase II, we created and administered a draft instrument to 56 people with depression. Exploratory Factor Analysis revealed a 2-factor oblique solution of “Patient Factors” and “Therapy/Task Factors.” Internal consistency coefficients ranged from .80 to .95. Temporal stability was demonstrated through Pearson correlations of .72 (for the therapist/task subscale) to .95 (for the patient subscale) over periods of time that ranged from 2 days to 3 weeks. The patient subscale was able to satisfactorily classify patients (75 to 79 %) with low and high adherence at both sessions. Specificity was .66 at both time points. Sensitivity was .80 at sessions B and .77 at session C. There were no consistent predictors of assignment compliance when measured by the Assignment Compliance Rating Scale (Primakoff, Epstein, & Covi, 1986). The Rating Scale and subscale scores did, however, correlate significantly with assignment non-compliance (.32 to .46).

show abstract

“…127-129) has shown how differences in the magnitudes of traditional coefficients of reliability are explainable in terms for fixed versus random facets. In addition, Brennan (2001) and Lee, Brenn a n , and Frisbie (2000) have explained how reliability can vary depending on whether testlets are viewed as having fixed or random characteristics.…”

Section: Fixed Versus Random Facetsmentioning

confidence: 99%

Some Problems, Pitfalls, and Paradoxes in Educational Measurement

Brennan

2001

Educational Measurement

Self Cite

View full text Add to dashboard Cite

This article provides brief discussions of an eclectic set of problems, pitfalls, and paradoxes that challenge measurement theory and practice, especially for K–12 achievement testing. The technical issues considered are: true scores and error scores, fixed versus random facets, lower limits to reliability, the reliability‐validity paradox, reliability of difference scores, equating and linking, and scaling. The practical issues considered are: validation, standard setting, instructional improvement and high‐stakes accountability, multiple measures, adequate yearly progress, the National Assessment of Educational Progress as a confirmation model, and capacity.

show abstract

Incorporating the Testlet Concept in Test Score Analyses¹

Abstract: How should we think about the concept of the testlet? How can testlets be better incorporated into test score analysis? Can there be a one‐item testlet?

Cited by 30 publications

References 34 publications

Replenishing a computerized adaptive test of patient-reported daily activity functioning

Replenishing a computerized adaptive test of patient-reported daily activity functioning

Barriers to Cognitive Behavioral Therapy Homework Completion Scale–Depression Version: Development and Psychometric Evaluation

Some Problems, Pitfalls, and Paradoxes in Educational Measurement

Contact Info

Product

Resources

About

Incorporating the Testlet Concept in Test Score Analyses1

Abstract: How should we think about the concept of the testlet? How can testlets be better incorporated into test score analysis? Can there be a one‐item testlet?

Cited by 30 publications

References 34 publications

Replenishing a computerized adaptive test of patient-reported daily activity functioning

Replenishing a computerized adaptive test of patient-reported daily activity functioning

Barriers to Cognitive Behavioral Therapy Homework Completion Scale–Depression Version: Development and Psychometric Evaluation

Some Problems, Pitfalls, and Paradoxes in Educational Measurement

Contact Info

Product

Resources

About

Incorporating the Testlet Concept in Test Score Analyses¹