Testlet-Based Multidimensional Adaptive Testing

Frey, Andreas; Seitz, Nicki-Nils; Brandt, Steffen

doi:10.3389/fpsyg.2016.01758

Cited by 10 publications

(8 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When a test is constructed in a predictive manner based on an information measure or a posterior predictive probability, a remedial step is necessary to cope with the unknown testlet interaction and minimize bias in testlet selection. In fact, because of this challenge from the unknown testlet interactions, many earlier studies that investigated adaptive testing strategies under the RTM ignored the existence of the testlet effect and selected the testlets considering only the primary ability dimension (e.g., Boyd, Dodd, & Fitzpatrick, 2013;Davis & Dodd, 2003;Frey, Seitz, & Brandt, 2016;Murphy, Dodd, & Vaughn, 2010). Note that such treatment leads to overestimation of testlet information and introduces bias in the testlet selection.…”

Section: Random-effect Testlet Modelmentioning

confidence: 99%

Computerized adaptive testing for testlet‐based innovative items

Kang

Han

Betts³

et al. 2021

Brit J Math & Statis

View full text Add to dashboard Cite

Increasing use of innovative items in operational assessments has shedded new light on the polytomous testlet models. In this study, we examine performance of several scoring models when polytomous items exhibit random testlet effects. Four models are considered for investigation: the partial credit model (PCM), testlet-as-a-polytomousitem model (TPIM), random-effect testlet model (RTM), and fixed-effect testlet model (FTM). The performance of the models was evaluated in two adaptive testings where testlets have nonzero random effects. The outcomes of the study suggest that, despite the manifest random testlet effects, PCM, FTM, and RTM perform comparably in trait recovery and examinee classification. The overall accuracy of PCM and FTM in trait inference was comparable to that of RTM. TPIM consistently underestimated population variance and led to significant overestimation of measurement precision, showing limited utility for operational use. The results of the study provide practical implications for using the polytomous testlet scoring models.

show abstract

Section: Random-effect Testlet Modelmentioning

confidence: 99%

Computerized adaptive testing for testlet‐based innovative items

Kang

Han

Betts³

et al. 2021

Brit J Math & Statis

View full text Add to dashboard Cite

show abstract

“…Each set consists of one stimulus, which is used in three different items below it. The stimulus in question can be in the form of reading passages, diagrams, pictures, or other appropriate contexts (Frey, Seitz & Brandt, 2016). The computerized testlet is a testlet-type instrument presented in computerized form.…”

Section: Instruments and Proceduresmentioning

confidence: 99%

Shadow Education in Indonesia: Is It Relevant to Students’ Critical Thinking Skills in Chemistry Learning?

Kawedhar

Mulyani

Saputro

et al. 2020

IJLTER

View full text Add to dashboard Cite

Private tutoring, also known as shadow education (SE), is significant in the view of Indonesian society. It attains public reliance on children’s success at school. Although SE seems to be promising to elevate students’ achievements in school, this still needs to be proven. With the development of 21st-century skills, can SE make itself worthy of being a part of proper supplementary education? This research aims to reveal how relevant SE is to develop students’ 21st-century skills, especially critical thinking (CT) skills. This study involved 193 students from 3 different ranks of schools in Surakarta, Indonesia. This mixed methods explanatory research used a computerized testlet instrument to obtain quantitative data about students’ CT skills in Chemistry. Open-ended questionnaires and open-ended interviews were used to obtain qualitative data about students’ tendencies in their SE. Interviews were also performed to hear SE tutors’ voices about the SE field in which they worked. The quantitative data were analyzed using Rasch modeling to reveal the interaction between the items and students’ CT skills. The modeling results showed that SE still seems less significant in responding to the community’s wishes about their children’s success at school, especially as related to CT skills. However, the results also showed that SE can aid those with moderate cognitive ability. Furthermore, the qualitative results are worth considering by those who will take or build SE in future. Thus, SE can be a proper supplement for formal education.

show abstract

“…As such, it is documented that testlet-based IRT models may be used (e.g. Frey, Seitz & Brandt, 2016). Additionally, the bifactor model used within the scope of this study may be used for testlet-based tests (see DeMars, 2006).…”

Section: Recommendations For Future Researchmentioning

confidence: 99%

Development of a Multidimensional Computerized Adaptive Test based on the Bifactor Model

Şahin

Gelbal

2020

International Journal of Assessment Tools in Education

View full text Add to dashboard Cite

The purpose of this study was to conduct a real-time multidimensional computerized adaptive test (MCAT) using data from a previous paper-pencil test (PPT) regarding the grammar and vocabulary dimensions of an end-of-term proficiency exam conducted on students in a preparatory class at a university. An item pool was established through four separate 50-item sets applied in four different semesters. The fit between unidimensional, multi-unidimensional and bifactor IRT models was compared during item calibration, with the bifactor model providing the best fit for all data sets. This was followed by a hybrid simulation for 36 conditions obtained using six item selection methods, two ability estimation methods and three termination rules. The statistics and graphs obtained indicate Drule item selection, maximum a posteriori (MAP) ability estimation and standard error termination rule as the best algorithm for the real-time MCAT application. With the minimum number of items to be administered determined as 10, the realtime application conducted on 99 examinees yielded an average number of items of 13.4. The PPT format proficiency exam consists of 50 items, leading to the conclusion that the examinees participating in the real-time MCAT are administered an average of 74.4% fewer items than the PPT. Additionally, 86 of the examinees answered between 10-13 items. The item pool use rate is 30%. Lastly, the correlation between the PPT scores and general trait scores of 32 examinees was calculated as .77.

show abstract

Testlet-Based Multidimensional Adaptive Testing

Cited by 10 publications

References 52 publications

Computerized adaptive testing for testlet‐based innovative items

Computerized adaptive testing for testlet‐based innovative items

Shadow Education in Indonesia: Is It Relevant to Students’ Critical Thinking Skills in Chemistry Learning?

Development of a Multidimensional Computerized Adaptive Test based on the Bifactor Model

Contact Info

Product

Resources

About