Use of Resources and Method of Proctoring During the NBCRNA Continued Professional Certification Assessment: Analysis of Outcomes

Spence, Dennis; Ward, Robyn C; Wooden, Steve; Browne, Mary; Song, Hao; Hawkins, Robert J.; Wojnakowski, Mary

doi:10.1016/s2155-8256(19)30147-4

Cited by 5 publications

(6 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, these issues revealed that the system was vulnerable to the low quality of the personal gear of the candidates. Therefore, education institutes should guarantee a fair and safe exam environment for every candidate by offering high-quality infrastructure and logistics [ 4 , 12 , 13 ].…”

Section: Discussionmentioning

confidence: 99%

“…Further, the exam was performed within a restricted time span; therefore, students did not have time to cheat. Consequently, students were probably very reluctant to cheat, and this might have increased their stress levels [ 10 , 13 ].…”

Section: Discussionmentioning

confidence: 99%

“…The development and administration processes of proctored exams are still under construction and have been boosted during the COVID-19 pandemic [5]. The most commonly applied antifraud measures are direct face visualization and visual verification through a webcam [11][12][13]. However, screening all of the webcam images of large groups of students is time consuming, and image quality is highly dependent on webcam technology, a stable internet connection, and background lighting [11,12].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Efficiency, Usability, and Outcomes of Proctored Next-Level Exams for Proficiency Testing in Primary Care Education: Observational Study

Schoenmakers¹,

Wens²

2021

JMIR Form Res

View full text Add to dashboard Cite

Background The COVID-19 pandemic has affected education and assessment programs and has resulted in complex planning. Therefore, we organized the proficiency test for admission to the Family Medicine program as a proctored exam. To prevent fraud, we developed a web-based supervisor app for tracking and tracing candidates’ behaviors. Objective We aimed to assess the efficiency and usability of the proctored exam procedure and to analyze the procedure’s impact on exam scores. Methods The application operated on the following three levels to register events: the recording of actions, analyses of behavior, and live supervision. Each suspicious event was given a score. To assess efficiency, we logged the technical issues and the interventions. To test usability, we counted the number of suspicious students and behaviors. To analyze the impact that the supervisor app had on students’ exam outcomes, we compared the scores of the proctored group and those of the on-campus group. Candidates were free to register for off-campus participation or on-campus participation. Results Of the 593 candidates who subscribed to the exam, 472 (79.6%) used the supervisor app and 121 (20.4%) were on campus. The test results of both groups were comparable. We registered 15 technical issues that occurred off campus. Further, 2 candidates experienced a negative impact on their exams due to technical issues. The application detected 22 candidates with a suspicion rating of >1. Suspicion ratings mainly increased due to background noise. All events occurred without fraudulent intent. Conclusions This pilot observational study demonstrated that a supervisor app that records and registers behavior was able to detect suspicious events without having an impact on exams. Background noise was the most critical event. There was no fraud detected. A supervisor app that registers and records behavior to prevent fraud during exams was efficient and did not affect exam outcomes. In future research, a controlled study design should be used to compare the cost-benefit balance between the complex interventions of the supervisor app and candidates’ awareness of being monitored via a safe browser plug-in for exams.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Efficiency, Usability, and Outcomes of Proctored Next-Level Exams for Proficiency Testing in Primary Care Education: Observational Study

Schoenmakers¹,

Wens²

2021

JMIR Form Res

View full text Add to dashboard Cite

show abstract

“…Spence et al. (2019) conducted an experimental study involving 1,217 nurse anesthetists comparing OP and TC modes for recertification exams, crossed with type of resources available during testing. Their results showed a small decrease in pass rates in OP (91.8% vs. 95.1% in the TC condition), which they attributed to the “moderate problems” their candidates had with remote proctoring.…”

Section: The Unique Risks Posed By Online Proctored Examsmentioning

confidence: 99%

“…McLeod in Becker et al (2020) reported slightly lower average OP scores and lower test times in a medical maintenance of certification exam, where mode was self-selected, as well as score and test time distribution differences. Spence et al (2019) conducted an experimental study involving 1,217 nurse anesthetists comparing OP and TC modes for recertification exams, crossed with type of resources available during testing. Their results showed a small decrease in pass rates in OP (91.8% vs. 95.1% in the TC condition), which they attributed to the "moderate problems" their candidates had with remote proctoring.…”

Section: Observed Differences Between Op and Tc Scoresmentioning

confidence: 99%

Score Comparability between Online Proctored and In‐Person Credentialing Exams

Jones

Liu

et al. 2022

J Educational Measurement

View full text Add to dashboard Cite

This article studied two methods to detect mode effects in two credentialing exams.In Study 1, we used a "modal scale comparison approach," where the same pool of items was calibrated separately, without transformation, within two TC cohorts (TC1 and TC2) and one OP cohort (OP1) matched on their pool-based scale score distributions. The calibrations from all three groups were used to score the TC2 cohort, designated the validation sample. The TC1 item parameters and TC1-based thetas and pass rates were more like the native TC2 values than the OP1-based values, indicating mode effects, but the score and pass/fail decision differences were small. In Study 2, we used a "cross-modal repeater approach" in which test takers who failed their first attempt in one modality took the test again in either the same or different modality. The two pairs of repeater groups (TC → TC: TC → OP, and OP → OP: OP → TC) were matched exactly on their first attempt scores. Results showed increased pass rate and greater score variability in all conditions involving OP, with mode effects noticeable in both the TC → OP condition and lessstrongly in the OP → TC condition. Limitations of the study and implications for exam developers were discussed.

show abstract

Validity Evidence for the Continued Professional Certification Assessment: Professional Practice Analysis of the National Board of Certification and Recertification for Nurse Anesthetists

Ferris¹,

Gill²,

Preston³

et al. 2021

Journal of Nursing Regulation

View full text Add to dashboard Cite

Use of Resources and Method of Proctoring During the NBCRNA Continued Professional Certification Assessment: Analysis of Outcomes

Cited by 5 publications

References 5 publications

Efficiency, Usability, and Outcomes of Proctored Next-Level Exams for Proficiency Testing in Primary Care Education: Observational Study

Efficiency, Usability, and Outcomes of Proctored Next-Level Exams for Proficiency Testing in Primary Care Education: Observational Study

Score Comparability between Online Proctored and In‐Person Credentialing Exams

Validity Evidence for the Continued Professional Certification Assessment: Professional Practice Analysis of the National Board of Certification and Recertification for Nurse Anesthetists

Contact Info

Product

Resources

About