Assessing clinical reasoning in optometry using the script concordance test

Faucher, Caroline; Dufour‐guindon, Marie‐pier; Lapointe, Gilles; Gagnon, Robert; Charlin, Bernard

doi:10.1111/cxo.12354

Cited by 18 publications

(14 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Item-total correlation is the correlation between the score on a particular question and the collective score on all the remaining questions. In short, questions with low item-total correlations do not produce responses that are consistent with the remainder of the test (12). We used the item-total correlation to identify questions with low values (r<0.05).…”

Section: Discussionmentioning

confidence: 99%

Consensus Analysis for the Panel Responses and the Psychometrics Properties in Script Concordance Testing

Ganesan

Bhandary

Thulasingam

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: Clinical reasoning is an essential attribute in the teaching, learning, and assessment part of medical education for undergraduates. In using the Script Concordance Test (SCT) to foster clinical reasoning, expert panel members’ responses are initially created. There is no agreement in optimizing the panel members’ responses. Our study aimed to develop and validate an SCT and test the utility of the consensus index and panel response pattern. Methods: The methodology was an evolving pattern of constructing SCTs, administering them to the panel members, optimizing the panel with response pattern and consensus index. The SCT’s final items were chosen to be administered to the students. Item-total correlation and Cronbach’s alpha were calculated from the students’ scores. Results: Our study developed an SCT with 98 items and was administered to 20-panel members. The mean score of the panel members for these 98 items was 79.5 (+/- 4.4 SD). On optimizing with the panel responses, 14 items had a uniform response pattern, and 2 had bimodal response patterns. The consensus index calculated for the 98 item SCT ranged from 25.81 to 100. When the 16 items of bimodal and uniform response pattern were eliminated, the consensus index ranged from 58.65 to 100. We administered this 82 items SCT to 30 undergraduate and ten postgraduate students. The mean score of undergraduate students was 61.1 (+/-7.5 SD), and that of postgraduate students was 67.7 (+/- 6.3 SD), which was statistically significant using an independent t-test. Cronbach’s alpha for this 82 item SCT was 0.74. On analysing the item-total correlation, 22 items had a correlation of less than 0.05. Excluding these 22 poor items, the final SCT instrument of 60 items had a Cronbach’s alpha of 0.82. Conclusion: The consensus index can also be used to optimize the items for panel responses in SCT. Our study revealed that a consensus index of above 60 had a good item-total correlation with good internal consistency. Our study also revealed that the panel response clustering pattern could also be used to categorize the items though bimodal and uniform distribution patterns need further differentiation.

show abstract

Section: Discussionmentioning

confidence: 99%

Consensus Analysis for the Panel Responses and the Psychometrics Properties in Script Concordance Testing

Ganesan

Bhandary

Thulasingam

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Other professions or specialties have recently used SCT questions to evaluate clinical reasoning in a reliable manner with the ability to discriminate between various levels of expertise 8-11. In the future, it may be of interest to ask students who have not taken the course to complete the SCT to characterize how they may perform and potentially identify the robustness of this exam in determining psychiatric pharmacy knowledge.…”

Section: Discussionmentioning

confidence: 99%

Evaluation of the use of script concordance test in a multicampus psychiatric pharmacy elective course

Phan¹

2019

Mental Health Clinician

View full text Add to dashboard Cite

Introduction Evaluating a student's ability to accept complexity, uncertainty, and ambiguity as part of clinical practice is difficult in a classroom setting using written tests. This study was conducted to explore the feasibility and validation of using a script concordance test (SCT) to evaluate pharmacy student knowledge and clinical competence in a psychiatry elective course. Methods This study involved prospective validation of psychiatry-focused SCT questions using a panel of practicing psychiatric pharmacists and retrospective review of student performance on the same SCT questions. The reliability of the SCT was also evaluated using Cronbach alpha coefficient. Results A total of 13 practicing psychiatric pharmacists participated in the validation phase of the study of 75 questions. Pharmacy student scores (n = 17) averaged 39.79 (±5.02) points, and psychiatric pharmacist scores averaged 50.11 (±4.51) points, representing mean percentages of 61.2% and 77.1%, respectively, on the adjusted exam. The Cronbach alpha was 0.94. Discussion The development of a valid and reliable SCT to test student psychiatric pharmacy knowledge and clinical competence after taking a psychiatry elective course was feasible.

show abstract

“…optometry and physiotherapy). 11,12 There is evidence of SCT validity and reliability in the literature 9,13-15 but these remain issues of ongoing debate. 16,17 The reliability of SCT scores has been reported to be around a Cronbach's alpha value of 0.7-0.85.…”

Section: Q3 Carcinoma Of the Colon A Normal Digital Rectal Examinationmentioning

confidence: 99%

“…Script concordance testing has also been used to assess clinical reasoning in other health care professions (e.g. optometry and physiotherapy) . There is evidence of SCT validity and reliability in the literature but these remain issues of ongoing debate .…”

Section: Introductionmentioning

confidence: 99%

Improving the validity of script concordance testing by optimising and balancing items

2018

View full text Add to dashboard Cite

When constructing SCT examinations, apart from the rigorous pre-examination optimisation, it is desirable to achieve a balance between items that attract extreme responses and those that attract median response options. This could mitigate the validity threat to SCT scores, especially for the low-performing students who have previously been shown to only select median responses and avoid the extreme responses.

show abstract

Assessing clinical reasoning in optometry using the script concordance test

Cited by 18 publications

References 41 publications

Consensus Analysis for the Panel Responses and the Psychometrics Properties in Script Concordance Testing

Consensus Analysis for the Panel Responses and the Psychometrics Properties in Script Concordance Testing

Evaluation of the use of script concordance test in a multicampus psychiatric pharmacy elective course

Improving the validity of script concordance testing by optimising and balancing items

Contact Info

Product

Resources

About