Bridging language barriers in developing valid health policy research tools: insights from the translation and validation process of the SHEMESH questionnaire

Abstract

Background The use of research tools developed and validated in one cultural and linguistic context to another often faces challenges. One major challenge is poor performance of the tool in the new context. This potentially impact the legitimacy of health policy research conducted with informal adaptations of existing tools which have not been subjected to formal validation. Best practices exist to guide researchers in adapting and validating research tools effectively. We present here, as an extended example, our validation of the SHEMESH questionnaire ('Organizational Readiness to Change Assessment'; In Hebrew: 'SHE'elon Muchanut Ergunit le'SHinuy'), a Hebrew-language version of the Organizational Readiness to Change Assessment (ORCA). SHEMESH is tailored to support implementation science projects, whose aim is to promote a more rapid and complete adoption of evidence-based health policies and practices. Methods The SHEMESH included originally eleven questions from the Evidence (item 1–4) and Context (items 5–11) domains. We validated SHEMESH through the following steps: 1. Professional translation to Hebrew and discussion of the translation by multidisciplinary committee; 2. Back-translation into English by a different translator to detect discrepancies; 3. Eleven cognitive interviews with psychiatric emergency department physicians and nurses; and 4. Pilot testing and psychometric analyses, including Cronbach’s alpha for subscales and factor analyses. Results Following translation and cognitive interviews, SHEMESH was administered to 222 psychiatrists and nurses. Pearson correlation showed significant and strong correlations of items 1–4 to the Evidence construct and items 6–11 to the Context construct. Item 5 did not correlate with the other items, and therefore was removed from the other psychometric procedures and eventually from the SHEMESH. Factor analysis with the remaining 10 items yielded two factors, which together explained a total of 69.7% of variance. Cronbach's Alpha scores for the two subscales were high (Evidence, 0.887, and Context, 0.852). Conclusions This multi-step validation process of the SHEMESH questionnaire may serve as a comprehensive guideline for others who are willing to adapt research tools that were developed in other languages. Practically, SHEMESH has been validated for use in implementation science research projects in Israel.

Enabling Expedited Disposition of Emergencies Using Telepsychiatry in Israel: Protocol for a Hybrid Implementation Study

“…The SHEMESH will examine medical and nursing staff perceptions about the evidence of telepsychiatry and its implementation in the ED context. The SHEMESH has high Cronbach alpha scores for the 2 scales (total of 10 items on a 5-point Likert scale): evidence (4 items): α=.887; context (6 items): α=.852 [ 39 ]. The questionnaire will be administered online once before implementing telepsychiatry and again 3 months after the implementation.…”

Measuring Quality of Life through Validated Tools

“…The initial translation of an instrument into the local language should be substantiated with back-translation, engagement of several professional translators or addition of a committee tasked to compare the original to translated version. 5 Construct validity can be established by comparing the new form to other tools that determine the same concepts and is typically evaluated by degree of agreement between the two measures. 6 The last step in the process would include measurement of psychometric properties and pilot testing.…”

“…Psychometric index such as Cronbach’s alpha is often reported as a validation measure to demonstrate internal reliability among questionnaire items. 5 …”