One Size Does Not Fit All

Abstract

Mallinckrodt et al. developed the measure Everyday Multicultural Competencies and combined its items with those in the Scale of Ethnocultural Empathy to form a new scale to assess college multicultural programming. Mallinckrodt, Miles, and Recabarren used this scale as an example to provide recommendations for using focus groups and item response theory (IRT) in instrument development. In this commentary, we focus on two areas of major concerns: (a) paradigmatic and methodological issues pertaining to the use of mixed methods research design, focus groups, and IRT in developing psychological instruments aiming to assess multicultural constructs and (b) the importance of attending to the multidimensionality and context of these constructs. We believe it is important for future researchers to provide a thorough accounting of these paradigmatic and methodological issues in their studies and a clear explication of the target construct in scale development research.

Reconsidering Focus Groups and Rasch Model Item Response Theory in Instrument Development

“…We agree with all our colleagues Harvey (2016), Wang et al (2016), and Whittaker and Worthington (2016) that techniques based on the Rasch model should never be uncritically adopted and that one model must never be assumed best in all circumstances. We pointed out that many of the powerful benefits of the Rasch model derive from its assumption of specific objectivity, which in turn depends on a simplified view of how latent constructs, test items, and test takers interact.…”

“…However, we acknowledge that this diversity, and the diversity of the samples used in the validation study, was limited to the context of a predominantly White institution, which limits the applicability of the current EMC/RSEE to White students. Wang et al (2016) also bring up the issue of power dynamics that may exist within preexisting groups. Although we tried to minimize this to the extent possible by conducting separate groups for individuals presumed to have similar levels of power (i.e., undergraduate students, graduate students, and university administrators), we agree that a further examination of power dynamics and preexisting relationships could aid in the interpretation of focus group data.…”

“…Mallinckrodt and Tekie created the Brief Alliance Inventory by selecting 16 items that preserved as much of the original bandwidth and information value of the 36-item WAI as possible. Wang et al (2016) raised the important concern that "everyday multicultural competencies are not an end product that can be measured using a brief scale." We appreciate the quote from Desmond Tutu about "little bits of good" creating social change because it reflects our own views on social justice.…”

“…Part of the concern of Wang et al (2016) arises from the fact that the EMC/ RSEE is limited in scope to diversity related to race and ethnicity, and that diversity programming on college campuses often focuses on other forms of culture (e.g., gender, sexual orientation) and associated forms of oppression or "-isms" (e.g., sexism, heterosexism, respectively). This is true on our campus as well, and so we see the limitation of the EMC/RSEE as a measure focused on multicultural competencies related to race and ethnicity.…”

“…We view the latter as mostly differences of emphasis rather than as diametrically opposite viewpoints. Wang et al (2016) noted that we did not clearly articulate our paradigmatic stance or attend to epistemological issues in our recommendations for using focus groups. We agree that this is important in mixed-methods research.…”