A Multilevel CFA–MTMM Approach for Multisource Feedback Instruments: Presentation and Application of a New Statistical Model

Mahlke, Jana; Schultze, Martin; Koch, Tobias; Eid, Michael; Eckert, Regina; Brodbeck, Felix C.

doi:10.1080/10705511.2014.990153

Cited by 16 publications

(19 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Figure and in the simulation study we used a model with a general trait factor of the self‐report that is measured by three unidimensional indicators per trait. The model can also be defined with a different number of indicators or with indicator‐specific trait factors (Mahlke et al ., ).…”

Section: The Simulation Modelmentioning

confidence: 97%

“…To our knowledge, up to the publication of the model by Mahlke et al . (), there were no comparable multilevel models with two (or more) distinct sets of level‐1 units nested within the same level‐2 units. Therefore, there is no available option or command in statistical software to handle such data structures.…”

Section: The Simulation Modelmentioning

confidence: 99%

“…Therefore, one cannot directly derive recommendations for the application of the model by Mahlke et al . (). Additionally, it needs to be tested whether the implementation in statistical software works as expected.…”

Section: Introductionmentioning

confidence: 97%

“…Mahlke et al . () recently presented a multilevel confirmatory factor analysis (CFA)‐MTMM model that was specifically developed to handle these data structures. The model can be applied to analyse the reliability, discriminant validity and different measures of convergent validity, that is, the agreement between the target leader and the group of subordinates or the group of peers, the agreement among peers or among subordinates, and the agreement between peers and subordinates.…”

Section: Introductionmentioning

confidence: 99%

“…While the model and its implementation in statistical software have been thoroughly described by Mahlke et al . (), it is not yet clear under what conditions the model provides appropriate estimates. Researchers who want to apply the model need to know what sample sizes are required and how model characteristics such as model size or the degree of discriminant and convergent validity affect the estimation.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Analysing multisource feedback with multilevel structural equation models: Pitfalls and recommendations from a simulation study

Mahlke

Schultze

Eid

2019

Brit J Math & Statis

Self Cite

View full text Add to dashboard Cite

When multisource feedback instruments, for example, 360-degree feedback tools, are validated, multilevel structural equation models are the method of choice to quantify the amount of reliability as well as convergent and discriminant validity. A non-standard multilevel structural equation model that incorporates self-ratings (level-2 variables) and others' ratings from different additional perspectives (level-1 variables), for example, peers and subordinates, has recently been presented. In a Monte Carlo simulation study, we determine the minimal required sample sizes for this model. Model parameters are accurately estimated even with the smallest simulated sample size of 100 self-ratings and two ratings of peers and of subordinates. The precise estimation of standard errors necessitates sample sizes of 400 self-ratings or at least four ratings of peers and subordinates. However, if sample sizes are smaller, mainly standard errors concerning common method factors are biased. Interestingly, there are trade-off effects between the sample sizes of self-ratings and others' ratings in their effect on estimation bias. The degree of convergent and discriminant validity has no effect on the accuracy of model estimates. The v 2 test statistic does not follow the expected distribution. Therefore, we suggest using a corrected level-specific standardized root mean square residual to analyse model fit and conclude with further recommendations for applied organizational research. Supporting InformationThe following supporting information may be found in the online edition of the article: Appendix S1. R input for data generation.Appendix S2. Mplus input for model estimation.Analysing multisource feedback 315

show abstract

Section: The Simulation Modelmentioning

confidence: 97%

Section: The Simulation Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 97%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations