“…Third, in almost all studies where individual level data are aggregated, the author does not eliminate any group on the basis of a low or out-of-bound interrater agreement result, and the author decides to aggregate even when mean (or median) interrater agreement values are below rules-of-thumb cutoff values (e.g., see Binci, 2011;Boehm, Dwertmann, Bruch, & Shamir, 2015;Borucki & Burke, 1999;de Jong, de Ruyter, & Lemmink, 2005;Hofmann & Mark, 2006;McKay, Avery, & Morris, 2009;Morrison et al, 2011;Patterson et al, 2005;Simons & Roberson, 2003). In a very small number of studies, the researcher will eliminate groups based on r WG values being below .70 (e.g., see Aryee, Chen, & Budhwar, 2004;Riordan et al, 2005;Susskind, Kacmar, & Borchgrevink, 2003).…”