Detecting group differences in sequential association using sampled permutations: Log odds, kappa, and phi compared

Abstract

When determining whether a particular transition is more characteristic of one group than of another, two things are required: an index associated with the transition of interest and a statistical test that can determine whether group membership systematically affects values for that index. Here the familiar parametric t test is compared with a test based on sampled permutations. Indices considered are the odds and log odds ratio, Yule's Q, Wampold's (1989) transformed kappa, and phi. The odds and log odds ratio are monotonically increasing functions of Yule'sQ and so give similar results. Yule's Q and phi are essentially rank order invariant and usually give similar results. Transformed kappa, however, rank orders subjects somewhat differently than the others; moreover, it appears somewhat biased. With respect to the tests, when subjects are 20 or more it does not matter much whether sampled permutation or parametric t tests are used; both yield essentially the same result. However, when subjects are fewer than 20, or whenever there is any other reason to think that parametric assumptions may not be met, permutation tests are recommended. A computer program that effects such tests is described.

Ad hoc procedure for optimising agreement between observational records

“…As can be seen, the kappa values obtained in the initial phase (before alignment) indicated very poor agreement between the data sets. The application of our adjustment procedure, however, ensured alignment of sequences (Bakeman et al, 1996;Quera et al, 2007) and the kappa value for agreement between data packages B-1 and B-2 increased from 0.5601 in the initial phase to 0.8420 in the adjustment phase.…”

“…While special alignment software programs exist for event sequences (Bakeman, 1978), such as Global Sequence Alignment (GSA) and Event Alignment (ELign) developed by Quera et al (2007), they are not applicable to multievent data (Bakeman & Quera, 1995). The second aim of this study thus was to describe how we overcame the problem of misaligned data (soccer play sequences in our case) to calculate the kappa statistic using order as the primary parameter (Bakeman, McArthur & Quera, 1996).…”

Proximity coefficients as a measure of interrelationships in sequences of behavior

“…One approach focuses on patterns and regularities in the step-by-step transitions among behaviors. The approach involves counting the number of times one behavior does and does not precede a second behavior, and from these counts computing a measure of the association between the two behaviors (Bakeman, McArthur, & Quera, 1996;Fagen & Mankovich, 1980;Wampold, 1989). In more sophisticated analyses, the frequencies may be combined with other conditional frequencies and submitted to a lag or log analysis as a way of determining the relative associations among combinations of behaviors (Chatfield & Lemon, 1970;Iacobucci & Wasserman, 1988).…”

“…The resulting associations (or "chains" of these associations) may be interpreted directly to provide insights into the character and underlying units of different types of sequences (Gottman, Markman, & Notarius, 1977;Taylor & Donald, 2003). Alternatively, they may be compared across groups to determine whether or not the two groups differ with respect to the transitions of interest (Bakeman et al, 1996;Yoder, Bruce, & Tapp, 2001).…”

“…The typical behavioral chains, the impact of different strategies on the other party's response, and the long-term ordering of behavior are just three aspects of behavior that can be examined by comparing such matrices. For most purposes, such comparisons may be achieved by computing a permutation test on the differences among coefficient values, as outlined by Efron and Tibshirani (1986), Bakeman et al (1996), and others. The analysis may assume independence of sampling when calculations are made across different sequences but would require adjustment to account for dependence among observations when comparing within a single sequence.…”