Assumptions behind Intercoder Reliability Indices

Zhao, Xinshu; Liu, Jun S.; Deng, Ke

doi:10.1080/23808985.2013.11679142

Cited by 94 publications

(161 citation statements)

References 1 publication

Supporting

Mentioning

136

Contrasting

Unclassified

Order By: Relevance

“…Intersystem agreement between the CNN and manual FACS coding (i.e., ground truth) was quantified using AUC, F1 (positive agreement), NA (negative agreement), and free-marginal kappa [3, 4], which estimates chance agreement by assuming that each category is equally likely to be chosen at random [32]. (Table 1).…”

Section: Methodsmentioning

confidence: 99%

Automated Affect Detection in Deep Brain Stimulation for Obsessive-Compulsive Disorder

Cohn

Jeni

Ertuğrul

et al. 2018

Proceedings of the 20th ACM International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Automated measurement of affective behavior in psychopathology has been limited primarily to screening and diagnosis. While useful, clinicians more often are concerned with whether patients are improving in response to treatment. Are symptoms abating, is affect becoming more positive, are unanticipated side effects emerging? When treatment includes neural implants, need for objective, repeatable biometrics tied to neurophysiology becomes especially pressing. We used automated face analysis to assess treatment response to deep brain stimulation (DBS) in two patients with intractable obsessive-compulsive disorder (OCD). One was assessed intraoperatively following implantation and activation of the DBS device. The other was assessed three months post-implantation. Both were assessed during DBS on and o conditions. Positive and negative valence were quantified using a CNN trained on normative data of 160 non-OCD participants. Thus, a secondary goal was domain transfer of the classifiers. In both contexts, DBS-on resulted in marked positive affect. In response to DBS-off, affect flattened in both contexts and alternated with increased negative affect in the outpatient setting. Mean AUC for domain transfer was 0.87. These findings suggest that parametric variation of DBS is strongly related to affective behavior and may introduce vulnerability for negative affect in the event that DBS is discontinued.

show abstract

Section: Methodsmentioning

confidence: 99%

Automated Affect Detection in Deep Brain Stimulation for Obsessive-Compulsive Disorder

Cohn

Jeni

Ertuğrul

et al. 2018

Proceedings of the 20th ACM International Conference on Multimodal Interaction

View full text Add to dashboard Cite

show abstract

“…The second coder then recoded the initial group and coded one other randomly selected group. At the end of this process, a Cohen's Kappa, which has been the most often used index of reliability in social science (Zhao, Liu, & Deng, 2013), was calculated to determine the extent of agreement between the author and the second coder. The final Cohen's kappa is .83, indicating very acceptable agreement (Neuendorf, 2002).…”

Section: Reliabilitymentioning

confidence: 99%

Beyond mandatory use: Probing the affordances of social media for formal learning in the voluntary context

Zhou

Lee

2018

Proc. Assoc. Info. Sci. Tech.

View full text Add to dashboard Cite

The integration of social media in formal education has received consistent interest in the recent decade. While most studies have focused on the mandatory use of social media as pedagogical tools, less explored is the context in which students voluntarily choose and use social media to supplement their formal learning. In this study, we employ an affordance approach to exploring the resources in social media that students perceive and use voluntarily for formal learning. Drawing on a qualitative study using focus groups of university students, we identified ten categories of social media affordances for learning in the voluntary context: association, editability, immediacy, interactivity, manageability, modality, persistence, scalability, searchability, and visibility. The affordances mentioned most frequently by students were visibility, persistence, and interactivity. These identified affordances encompass the major resources offered by social media for learning in the voluntary context, and provide insights into understanding, evaluating, and guiding social media adoption in formal education as well as lifelong learning. This paper concludes by presenting limitations and implications for research and practice.

show abstract

“…It estimates chance agreement by assuming that each category is equally likely to be chosen at random [39]. When applied to two raters assigning objects to dichotomous categories, the S score is calculated using (1), where n 00 is the number of objects that both raters assigned to the negative (i.e., absent) category and n 11 is the number of objects that both raters assigned to the positive (i.e., present) category.…”

Section: Baseline Methodsmentioning

confidence: 99%

Sayette Group Formation Task (GFT) Spontaneous Facial Expression Database

Girard

Chu

Jeni

et al. 2017

2017 12th IEEE International Conference on Automatic Face &Amp; Gesture Recognition (FG 2017)

View full text Add to dashboard Cite

Despite the important role that facial expressions play in interpersonal communication and our knowledge that interpersonal behavior is influenced by social context, no currently available facial expression database includes multiple interacting participants. The Sayette Group Formation Task (GFT) database addresses the need for well-annotated video of multiple participants during unscripted interactions. The database includes 172,800 video frames from 96 participants in 32 three-person groups. To aid in the development of automated facial expression analysis systems, GFT includes expert annotations of FACS occurrence and intensity, facial landmark tracking, and baseline results for linear SVM, deep learning, active patch learning, and personalized classification. Baseline performance is quantified and compared using identical partitioning and a variety of metrics (including means and confidence intervals). The highest performance scores were found for the deep learning and active patch learning methods. Learn more at http://osf.io/7wcyz.

show abstract

Assumptions behind Intercoder Reliability Indices

Cited by 94 publications

References 1 publication

Automated Affect Detection in Deep Brain Stimulation for Obsessive-Compulsive Disorder

Automated Affect Detection in Deep Brain Stimulation for Obsessive-Compulsive Disorder

Beyond mandatory use: Probing the affordances of social media for formal learning in the voluntary context

Sayette Group Formation Task (GFT) Spontaneous Facial Expression Database

Contact Info

Product

Resources

About