“I Think It Might Help If We Multiply, and Not Add”: Detecting Indirectness in Conversation

"You might think about slightly revising the title”: Identifying Hedges in Peer-tutoring Interactions

“…This is consistent with the feature analysis in Table 5, suggesting that tutoring moves do not significantly improve the performance of the classifier; (v) Nonverbal behaviors do not appear as important features for the classification. This is coherent with results from (Goel et al, 2019). Note that prosody might play a role in detecting instructions that trail off, but, as described, paraverbal features were not available; (vi) Would plays an important role in the production of hedges, as it is strongly associated to Propositional hedges (n=2).…”

“…In the appendix (see Table 8 and Table 9) we indicate the confidence intervals to represent the significance of the differences between the models. First, and perhaps surprisingly, we notice that the use of "Knowledge-Driven" features based on rules built from linguistic knowledge of hedges in the LightGBM model outperforms the use of pre-trained embeddings within a fine-tuned BERT model (79.0 vs. 70.6), and in the neural baseline from (Goel et al, 2019) Table 4: Averaged weighted F1-scores (and standard deviation) for the three minority classes and for the 4 classes, for all models. "KD" stands for "Knowledge-Driven", meaning that the features are derived from lexicon, n-gram models and annotations.…”

“…We also consider apologies when they serve a mitigation function (we then call them Apologizers), as was done by the authors of our corpus, and we also use the term subjectivizers as defined below, to be able to compare directly with the previous work carried out on this corpus. As far as we know, only Goel et al (2019) have worked with a peertutoring dataset (the same one that we also use), and they achieved their best classification result by employing an Attention-CNN model, inspired by Adel and Schütze (2017).…”

“…Thus, in the work described here, based on linguistic descriptions of hedges (Brown and Levinson, 1987;Fraser, 2010), we built a rule-based classifier. We show that this classifier in combination with additional multimodal interpretable context-dependent features significantly improves the performance of a machine learning model for hedges, compared to a less interpretable deep learning baseline from Goel et al (2019) using word embeddings. We also relied on a machine learning model explanation tool (Lundberg and Lee, 2017) to investigate the linguistic features related to hedges in the context of peer-tutoring, primarily to see if we could discover surprising features that the classification model would associate to hedges in this context, and we describe those below.…”

“…Nonverbal and paraverbal behaviors: As in Goel et al (2019), we included the nonverbal and paraverbal behaviors that are related to hedges. Specifically, we consider laughter and smiles, that have been shown to be effective methods of mitigation (Warner-Garcia, 2014), cut-offs indicating selfrepairs, fillers like "Um", gaze shifts (annotated as 'Gaze at Partner', 'Gaze at the Math Worksheet', and 'Gaze elsewhere'), and head nods.…”