A Role-Selected Sharing Network for Joint Machine-Human Chatting Handoff and Service Satisfaction Analysis

Abstract

Chatbot is increasingly thriving in different domains, however, because of unexpected discourse complexity and training data sparseness, its potential distrust hatches vital apprehension. Recently, Machine-Human Chatting Handoff (MHCH), predicting chatbot failure and enabling human-algorithm collaboration to enhance chatbot quality, has attracted increasing attention from industry and academia. In this study, we propose a novel model, Role-Selected Sharing Network (RSSN), which integrates both dialogue satisfaction estimation and handoff prediction in one multi-task learning framework. Unlike prior efforts in dialog mining, by utilizing local user satisfaction as a bridge, global satisfaction detector and handoff predictor can effectively exchange critical information. Specifically, we decouple the relation and interaction between the two tasks by the role information after the shared encoder. Extensive experiments on two public datasets demonstrate the effectiveness of our model.

Causal Inference for Chatting Handoff

“…We evaluate our approach on four datasets including Clothing (Liu et al, 2021a), Makeup1 (Liu et al, 2021a), Clothes (Liu et al, 2021b), and Makeup2 (Liu et al, 2021b). The statistics of the data are shown in Table 2.…”

“…The statistics of the data are shown in Table 2. To verify the effectiveness of CEM and fairly compare with the baselines on the same datasets, we evaluate the performance of our CEM on the classification model DAMI (Liu et al, 2021a) by using Clothing and Makeup1 while test the performance of CEM on the multi-task model RSSN (Liu et al, 2021b) with Clothes and Makeup2, for the reason that these models are stateof-the-art. Because DAMI only models the relationship of D → Y , not D → LS → GS, it is not possible to use DAMI to get U S, so we only add cost adjustment on DAMI named as CEM-DAMI (C).…”

“…DCR-Net (Qin et al, 2020): It considers the cross-impact and model the interaction between the two tasks by introducing a co-interactive relation layer. RSSN (Liu et al, 2021b): It integrates both dialogue satisfaction estimation and handoff prediction in one multi-task learning framework.…”

“…In fact, a high-quality MHCH service should consider multiple factors, such as dialogue context, local sentiments, global satisfaction, user state, and labor cost, etc. However, most existing MHCH methods mainly concerned on the dialogue context (Liu et al, 2021a) or assisting with global satisfaction prediction under the multi-task learning setting (Liu et al, 2021b), ignoring the grounded relationships among the other causal variables of MHCH, like the user state and human cost.…”