From Monolingual to Multilingual FAQ Assistant using Multilingual Co-training

Abstract

Recent research on cross-lingual transfershow state-of-the-art results on benchmark datasets using pre-trained language representation models (PLRM) like BERT. These results are achieved with the traditional training approaches, such as Zero-shot with no data, Translate-train or Translate-test with machine translated data. In this work, we propose an approach of "Multilingual Co-training" (MCT) where we augment the expert annotated dataset in the source language (English) with the corresponding machine translations in the target languages (e.g. Arabic, Spanish) and fine-tune the PLRM jointly. We observe that the proposed approach provides consistent gains in the performance of BERT for multiple benchmark datasets (e.g. 1.0% gain on MLDocs, and 1.2% gain on XNLI over translate-train with BERT), while requiring a single model for multiple languages. We further consider a FAQ dataset where the available English test dataset is translated by experts into Arabic and Spanish. On such a dataset, we observe an average gain of 4.9% over all other cross-lingual transfer protocols with BERT. We further observe that domainspecific joint pre-training of the PLRM using HR policy documents in English along with the machine translations in the target languages, followed by the joint finetuning, provides a further improvement of 2.8% in average accuracy.

The acceptance of chatbot technology to support academic activity at the University of KwaZulu-Natal.

“…Pattern-based model (Joshi et al, 2015), the Retrieval-based model (Wu et al, 2016), and the Generative model (Patidar et al, 2017) are the three models utilized to generate the correct answers. In the Pattern-based paradigm, chatbots generate responses by comparing incoming user messages against a database of predefined question-and-answer patterns.…”

“…The increasing ratio of students to teachers is evidence of this (Ketakee & Champaneria, 2017). In other words, less time and resources will be devoted by teachers to helping each student (Patidar et al, 2017). Because of this, many students could not learn effectively and eventually left school (Hone & El Said, 2016;Patidar et al, 2017).…”

“…In other words, less time and resources will be devoted by teachers to helping each student (Patidar et al, 2017). Because of this, many students could not learn effectively and eventually left school (Hone & El Said, 2016;Patidar et al, 2017). Numerous proposed solutions have been proposed, but most can't be executed due to a lack of resources and insufficient infrastructure (Oeste et al, 2015).…”