Towards integrated dialogue policy learning for multiple domains and intents using Hierarchical Deep Reinforcement Learning

Saha, Tulika; Gupta, Dhawal; Saha, Sriparna; Bhattacharyya, Pushpak

doi:10.1016/j.eswa.2020.113650

Cited by 16 publications

(12 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To improve investigation efficacy and patient satisfaction, clinics used to have different departments such as ENT (Ear, Nose, and Throat) and pediatrics, etc. Motivated by the real-world scenario and the promising results obtained by Liao et al [ 12 , 37 ], we also utilized a hierarchical policy learning method, where the higher-level policy (controller) activates one of the lower-level policies (departmental) depending on patients’ self-report and other symptoms and the department policy conducts group-specific symptom investigation.…”

Section: Methodsmentioning

confidence: 99%

Symptoms are known by their companies: towards association guided disease diagnosis assistant

Tiwari

Saha

et al. 2022

BMC Bioinformatics

Self Cite

View full text Add to dashboard Cite

Over the last few years, dozens of healthcare surveys have shown a shortage of doctors and an alarming doctor-population ratio. With the motivation of assisting doctors and utilizing their time efficiently, automatic disease diagnosis using artificial intelligence is experiencing an ever-growing demand and popularity. Humans are known by the company they keep; similarly, symptoms also exhibit the association property, i.e., one symptom may strongly suggest another symptom’s existence/non-existence, and their association provides crucial information about the suffering condition. The work investigates the role of symptom association in symptom investigation and disease diagnosis process. We propose and build a virtual assistant called Association guided Symptom Investigation and Diagnosis Assistant (A-SIDA) using hierarchical reinforcement learning. The proposed A-SIDDA converses with patients and extracts signs and symptoms as per patients’ chief complaints and ongoing dialogue context. We infused association-based recommendations and critic into the assistant, which reinforces the assistant for conducting context-aware, symptom-association guided symptom investigation. Following the symptom investigation, the assistant diagnoses a disease based on the extracted signs and symptoms. The assistant then diagnoses a disease based on the extracted signs and symptoms. In addition to diagnosis accuracy, the relevance of inspected symptoms is critical to the usefulness of a diagnosis framework. We also propose a novel evaluation metric called Investigation Relevance Score (IReS), which measures the relevance of symptoms inspected during symptom investigation. The obtained improvements (Diagnosis success rate-5.36%, Dialogue length-1.16, Match rate-2.19%, Disease classifier-6.36%, IReS-0.3501, and Human score-0.66) over state-of-the-art methods firmly establish the crucial role of symptom association that gets uncovered by the virtual agent. Furthermore, we found that the association guided symptom investigation greatly increases human satisfaction, owing to its seamless topic (symptom) transition.

show abstract

Section: Methodsmentioning

confidence: 99%

Symptoms are known by their companies: towards association guided disease diagnosis assistant

Tiwari

Saha

et al. 2022

BMC Bioinformatics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Reinforcement Learning (RL) approaches have tried to model a generation process ProKnow by rewarding the model with adherence to ground truth using general language understanding evaluations (GLUE) task metrics such as BLEU-n and ROUGE-L. However, they do not explicitly model clinically practiced ProKnow which enables explainable NLG that end-users and domain experts can trust (Wang et al, 2018 ; Zhang and Bansal, 2019 ; Saha et al, 2020 ). Hence, a method that effectively utilizes ProKnow will contribute to algorithmic explainability in the NLG process (Gaur et al, 2021 ; Sheth et al, 2021 ).…”

Section: Related Workmentioning

confidence: 99%

ProKnow: Process knowledge for safety constrained and explainable question generation for mental health diagnostic assistance

Roy¹,

Soltani²,

Rawte³

et al. 2023

Front. Big Data

View full text Add to dashboard Cite

Virtual Mental Health Assistants (VMHAs) are utilized in health care to provide patient services such as counseling and suggestive care. They are not used for patient diagnostic assistance because they cannot adhere to safety constraints and specialized clinical process knowledge (ProKnow) used to obtain clinical diagnoses. In this work, we define ProKnow as an ordered set of information that maps to evidence-based guidelines or categories of conceptual understanding to experts in a domain. We also introduce a new dataset of diagnostic conversations guided by safety constraints and ProKnow that healthcare professionals use (ProKnow-data). We develop a method for natural language question generation (NLG) that collects diagnostic information from the patient interactively (ProKnow-algo). We demonstrate the limitations of using state-of-the-art large-scale language models (LMs) on this dataset. ProKnow-algo incorporates the process knowledge through explicitly modeling safety, knowledge capture, and explainability. As computational metrics for evaluation do not directly translate to clinical settings, we involve expert clinicians in designing evaluation metrics that test four properties: safety, logical coherence, and knowledge capture for explainability while minimizing the standard cross entropy loss to preserve distribution semantics-based similarity to the ground truth. LMs with ProKnow-algo generated 89% safer questions in the depression and anxiety domain (tested property: safety). Further, without ProKnow-algo generations question did not adhere to clinical process knowledge in ProKnow-data (tested property: knowledge capture). In comparison, ProKnow-algo-based generations yield a 96% reduction in our metrics to measure knowledge capture. The explainability of the generated question is assessed by computing similarity with concepts in depression and anxiety knowledge bases. Overall, irrespective of the type of LMs, ProKnow-algo achieved an averaged 82% improvement over simple pre-trained LMs on safety, explainability, and process-guided question generation. For reproducibility, we will make ProKnow-data and the code repository of ProKnow-algo publicly available upon acceptance.

show abstract

“…RL does not require any data to be given in advance, which obtains the reward by the continuous interaction between agent and environment. By employing the RL, a system dynamically adjusts the parameters to maximize the accumulated reward [ 51 , 52 ]. In RL, the return function is usually defined to represent the sum of the discounts of all rewards observed by the agent after a certain state, i.e.,

where,

is the discount factor (

), which represents the weight relationship between future rewards and immediate reward, and R is the immediate reward.…”

Section: Preliminariesmentioning

confidence: 99%

A Novel Conflict Management Method Based on Uncertainty of Evidence and Reinforcement Learning for Multi-Sensor Information Fusion

Huang

Wang

Deng

2021

Entropy

View full text Add to dashboard Cite

Dempster–Shafer theory (DST), which is widely used in information fusion, can process uncertain information without prior information; however, when the evidence to combine is highly conflicting, it may lead to counter-intuitive results. Moreover, the existing methods are not strong enough to process real-time and online conflicting evidence. In order to solve the above problems, a novel information fusion method is proposed in this paper. The proposed method combines the uncertainty of evidence and reinforcement learning (RL). Specifically, we consider two uncertainty degrees: the uncertainty of the original basic probability assignment (BPA) and the uncertainty of its negation. Then, Deng entropy is used to measure the uncertainty of BPAs. Two uncertainty degrees are considered as the condition of measuring information quality. Then, the adaptive conflict processing is performed by RL and the combination two uncertainty degrees. The next step is to compute Dempster’s combination rule (DCR) to achieve multi-sensor information fusion. Finally, a decision scheme based on correlation coefficient is used to make the decision. The proposed method not only realizes adaptive conflict evidence management, but also improves the accuracy of multi-sensor information fusion and reduces information loss. Numerical examples verify the effectiveness of the proposed method.

show abstract

Towards integrated dialogue policy learning for multiple domains and intents using Hierarchical Deep Reinforcement Learning

Cited by 16 publications

References 16 publications

Symptoms are known by their companies: towards association guided disease diagnosis assistant

Symptoms are known by their companies: towards association guided disease diagnosis assistant

ProKnow: Process knowledge for safety constrained and explainable question generation for mental health diagnostic assistance

A Novel Conflict Management Method Based on Uncertainty of Evidence and Reinforcement Learning for Multi-Sensor Information Fusion

Contact Info

Product

Resources

About