Knowledge Guided Hierarchical Multi-Label Classification Over Ticket Data

Zeng, Chunqiu; Zhou, Wubai; Li, Tao; Shwartz, Larisa; Grabarnik, Genady Ya.

doi:10.1109/tnsm.2017.2668363

Cited by 36 publications

(23 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other application domains have also explored HMC, such as managing IT services [37,38], text classification on social media [39], large scale document classification [40] and annotation of non-coding RNA [41]. It can even be applied to non-hierarchical multi-label problems where artificial hierarchies are created [42].…”

Section: Related Workmentioning

confidence: 99%

Machine learning for discovering missing or wrong protein function annotations

Nakano

Lietaert²,

Vens

2019

BMC Bioinformatics

View full text Add to dashboard Cite

Background A massive amount of proteomic data is generated on a daily basis, nonetheless annotating all sequences is costly and often unfeasible. As a countermeasure, machine learning methods have been used to automatically annotate new protein functions. More specifically, many studies have investigated hierarchical multi-label classification (HMC) methods to predict annotations, using the Functional Catalogue (FunCat) or Gene Ontology (GO) label hierarchies. Most of these studies employed benchmark datasets created more than a decade ago, and thus train their models on outdated information. In this work, we provide an updated version of these datasets. By querying recent versions of FunCat and GO yeast annotations, we provide 24 new datasets in total. We compare four HMC methods, providing baseline results for the new datasets. Furthermore, we also evaluate whether the predictive models are able to discover new or wrong annotations, by training them on the old data and evaluating their results against the most recent information. Results The results demonstrated that the method based on predictive clustering trees, Clus-Ensemble, proposed in 2008, achieved superior results compared to more recent methods on the standard evaluation task. For the discovery of new knowledge, Clus-Ensemble performed better when discovering new annotations in the FunCat taxonomy, whereas hierarchical multi-label classification with genetic algorithm (HMC-GA), a method based on genetic algorithms, was overall superior when detecting annotations that were removed. In the GO datasets, Clus-Ensemble once again had the upper hand when discovering new annotations, HMC-GA performed better for detecting removed annotations. However, in this evaluation, there were less significant differences among the methods. Conclusions The experiments have showed that protein function prediction is a very challenging task which should be further investigated. We believe that the baseline results associated with the updated datasets provided in this work should be considered as guidelines for future studies, nonetheless the old versions of the datasets should not be disregarded since other tasks in machine learning could benefit from them.

show abstract

Section: Related Workmentioning

confidence: 99%

Machine learning for discovering missing or wrong protein function annotations

Nakano

Lietaert²,

Vens

2019

BMC Bioinformatics

View full text Add to dashboard Cite

show abstract

“…This is more related to the issue of service times and becomes particularly relevant when the ticket that has Figure 1: Architecture of the proposed system been dispatched to a group needs to be assigned to an agent. There are some papers, which apply text classification techniques to handle tickets (Dasgupta et al 2014) (Zeng et al 2017). The idea is that once ticket category is identified, then the assignment to resolver groups can be done by manual dispatchers quickly.…”

Section: Related Workmentioning

confidence: 99%

Automated Dispatch of Helpdesk Email Tickets: Pushing the Limits with AI

Mandal

Malhotra

Agarwal

et al. 2019

AAAI

View full text Add to dashboard Cite

Ticket assignment/dispatch is a crucial part of service delivery business with lot of scope for automation and optimization. In this paper, we present an end-to-end automated helpdesk email ticket assignment system, which is also offered as a service. The objective of the system is to determine the nature of the problem mentioned in an incoming email ticket and then automatically dispatch it to an appropriate resolver group (or team) for resolution.The proposed system uses an ensemble classifier augmented with a configurable rule engine. While design of a classifier that is accurate is one of the main challenges, we also need to address the need of designing a system that is robust and adaptive to changing business needs. We discuss some of the main design challenges associated with email ticket assignment automation and how we solve them. The design decisions for our system are driven by high accuracy, coverage, business continuity, scalability and optimal usage of computational resources.Our system has been deployed in production of three major service providers and currently assigning over 90,000 emails per month, on an average, with an accuracy close to 90% and covering at least 90% of email tickets. This translates to achieving human-level accuracy and results in a net saving of more than 50000 man-hours of effort per annum. Till date, our deployed system has already served more than 700,000 tickets in production.

show abstract

“…This is more related to the issue of service times and becomes particularly relevant when the ticket that has been dispatched to a group needs to be assigned to an agent. There are some papers, which apply text classification techniques to handle tickets [6] [7]. The idea is that once ticket category is identified, then the assignment to resolver groups can be done by manual dispatchers quickly.…”

Section: Related Workmentioning

confidence: 99%

Automated dispatch of helpdesk email tickets: Pushing the limits with AI

Mandal

Malhotra

Agarwal

et al. 2018

EasyChair Preprints

View full text Add to dashboard Cite

Ticket assignment/dispatch is a crucial part of service delivery business with lot of scope for automation and optimization. In this paper, we present an end-to-end automated helpdesk email ticket assignment system, which is also offered as a service. The objective of the system is to determine the nature of the problem mentioned in an incoming email ticket and then automatically dispatch it to an appropriate resolver group (or team) for resolution.<br/>The proposed system uses an ensemble classifier augmented with a configurable rule engine. While design of classifier that is accurate is one of the main challenges, we also need to address the need for designing a system that is robust and adaptive to changing business needs. We discuss some of the main design challenges associated with email ticket assignment automation and how we solve them. The design decisions for our system are driven by high accuracy, coverage, business continuity, scalability and optimal usage of computational resources.<br/>Our system has been deployed in production of three major service providers and currently assigning over 90,000 emails per month, on an average, with an accuracy close to 90% and covering at least 90% of email tickets. This translates to achieving human-level accuracy and results in a net saving of more than 50000 man-hours of effort per annum.

show abstract

Knowledge Guided Hierarchical Multi-Label Classification Over Ticket Data

Cited by 36 publications

References 28 publications

Machine learning for discovering missing or wrong protein function annotations

Machine learning for discovering missing or wrong protein function annotations

Automated Dispatch of Helpdesk Email Tickets: Pushing the Limits with AI

Automated dispatch of helpdesk email tickets: Pushing the limits with AI

Contact Info

Product

Resources

About