HDLTex: Hierarchical Deep Learning for Text Classification

Kowsari, Kamran; Brown, Donald E.; Heidarysafa, Mojtaba; Meimandi, Kiana Jafari; Gerber, Matthew S.; Barnes, Laura E.

doi:10.1109/icmla.2017.0-134

Cited by 317 publications

(233 citation statements)

References 35 publications

Supporting

Mentioning

229

Contrasting

Unclassified

Order By: Relevance

“…• Web of Science (WOS): This dataset was used in two previous works on hierarchical text classification (Kowsari et al, 2017;Sinha et al, 2018). It contains 134 topics, split across 7 parent categories.…”

Section: Methodsmentioning

confidence: 99%

“…Thus, adding or removing any label requires changing the model architecture. Second, while it is possible to retain some model parameters, such as in hierarchical classification models, these architectures must still learn separate weights for every new class or sub-class (Cai and Hofmann, 2004;Kowsari et al, 2017). This is problematic because the new class labels often come with very few training examples, providing insufficient information for learning accurate model weights.…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, these models do not leverage information across similar labels, which weakens their ability to adapt to new target labels (Kowsari et al, 2017;Tsochantaridis et al, 2005;Cai and Hofmann, 2004).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Metric Learning for Dynamic Text Classification

Wohlwend¹,

Elenberg²,

Altschul³

et al. 2019

Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019)

View full text Add to dashboard Cite

Traditional text classifiers are limited to predicting over a fixed set of labels. However, in many real-world applications the label set is frequently changing. For example, in intent classification, new intents may be added over time while others are removed.We propose to address the problem of dynamic text classification by replacing the traditional, fixed-size output layer with a learned, semantically meaningful metric space. Here the distances between textual inputs are optimized to perform nearest-neighbor classification across overlapping label sets. Changing the label set does not involve removing parameters, but rather simply adding or removing support points in the metric space. Then the learned metric can be fine-tuned with only a few additional training examples.We demonstrate that this simple strategy is robust to changes in the label space. Furthermore, our results show that learning a non-Euclidean metric can improve performance in the low data regime, suggesting that further work on metric spaces may benefit lowresource research. 1

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Metric Learning for Dynamic Text Classification

Wohlwend¹,

Elenberg²,

Altschul³

et al. 2019

Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019)

View full text Add to dashboard Cite

show abstract

“…Text classification problems have been widely studied and addressed in many real applications [1][2][3][4][5][6][7][8] over the last few decades. Especially with recent breakthroughs in Natural Language Processing (NLP) and text mining, many researchers are now interested in developing applications that leverage text classification methods.…”

Section: Introductionmentioning

confidence: 99%

Text Classification Algorithms: A Survey

et al. 2019

View full text Add to dashboard Cite

In recent years, there has been an exponential growth in the number of complex documents and texts that require a deeper understanding of machine learning methods to be able to accurately classify texts in many applications. Many machine learning approaches have achieved surpassing results in natural language processing. The success of these learning algorithms relies on their capacity to understand complex models and non-linear relationships within data. However, finding suitable structures, architectures, and techniques for text classification is a challenge for researchers. In this paper, a brief overview of text classification algorithms is discussed. This overview covers different text feature extractions, dimensionality reduction methods, existing algorithms and techniques, and evaluations methods. Finally, the limitations of each technique and their application in real-world problems are discussed.Spelling correction is an optional pre-processing step. Typos (short for typographical errors) are commonly present in texts and documents, especially in social media text data sets (e.g., Twitter). Many algorithms, techniques, and methods have addressed this problem in NLP [49]. Many techniques and methods are available for researchers including hashing-based and context-sensitive spelling correction techniques [50], as well as spelling correction using Trie and Damerau-Levenshtein distance bigram [51]. StemmingIn NLP, one word could appear in different forms (i.e., singular and plural noun form) while the semantic meaning of each form is the same [52]. One method for consolidating different forms of a word into the same feature space is stemming. Text stemming modifies words to obtain variant word forms using different linguistic processes such as affixation (addition of affixes) [53,54]. For example, the stem of the word "studying" is "study". LemmatizationLemmatization is a NLP process that replaces the suffix of a word with a different one or removes the suffix of a word completely to get the basic word form (lemma) [54][55][56]. Syntactic Word RepresentationMany researchers have worked on this text feature extraction technique to solve the loosing syntactic and semantic relation between words. Many researchers addressed novel techniques for solving this problem, but many of these techniques still have limitations. In [57], a model was introduced in which the usefulness of including syntactic and semantic knowledge in the text representation for the selection of sentences comes from technical genomic texts. The other solution for syntactic problem is using the n-gram technique for feature extraction. N-GramThe n-gram technique is a set of n-word which occurs "in that order" in a text set. This is not a representation of a text, but it could be used as a feature to represent a text.BOW is a representation of a text using its words (1-gram) which loses their order (syntactic). This model is very easy to obtain and the text can be represented through a vector, generally of a manageable size of the text. On the ...

show abstract

“…The collection and curation of large repositories of information has led to significant advancements in the training of machine learning approaches in areas as diverse as image recognition [42,89], textual analysis [47,63,98], and speech recognition [45,77,83,102]. In particular, in the realm of biomedical sciences, large relatively mature collections of information have been assembled covering areas such as genes and proteins [12,58], biological processes and pathways [5,57,71], drugs [1,46,76], and diseases [53,80,85].…”

Section: Introductionmentioning

confidence: 99%

Leveraging Distributed Biomedical Knowledge Sources to Discover Novel Uses for Known Drugs

Womack

McClelland

Koslicki

2019

Preprint

View full text Add to dashboard Cite

Computational drug repurposing, also called drug repositioning, is a low cost, promising tool for finding new uses for existing drugs. With the continued growth of repositories of biomedical data and knowledge, increasingly varied kinds of information are available to train machine learning approaches to drug repurposing. However, existing efforts to integrate a diversity of data sources have been limited to only a small selection of data types, typically gene expression data, drug structural information, and protein interaction networks. In this study, we leverage a graph-based approach to integrate biological knowledge from 20 publicly accessible repositories to represent information involving 11 distinct bioentity types. We then employ a graph node embedding scheme and use utilize a random forest model to make novel predictions about which drugs can be used to treat certain diseases. Utilizing this approach, we find a performance improvement over existing computational drug repurposing approaches and find promising drug repositioning targets, including drug and disease pairs currently in clinical trials.

show abstract

HDLTex: Hierarchical Deep Learning for Text Classification

Cited by 317 publications

References 35 publications

Metric Learning for Dynamic Text Classification

Metric Learning for Dynamic Text Classification

Text Classification Algorithms: A Survey

Leveraging Distributed Biomedical Knowledge Sources to Discover Novel Uses for Known Drugs

Contact Info

Product

Resources

About