Deep Contextualized Word Representations

Peters, Matthew E.; Neumann, Mark E; Iyyer, Mohit; Gardner, Matt; Clark, Christopher; Lee, Kenton; Zettlemoyer, Luke

doi:10.18653/v1/n18-1202

Cited by 9,271 publications

(7,187 citation statements)

References 38 publications

Supporting

Mentioning

6,689

Contrasting

Unclassified

Order By: Relevance

“…Our model in the SQuAD dataset for ensemble showed performance of 81.5% F1 and 73.3% EM. Moreover, we performed performance measurements by combining S 2 ‐Net and ELMo , which had performance of 83.1% F1 (74.6% EM) and 2.3% more improvement than the S 2 ‐Net model. ELMo is a function of the internal states of a deep bidirectional language model (biLM) that is pretrained on a large text corpus.…”

Section: Resultsmentioning

confidence: 99%

S ² ‐Net: Machine reading comprehension with SRU‐based self‐matching networks

et al. 2019

View full text Add to dashboard Cite

Machine reading comprehension is the task of understanding a given context and finding the correct response in that context. A simple recurrent unit (SRU) is a model that solves the vanishing gradient problem in a recurrent neural network (RNN) using a neural gate, such as a gated recurrent unit (GRU) and long short‐term memory (LSTM); moreover, it removes the previous hidden state from the input gate to improve the speed compared to GRU and LSTM. A self‐matching network, used in R‐Net, can have a similar effect to coreference resolution because the self‐matching network can obtain context information of a similar meaning by calculating the attention weight for its own RNN sequence. In this paper, we construct a dataset for Korean machine reading comprehension and propose an S2‐Net model that adds a self‐matching layer to an encoder RNN using multilayer SRU. The experimental results show that the proposed S2‐Net model has performance of single 68.82% EM and 81.25% F1, and ensemble 70.81% EM, 82.48% F1 in the Korean machine reading comprehension test dataset, and has single 71.30% EM and 80.37% F1 and ensemble 73.29% EM and 81.54% F1 performance in the SQuAD dev dataset.

show abstract

Section: Resultsmentioning

confidence: 99%

S ² ‐Net: Machine reading comprehension with SRU‐based self‐matching networks

et al. 2019

View full text Add to dashboard Cite

show abstract

“…12 Of the best-performing machine learning models tested, XGBoost had 91.9% accuracy. The ANN using an ELMo-BiLSTM-CNN-CRF architecture 13,14 had an accuracy of 94.4%.The methods used were only applied to the VetCompass corpus for companion animals. However, results could be expanded to other species and applied to other datasets within VetCompass.…”

Section: Discussionmentioning

confidence: 99%

Using natural language processing and VetCompass to understand antimicrobial usage patterns in Australia

et al. 2019

View full text Add to dashboard Cite

Background Currently there is an incomplete understanding of antimicrobial usage patterns in veterinary clinics in Australia, but such knowledge is critical for the successful implementation and monitoring of antimicrobial stewardship programs. Methods VetCompass Australia collects medical records from 181 clinics in Australia (as of May 2018). These records contain detailed information from individual consultations regarding the medications dispensed. One unique aspect of VetCompass Australia is its focus on applying natural language processing (NLP) and machine learning techniques to analyse the records, similar to efforts conducted in other medical studies. Results The free text fields of 4,394,493 veterinary consultation records of dogs and cats between 2013 and 2018 were collated by VetCompass Australia and NLP techniques applied to enable the querying of the antimicrobial usage within these consultations. Conclusion The NLP algorithms developed matched antimicrobial in clinical records with 96.7% accuracy and an F1 Score of 0.85, as evaluated relative to expert annotations. This dataset can be readily queried to demonstrate the antimicrobial usage patterns of companion animal practices throughout Australia.

show abstract

“…This approach learns lexical representations from a large unannotated corpus (Devlin, Chang, Lee, & Toutanova, ;Mikolov et al, 2013;Pennington et al, 2014;Peters et al, 2018). The classical model word2vec contains two models that can generate word embeddings (Mikolov et al, 2013).…”

Section: Corpus-based Approachmentioning

confidence: 99%

Combining and learning word embedding with WordNet for semantic relatedness and similarity measurement

Lee

Hao

Yen

et al. 2019

Asso for Info Science & Tech

View full text Add to dashboard Cite

In this research, we propose 3 different approaches to measure the semantic relatedness between 2 words: (i) boost the performance of GloVe word embedding model via removing or transforming abnormal dimensions; (ii) linearly combine the information extracted from WordNet and word embeddings; and (iii) utilize word embedding and 12 linguistic information extracted from WordNet as features for Support Vector Regression. We conducted our experiments on 8 benchmark data sets, and computed Spearman correlations between the outputs of our methods and the ground truth. We report our results together with 3 state-of-the-art approaches. The experimental results show that our method can outperform state-of-the-art approaches in all the selected English benchmark data sets.

show abstract

Deep Contextualized Word Representations

Cited by 9,271 publications

References 38 publications

S ² ‐Net: Machine reading comprehension with SRU‐based self‐matching networks

S ² ‐Net: Machine reading comprehension with SRU‐based self‐matching networks

Using natural language processing and VetCompass to understand antimicrobial usage patterns in Australia

Combining and learning word embedding with WordNet for semantic relatedness and similarity measurement

Contact Info

Product

Resources

About

Deep Contextualized Word Representations

Cited by 9,271 publications

References 38 publications

S 2 ‐Net: Machine reading comprehension with SRU‐based self‐matching networks

S 2 ‐Net: Machine reading comprehension with SRU‐based self‐matching networks

Using natural language processing and VetCompass to understand antimicrobial usage patterns in Australia

Combining and learning word embedding with WordNet for semantic relatedness and similarity measurement

Contact Info

Product

Resources

About

S ² ‐Net: Machine reading comprehension with SRU‐based self‐matching networks

S ² ‐Net: Machine reading comprehension with SRU‐based self‐matching networks