MMR-based active machine learning for bio named entity recognition

Abstract

This paper presents a new active learning paradigm which considers not only the uncertainty of the classifier but also the diversity of the corpus. The two measures for uncertainty and diversity were combined using the MMR (Maximal Marginal Relevance) method to give the sampling scores in our active learning strategy. We incorporated MMR-based active machinelearning idea into the biomedical namedentity recognition system. Our experimental results indicated that our strategies for active-learning based sample selection could significantly reduce the human effort.

An analysis of active learning strategies for sequence labeling tasks

“…Note, however, that the number of possible labelings grows exponentially with the length of x. To make this feasible, previous work (Kim et al, 2006) has employed an approximation we call N-best sequence entropy (NSE):…”

“…A few methods have been proposed, based mostly on the conventions of uncertainty sampling, where the learner queries the instance about which it has the least certainty (Scheffer et al, 2001;Culotta and McCallum, 2005;Kim et al, 2006), or query-by-committee, where a "committee" of models selects the instance about which its members most disagree (Dagan and Engelson, 1995). We provide more detail on these and the new strategies we propose in Section 3.…”

Proactive Learning for Named Entity Recognition

“…This criterion assumes that the most uncertain sentence is the most useful instance for learning an NER model. There are several ways to implement this, such as least confidence (Culotta and McCallum, 2005)-the lower the probability of a sequence of labels, the less confidence the model, and entropy (Kim et al, 2006) that can measure the uncertainty of a probability distribution. Some other criteria are a diversity measurement (Kim et al, 2006) and a density criterion (Settles and Craven, 2008).…”

“…We experimented with the following criteria: least confidence (Culotta and McCallum, 2005), normalized entropy (Kim et al, 2006), MMR (Maximal Marginal Relevance) (Kim et al, 2006), density (Settles and Craven, 2008) when using feature vectors and word embeddings, and the combination of least confidence and density criterion. Equation 8 describes the combination criterion used in our experiments.…”

Automatic acquisition of grammatical types for nouns

“…Kim et al (2006) propose using entropy as a confidence estimator in active learning in CRFs, where examples with the most uncertainty are selected for presentation to humans labelers. In practice, they approximate the entropy of the labels given the N-best labels.…”