Scaling to very very large corpora for natural language disambiguation

Banko, Michele; Brill, Eric

doi:10.3115/1073012.1073017

Cited by 457 publications

(341 citation statements)

References 18 publications

(11 reference statements)

Supporting

Mentioning

302

Contrasting

Unclassified

Order By: Relevance

“…An important future direction lies in expanding the corpus. Increasing the amount of data can be beneficial for machine learning algorithms (Banko and Brill 2001). Therefore, we should expand the corpus in terms of size which could be done via focused search (as for LWGC-B) or by annotating random web pages (as for LWGC-R).…”

Section: Discussionmentioning

confidence: 99%

Crowdsourcing for web genre annotation

Asheghi

Sharoff

Markert

2016

Lang Resources & Evaluation

View full text Add to dashboard Cite

Recently, genre collection and automatic genre identification for the web has attracted much attention. However, currently there is no genre-annotated corpus of web pages where inter-annotator reliability has been established, i.e. the corpora are either not tested for inter-annotator reliability or exhibit low inter-coder agreement. Annotation has also mostly been carried out by a small number of experts, leading to concerns with regard to scalability of these annotation efforts and transferability of the schemes to annotators outside these small expert groups. In this paper, we tackle these problems by using crowd-sourcing for genre annotation, leading to the Leeds Web Genre Corpus-the first web corpus which is, demonstrably reliably annotated for genre and which can be easily and cost-effectively expanded using naive annotators. We also show that the corpus is source and topic diverse.

show abstract

Section: Discussionmentioning

confidence: 99%

Crowdsourcing for web genre annotation

Asheghi

Sharoff

Markert

2016

Lang Resources & Evaluation

View full text Add to dashboard Cite

show abstract

“…Do other types of anomaly detectors, or more generally, learning algorithms for other data mining tasks also exhibit the gravity-defiant behaviour? In a complex domain such as natural language processing, millions of additional data has been shown to continue to improve the performance of trained models (Halevy et al 2009;Banko and Brill 2001). Is this the domain for which algorithms always comply with the learning curve?…”

Section: Implications and Potential Future Workmentioning

confidence: 99%

Defying the gravity of learning curve: a characteristic of nearest neighbour anomaly detectors

et al. 2016

View full text Add to dashboard Cite

Conventional wisdom in machine learning says that all algorithms are expected to follow the trajectory of a learning curve which is often colloquially referred to as 'more data the better'. We call this 'the gravity of learning curve', and it is assumed that no learning algorithms are 'gravity-defiant'. Contrary to the conventional wisdom, this paper provides the theoretical analysis and the empirical evidence that nearest neighbour anomaly detectors are gravity-defiant algorithms.

show abstract

“…Earlier work in ESL error correction follows the methodology of the context-sensitive spelling correction task (Golding and Roth, 1996;Golding and Roth, 1999;Banko and Brill, 2001;Carlson et al, 2001;Carlson and Fette, 2007). Most of the effort in ESL error correction so far has been on article and preposition usage errors, as these are some of the most common mistakes among non-native English speakers (Dalgish, 1985;Leacock et al, 2010).…”

Section: Related Workmentioning

confidence: 99%

Correcting Grammatical Verb Errors

Rozovskaya

Roth

Srikumar

2014

Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics

View full text Add to dashboard Cite

Verb errors are some of the most common mistakes made by non-native writers of English but some of the least studied. The reason is that dealing with verb errors requires a new paradigm; essentially all research done on correcting grammatical errors assumes a closed set of triggers -e.g., correcting the use of prepositions or articles -but identifying mistakes in verbs necessitates identifying potentially ambiguous triggers first, and then determining the type of mistake made and correcting it. Moreover, once the verb is identified, modeling verb errors is challenging because verbs fulfill many grammatical functions, resulting in a variety of mistakes. Consequently, the little earlier work done on verb errors assumed that the error type is known in advance.We propose a linguistically-motivated approach to verb error correction that makes use of the notion of verb finiteness to identify triggers and types of mistakes, before using a statistical machine learning approach to correct these mistakes. We show that the linguistically-informed model significantly improves the accuracy of the verb correction approach.

show abstract

Scaling to very very large corpora for natural language disambiguation

Cited by 457 publications

References 18 publications

Crowdsourcing for web genre annotation

Crowdsourcing for web genre annotation

Defying the gravity of learning curve: a characteristic of nearest neighbour anomaly detectors

Correcting Grammatical Verb Errors

Contact Info

Product

Resources

About