Memory-Based Language Processing

Abstract

Machine learning has become the predominant problem-solving strategy for computational linguistics problems in the last decade. Many researchers work on improving algorithms, developing new ones, testing feature representation issues, and so forth. Other researchers, however, apply machine-learning techniques as off-the-shelf implementation, often with little knowledge about the algorithms and intricacies of data representation issues. In this book, Daelemans and van den Bosch provide an indepth introduction to Memory-Based Language Processing (MBLP) that shows for different problems in NLP how the technique is successfully applied. Apart from the more practical issues, the book also explores the suitability of the chosen learning paradigm, memory-based learning (Stanfill and Waltz 1986), for NLP problems. Thus the book is a valuable source of information for a wide range of readers from the linguist interested in applying machine-learning techniques or the machine-learning specialist with no prior experience in NLP to the expert in machine learning wanting to learn more about the appropriateness of the MBLP bias for NLP problems.Memory-based learning is a machine-learning method based on the idea that examples can be re-used directly in processing natural language problems. Training examples are stored without modification or abstraction. During the classification process, the most similar examples from the training data are located, and their class is used to classify the new example.The book addresses different levels of understanding and working with MBLP: On one level, it explains the theoretical concepts of memory-based learning; on another, it provides more practical information: The implementation of memory-based learning, TiMBL, is described as well as different extensions such as FamBL and MBT. On a different level, the application of these techniques is described for typical problems in natural language processing. The reader learns how to model standard classification problems such as POS tagging, as well as sequence-learning problems, which are more difficult to model as classification problems. Daelemans and van den Bosch also cover critical issues, such as problems that arise in the evaluation of such experiments and the automation of searching for suitable system parameter settings. On a more abstract level, they approach the question of how suitable the bias of MBLP is. In chapter 6, they compare memory-based learning as an instance of lazy learning to an instance of eager learning, rule induction, with regard to their classification accuracy if, for example, more abstraction is introduced. Since MBLP does not abstract over the training data, it is called a lazy learning approach. Rule induction, in contrast, learns rules and does not go back to the actual training data during classification.

Automatic semantic role labelling using a memory-based learning system

“…L'aprenentatge basat en memòria, memory-based learning (MBL) (Daelemans et al, 2005), és un mètode simbòlic d'aprenentatge automàtic que es basa en la idea que el comportament intel⋅ligent pot ser el resultat d'establir analogies, en comptes de ser el resultat d'aplicar un con�unt de regles abstractes. En aquest sentit contrasta amb el processament basat en regles.…”

“…Segons Daelemans et al (2005), l'aprenentatge automàtic és fonamentalment un paradigma de classificació: si tenim una representació d'un input en termes de parells atribut-valor, és a dir, un vector d'atributs, el sistema ha de generar una etiqueta de classe. Quan l'aprenentatge és supervisat aquesta etiqueta es pren d'un con�unt d'etiquetes conegut a priori.…”

“…La diferència fonamental entre aquest sistema i els sistemes esmentats anteriorment és que es basa en sintaxi de dependències, en comptes de basar-se en sintaxi de constituents. La frase (1) es representaria en sintaxi de constituents com es mostra a (6), i en sintaxi de dependències, com es mostra a (7): (Daelemans et al, 2005), desenvolupat pels grups d'investigació ILK, [www1] de la Universitat de Tilburg, i CNTS, [www2] de la Universitat d'Anvers. L'article s'organitza de la manera següent: a la secció 1 presentem el corpus Cast3LB-CoNLL-SemRol, amb el qual hem entrenat el sistema; a la secció 2 introduïm el mètode d'aprenentatge basat en memòria; a la secció 3 descrivim el sistema d'EARS, els resultats del qual apareixen a la secció �; i finalment, a la secció 5 apuntem algunes conclusions.…”

A K-Means-Based Multi-Prototype High-Speed Learning System with FPGA-Implemented Coprocessor for 1-NN Searching

“…Which is better depends on what the application is and what kind of feature extraction technique is used [42]. The NN-based learning algorithms have been widely used in pattern recognition [43]- [45], text categorization [46], [47], ranking models [48], event tracking [49], and object recognition [50].…”

The Classification of the Applicable Machine Learning Methods in Robot Manipulators

“…One advantage that instance-based learning has over other methods of machine learning is its ability to adapt its model to previously unseen data. Where other methods generally require the entire set of training data to be re-examined when one instance is changed, instance-based learners may simply store a new instance or throw an old instance away [32]. A simple example of an instance-based learning algorithm is the k-nearest neighbor algorithm.…”