Growing a list

Abstract

It is easy to find expert knowledge on the Internet on almost any topic, but obtaining a complete overview of a given topic is not always easy: information can be scattered across many sources and must be aggregated to be useful. We introduce a method for intelligently growing a list of relevant items, starting from a small seed of examples. Our algorithm takes advantage of the wisdom of the crowd, in the sense that there are many experts who post lists of things on the Internet. We use a collection of simple machine learning components to find these experts and aggregate their lists to produce a single complete and meaningful list. We use experiments with gold standards and open-ended experiments without gold standards to show that our method significantly outperforms the state of the art. Our method uses the ranking algorithm Bayesian Sets even when its underlying independence assumption is violated, and we provide a theoretical generalization bound to motivate its use.

Machine learning for science and society

“…The editors of this special issue have worked on both theoretical and applied topics, where the applied topics between us include criminology , crop yield prediction (Wagstaff et al 2008), the energy grid (Rudin et al 2010, healthcare (Letham et al 2013b;McCormick et al 2012), information retrieval (Letham et al 2013a), interpretable models (Letham et al 2013b;McCormick et al 2012;Ustun et al 2013), robotic space exploration (Castano et al 2007;Wagstaff and Bornstein 2009;Wagstaff et al 2013b), and scientific discovery (Wagstaff et al 2013a). In our experience, working in applied areas strongly motivates the development of algorithms and theory that can go beyond the single application domain for which they were designed.…”

Learning to Detect Patterns of Crime

“…Clustering is usually unsupervised, whereas our method is supervised. Work on (unsupervised) set expansion in information retrieval (e.g., [20,21]) is very relevant to ours. In set expansion, they (like us) start with a small seed of instances, possess a sea of unlabeled entities (webpages), most of which are not relevant, and attempt to grow members of the same set as the seed.…”

“…The algorithms for set expansion do not adapt to the set as it develops, which is important for crime pattern detection. The baseline algorithms we compare with are similar to methods like Bayesian Sets applied in the context of Growing a List [20,21] in that they use a type of inner product as the distance metric.…”

“…HAC and NN were used with three different criteria for cluster-cluster or cluster-crime similarity: Single Linkage (SL), which considers the most similar pair of crimes; Complete Linkage (CL), which considers the most dissimilar pair of crimes, and Group Average (GA), which uses the averaged pairwise similarity [24]. The incremental nearest neighbor algorithm using the S GA measure, with the weights provided by the crime analysts, becomes similar in spirit to the Bayesian Sets algorithm [20] and how it is used in information retrieval applications [21].…”

Processing Collective Knowledge from Autonomous Individuals: A Literature Review

“…They are used to create, maintain, and provide access to knowledge that has not been available before. Taking advantage of wisdom of crowd with a lot of things posted by experts existing on internet [21], authors have introduced a method for intelligently growing a list of relevant items. Authors used a collective of simple machine learning components to find these experts and aggregate their lists to produce a single complete and meaningful list.…”