Enhancing First Story Detection using Word Embeddings

Moran, Seán; McCreadie, Richard; Macdonald, Craig; Ounis, Iadh

doi:10.1145/2911451.2914719

Cited by 24 publications

(26 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This approach supports arbitrary similarity functions, but it suffers from the problem of the vocabulary mismatch [4,63,23]. The second approach involves carrying out the k-NN search via LSH [53,65,46,37]. It is most appropriate for the cosine similarity.…”

Section: Discussion and Related Workmentioning

confidence: 99%

Off the Beaten Path

Boytsov

Novák

Malkov

et al. 2016

Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

View full text Add to dashboard Cite

Retrieval pipelines commonly rely on a term-based search to obtain candidate records, which are subsequently re-ranked. Some candidates are missed by this approach, e.g., due to a vocabulary mismatch. We address this issue by replacing the term-based search with a generic k-NN retrieval algorithm, where a similarity function can take into account subtle term associations. While an exact brute-force k-NN search using this similarity function is slow, we demonstrate that an approximate algorithm can be nearly two orders of magnitude faster at the expense of only a small loss in accuracy. A retrieval pipeline using an approximate k-NN search can be more effective and efficient than the term-based pipeline. This opens up new possibilities for designing effective retrieval pipelines. Our software (including data-generating code) and derivative data based on the Stack Overflow collection is available online.1 This revision is a slightly extended version of the respective CIKM'16 paper.

show abstract

Section: Discussion and Related Workmentioning

confidence: 99%

Off the Beaten Path

Boytsov

Novák

Malkov

et al. 2016

Proceedings of the 25th ACM International on Conference on Information and Knowledge Management

View full text Add to dashboard Cite

show abstract

“…demonstrated encouraging results using the word2vec Skip-gram model to generate event timelines from tweets. Moran et al (2016) achieved an improvement over the state-of-the-art first story detection (FSD) results by expanding the tweets with their semantically related terms using word2vec.…”

Section: Neural Embeddingsmentioning

confidence: 99%

Character-based Neural Embeddings for Tweet Clustering

Vakulenko¹,

Nixon²,

Lupu³

2017

Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media

View full text Add to dashboard Cite

show abstract

“…They are only related in that event detection must be performed as a part of finding first stories. Moreover, while [19] focused on detecting first story with paraphrasing, our concern is on identifying and classifying news events that are worth presenting to the user.…”

Section: Related Workmentioning

confidence: 99%

Extracting news events from microblogs

Repp

Ramampiaro

2018

Journal of Statistics and Management Systems

View full text Add to dashboard Cite

Twitter stream has become a large source of information for many people, but the magnitude of tweets and the noisy nature of its content have made harvesting the knowledge from Twitter a challenging task for researchers for a long time. Aiming at overcoming some of the main challenges of extracting the hidden information from tweet streams, this work proposes a new approach for real-time detection of news events from the Twitter stream. We divide our approach into three steps. The first step is to use a neural network or deep learning to detect news-relevant tweets from the stream. The second step is to apply a novel streaming data clustering algorithm to the detected news tweets to form news events. The third and final step is to rank the detected events based on the size of the event clusters and growth speed of the tweet frequencies. We evaluate the proposed system on a large, publicly available corpus of annotated news events from Twitter. As part of the evaluation, we compare our approach with a related state-of-the-art solution. Overall, our experiments and user-based evaluation show that our approach on detecting current (real) news events delivers a state-of-the-art performance.

show abstract

Enhancing First Story Detection using Word Embeddings

Cited by 24 publications

References 10 publications

Off the Beaten Path

Off the Beaten Path

Character-based Neural Embeddings for Tweet Clustering

Extracting news events from microblogs

Contact Info

Product

Resources

About