Near-duplicate detection for web-forums

Muthmann, Klemens; Barczyñski, Wojciech M.; Brauer, Falk; Löser, Alexander

doi:10.1145/1620432.1620447

Cited by 11 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…whether the problem presented in the thread is solved or not) classification of forum threads [15]. Additionally, threading information has been shown to enhance retrieval effectiveness for post-level retrieval [7,4], thread-level retrieval [4,5], sentencelevel shallow information extraction [16], and near-duplicate thread detection [17]. Moreover, Wang and Rose [13] demonstrated that initiation-response pairs (e.g.…”

Section: Related Workmentioning

confidence: 99%

The Utility of Discourse Structure in Forum Thread Retrieval

Wang

Kim

Baldwin

2013

Information Retrieval Technology

View full text Add to dashboard Cite

Abstract. Web user forums are a valuable means for users to resolve specific information needs, both interactively for the participants and statically for users who search/browse over historical thread data. However, the complex structure of forum threads can make it difficult for users to extract relevant information. Information retrieval (IR) over forum threads is one important way to obtain useful information on questions asked by others. In this paper, we investigate the task of IR over web user forums by utilising the discourse structure of forum threads. Experimental results show that exploiting the characteristics of discourse structure of forum threads can benefit IR, when compared to previouslypublished results.

show abstract

Section: Related Workmentioning

confidence: 99%

The Utility of Discourse Structure in Forum Thread Retrieval

Wang

Kim

Baldwin

2013

Information Retrieval Technology

View full text Add to dashboard Cite

show abstract

“…Besides the approaches focused on Web pages or documents, Muthmann et al [20] proposed their work to identify threads with near-duplicate content and to group these threads in the search results. They incorporated text-based features, features based on extracted entities for products, and structure-based features to capture the near-duplicate threads.…”

Section: Related Workmentioning

confidence: 99%

“…Along with the increasing requirements, nearduplicate detection has received much attentions in recent years [24,25,11,26,20].…”

Section: Introductionmentioning

confidence: 99%

Efficient partial-duplicate detection based on sequence matching

Zhang

et al. 2010

Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

With the ever-increasing growth of the Internet, numerous copies of documents become serious problem for search engine, opinion mining and many other web applications. Since partial-duplicates only contain a small piece of text taken from other sources and most existing near-duplicate detection approaches focus on document level, partial duplicates can not be dealt with well. In this paper, we propose a novel algorithm to realize the partial-duplicate detection task. Besides the similarities between documents, our proposed algorithm can simultaneously locate the duplicated parts. The main idea is to divide the partial-duplicate detection task into two subtasks: sentence level near-duplicate detection and sequence matching. For evaluation, we compare the proposed method with other approaches on both English and Chinese web collections. Experimental results appear to support that our proposed method is effectively and efficiently to detect both partial-duplicates on large web collections.

show abstract

“…However, finding a solution to a specific problem is not as easy as it might seem. In fact, it has been estimated that many of the forum questions have already been answered at the time they are posted, but were not found [41].…”

Section: Searching In Developer Networkmentioning

confidence: 99%

Graph-based concept identification and disambiguation for enterprise search

Brauer

Huber²,

Hackenbroich

et al. 2010

Proceedings of the 19th International Conference on World Wide Web

Self Cite

View full text Add to dashboard Cite

Enterprise Search (ES) is different from traditional IR due to a number of reasons, among which the high level of ambiguity of terms in queries and documents and existence of graph-structured enterprise data (ontologies) that describe the concepts of interest and their relationships to each other, are the most important ones.Our method identifies concepts from the enterprise ontology in the query and corpus. We propose a ranking scheme for ontology sub-graphs on top of approximately matched token q-grams. The ranking leverages the graph-structure of the ontology to incorporate not explicitly mentioned concepts. It improves previous solutions by using a fine-grained ranking function that is specifically designed to cope with high levels of ambiguity. This method is able to capture much more of the semantics of queries and documents than previous techniques. We prove this claim by an evaluation of our method in three real-life scenarios from two different domains, and found it to consistently be superior both in terms of precision and recall.

show abstract

Near-duplicate detection for web-forums

Cited by 11 publications

References 9 publications

The Utility of Discourse Structure in Forum Thread Retrieval

The Utility of Discourse Structure in Forum Thread Retrieval

Efficient partial-duplicate detection based on sequence matching

Graph-based concept identification and disambiguation for enterprise search

Contact Info

Product

Resources

About