Scalable entity-based summarization of web search results using MapReduce

Kitsos, Ioannis; Magoutis, Kostas; Tzitzikas, Yannis

doi:10.1007/s10619-013-7133-7

Cited by 19 publications

(13 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Researchers adopted different techniques to approach the problem, mainly focusing on the way data are displayed [13] and on how significant aspects are detected [14], [16]. While similar in the philosophical attitude of keeping an open mind, EC systems have the different objective of supporting exploration as a process, which is not the aim of Exploratory Data Analysis.…”

Section: Related Workmentioning

confidence: 99%

Exploratory computing: a draft Manifesto

Blas

Mazuran

Puppa

et al. 2014

2014 International Conference on Data Science and Advanced Analytics (DSAA)

View full text Add to dashboard Cite

The advent of the Big Data challenge has stimulated research on methods and techniques to deal with the problem of managing data abundance. Many approaches have been developed, but for the most part, they attack one specific side of the problem: e.g. efficient querying, analysis techniques that summarize data or reduce its dimensionality, data visualization, etc. The approach proposed in this paper aims instead at taking a comprehensive view: first of all, it takes into account that human exploration is an iterative and multi-step process and therefore allows building upon a previous query on to the next, in a sort of "dialogue" between the user and the system. Second, it aims at supporting a variety of user experiences, like investigation, inspiration seeking, monitoring, comparison, decision-making, research, etc. Third, and probably most important, it adds to the notion of "big" the notion of "rich": Exploratory Computing (EC) aims at dealing with datasets of semantically complex items, whose inspection may reach beyond the user's previous knowledge or expectations: an exploratory experience basically consists in creating, refining, modifying, comparing various datasets in order to "make sense" of these meanings.

show abstract

Section: Related Workmentioning

confidence: 99%

Exploratory computing: a draft Manifesto

Blas

Mazuran

Puppa

et al. 2014

2014 International Conference on Data Science and Advanced Analytics (DSAA)

View full text Add to dashboard Cite

show abstract

“…Currently, there are disk-resident approaches [29][30][31] founded on the popular Hadoop Map-Reduce [32] programming paradigm, as well as main-memory and stream-oriented processing approaches of big data, founded on frameworks such as Apache Storm, Apache Spark and Stratosphere.…”

Section: Big Datamentioning

confidence: 99%

Managing big data experiments on smartphones

Larkou

Mintzis

Andreou

et al. 2014

Distrib Parallel Databases

View full text Add to dashboard Cite

The explosive number of smartphones with ever growing sensing and computing capabilities have brought a paradigm shift to many traditional domains of the computing field. Re-programming smartphones and instrumenting them for application testing and data gathering at scale is currently a tedious and time-consuming process that poses significant logistical challenges. Next generation smartphone applications are expected to be much larger-scale and complex, demanding that these undergo evaluation and testing under different real-world datasets, devices and conditions. In this paper, we present an architecture for managing such large-scale data management experiments on real smartphones. We particularly present the building blocks of our architecture that encompassed smartphone sensor data collected by the crowd and organized in our big data repository. The given datasets can then be replayed on our testbed comprising of real and simulated smartphones accessible to developers through a web-based interface. We present the applicability of our architecture through a case study that involves the evaluation of individual components that are part of a complex Indoor Positioning System for smartphones, coined Anyplace, which we have developed over the years. The given study shows how our architecture allows us to derive novel insights into the performance of our algorithms and applications, by simplifying the management of large-scale data on smartphones.

show abstract

“…Use local module method to do the community detection referring to the diagram. Calculate the comprehensive eigenvalues of every node using formulas (1) and (2). Make the node with the largest comprehensive eigenvalue to be the initial node.…”

Section: The Implementation Process Of the Algorithmmentioning

confidence: 99%

“…Each node can be denoted as = { , , , , }; here is the node number, is the node degree, is the node clustering coefficient, is the node comprehensive eigenvalue, and is the mark number of the community the node belongs to. Calculate the comprehensive eigenvalues of the nodes using formulas (1) and (2). Map the information of all the nodes to the chain table.…”

Section: The Implementation Process Of the Algorithmmentioning

confidence: 99%

“…On the basis of hash functions' characteristic of the compressing texts corresponding with original data, we use hash functions [1] to compress the web data and ensure the reliability of data. According to the thought of data deduplication with MapReduce model [2][3][4], we use MapReduce model to cut the repeating data produced by different distributed hosts and to integrate the rest and finally we can get the no-repeat data. We still integrate community detection thought into storage structure division strategy.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DSMC: A Novel Distributed Store-Retrieve Approach of Internet Data Using MapReduce Model and Community Detection in Big Data

Zhao

et al. 2014

International Journal of Distributed Sensor Networks

View full text Add to dashboard Cite

The processing of big data is a hotspot in the scientific research. Data on the Internet is very large and also very important for the scientific researchers, so the capture and store of Internet data is a priority among priorities. The traditional single-host web spider and data store approaches have some problems such as low efficiency and large memory requirement, so this paper proposes a big data store-retrieve approach DSMC (distributed store-retrieve approach using MapReduce model and community detection) based on distributed processing. Firstly, the distributed capture method using MapReduce to deduplicate big data is presented. Secondly, the storage optimization method is put forward; it uses the hash functions with light-weight characteristics and the community detection to address the storage structure and solve the data retrieval problems. DSMC has achieved the high performance of large web data comparison and storage and gets the efficient data retrieval at the same time. The experimental results show that, in the Cloudsim platform, comparing with the traditional web spider, the proposed DSMC approach shows better efficiency and performance.

show abstract

Scalable entity-based summarization of web search results using MapReduce

Cited by 19 publications

References 39 publications

Exploratory computing: a draft Manifesto

Exploratory computing: a draft Manifesto

Managing big data experiments on smartphones

DSMC: A Novel Distributed Store-Retrieve Approach of Internet Data Using MapReduce Model and Community Detection in Big Data

Contact Info

Product

Resources

About