Generating polystore ingestion plans — A demonstration with the AWESOME system

Dasgupta, Subhasis; McKay, Charles W.; Gupta, Amarnath

doi:10.1109/bigdata.2017.8258297

Cited by 3 publications

(2 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work sits in the middle of graph analytics research underlying many data science applications [1], [11], [14] who use small data sets, and graph database research that promotes in-database graph analytics [12] who do not consider streaming input. We view the graph stream ingestion problem discussed in this paper as a component of optimized ingestion control in the AWESOME polystore system [5], [6] where multiple streams of heterogeneous data can flow into a component DBMS managed under the polystore. We expect that the general idea of using buffer control, data compression and resource monitoring for DBMS can be effectively applied.…”

Section: Discussionmentioning

confidence: 99%

Ingesting High-Velocity Streaming Graphs from Social Media Sources

Dasgupta¹,

Bagchi²,

Gupta³

2019

Preprint

Self Cite

View full text Add to dashboard Cite

Many data science applications like social network analysis use graphs as their primary form of data. However, acquiring graph-structured data from social media presents some interesting challenges. The first challenge is the high data velocity and bursty nature of the social media data. The second challenge is that the complex nature of the data makes the ingestion process expensive. If we want to store the streaming graph data in a graph database, we face a third challenge -the database is very often unable to sustain the ingestion of high-velocity, high-burst data. We have developed an adaptive buffering mechanism and a graph compression technique that effectively mitigates the problem. A novel aspect of our method is that the adaptive buffering algorithm uses the data rate, the data content as well as the CPU resources of the database machine to determine an optimal data ingestion mechanism. We further show that an ingestion-time graph-compression strategy improves the efficiency of the data ingestion into the database. We have verified the efficacy of our ingestion optimization strategy through extensive experiments.

show abstract

Section: Discussionmentioning

confidence: 99%

Ingesting High-Velocity Streaming Graphs from Social Media Sources

Dasgupta¹,

Bagchi²,

Gupta³

2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Social Media data is considered heterogeneous because it can be segmented to subsets of data, where each subset may belong to a different data model. In the case of BOUTIQUE , which builds on the AWESOME polystore platform [4], [5] where the social media is represented as a combination of relational, graph and text data where each type of data can be temporal. 2) It should enable the analyst to start with a simple operation like a keyword search (e.g., which users have at least k tweets on HIV or AIDS) and data browsing (e.g., sample tweets of the above group), but then then progressively deepens the enquiry through a series of retrieval and analytics tasks before deciding on the next information seeking step.…”

Section: Introductionmentioning

confidence: 99%

Multi-model Investigative Exploration of Social Media Data with BOUTIQUE: A Case Study in Public Health

Guo

Dasgupta

Gupta

2019

2019 15th International Conference on eScience (eScience)

Self Cite

View full text Add to dashboard Cite

We present our experience with a data science problem in Public Health, where researchers use social media (Twitter) to determine whether the public shows awareness of HIV prevention measures offered by Public Health campaigns. To help the researcher, we develop a investigative exploration system called BOUTIQUE that allows a user to perform a multistep visualization and exploration of data through a dashboard interface. Unique features of BOUTIQUE includes its ability to handle heterogeneous types of data provided by a polystore, and its ability to use computation as part of the investigative exploration process. In this paper, we present the design of the BOUTIQUE middleware and walk through an investigation process for a real-life problem.

show abstract