Addressing big data issues in Scientific Data Infrastructure

Abstract

Abstract-Big Data are becoming a new technology focus both in science and in industry. This paper discusses the challenges that are imposed by Big Data on the modern and future Scientific Data Infrastructure (SDI). The paper discusses a nature and definition of Big Data that include such features as Volume, Velocity, Variety, Value and Veracity. The paper refers to different scientific communities to define requirements on data management, access control and security. The paper introduces the Scientific Data Lifecycle Management (SDLM) model that includes all the major stages and reflects specifics in data management in modern e-Science. The paper proposes the SDI generic architecture model that provides a basis for building interoperable data or project centric SDI using modern technologies and best practices. The paper explains how the proposed models SDLM and SDI can be naturally implemented using modern cloud based infrastructure services provisioning model and suggests the major infrastructure components for Big Data Infrastructure.

The Challenges of Data Quality and Data Quality Assessment in the Big Data Era

“…But now, data collected and analyzed by enterprises have surpassed this scope. Big data sources are very wide, including: 1) data sets from the internet and mobile internet (Li & Liu, 2013); 2) data from the Internet of Things; 3) data collected by various industries; 4) scientific experimental and observational data (Demchenko, Grosso & Laat, 2013), such as high-energy physics experimental data, biological data, and space observation data. These sources produce rich data types.…”

A Survey on Machine Learning-Based Mobile Big Data Analysis: Challenges and Applications

“…This section discusses the five Vs (short for volume, velocity, variety, value, and veracity) features [80] deriving from big data towards the MBD. The five Vs features have been improved in M-Internet, while it makes users access Internet anytime and anywhere [81].…”

“…The veracity of MBD includes two parts: data consistency and trustworthiness [80]. It can also be summarized as data quality.…”

A Conceptual Framework and a Suite of Tools to Support Crisis Management

“…• Velocity refers to the frequency of data production and the time required to process the data [12], [13] and [14].…”