Designing a Data Management Pipeline for Pervasive Sensor Communication Systems

Ronkainen, Jussi; Iivari, Antti

doi:10.1016/j.procs.2015.07.193

Cited by 10 publications

(6 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Geotechnical monitoring systems tend to be comprised of groups of sensors in thermometric wells along the routes of the pipelines; the sensors are linked with wired or wireless technologies, and measurements are obtained using web-based software. Modern approaches to the design of distributed systems are listed in [2][3][4]. In [5] is noted that the quality of an open atmospheric optical channel is affected by the frequency of weather changes, fluctuations of supports, scintillation.…”

Section: Figure 1 Fields Of Research Where Remote Control Of Geotechnical Features Is Applicablementioning

confidence: 99%

Time Series Analysis of Geotechnical Data From a Sensor Network Controlling the Remoted Pipeline

Ay¹

2021

Preprint

View full text Add to dashboard Cite

Extensive, but remote oil and gas fields of the United States, Canada, and Russia require the construction and operation of extremely long pipelines. Global warming and local heating effects lead to rising soil temperatures and thus a reduction in the sub-grade capacity of the soils; this causes changes in the spatial positions and forms of the pipelines, consequently increasing the number of accidents. Oil operators are compelled to monitor the soil temperature along the routes of the remoted pipelines in order to be able to perform remedial measures in time. They are therefore seeking methods for the analysis of volumetric diagnostic information. To forecast soil temperatures at the different depths we propose compiling a multidimensional dataset, defining descriptive statistics; selecting uncorrelated time series; generating synthetic features; robust scaling temperature series, tuning the additive regression model to forecast soil temperatures.

show abstract

Section: Figure 1 Fields Of Research Where Remote Control Of Geotechnical Features Is Applicablementioning

confidence: 99%

Time Series Analysis of Geotechnical Data From a Sensor Network Controlling the Remoted Pipeline

Ay¹

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Several techniques and methods have been proposed illustrating the use of MQTT for data acquisition from different data sources via publish/subscribe model [23,24,25]. MQTT brokers can act as data carriers and can store data until subscribers consume it.…”

Section: Related Workmentioning

confidence: 99%

Serverless data pipeline approaches for IoT data in fog and cloud computing

Poojara¹,

Dehury²,

Jakovits³

et al. 2021

Preprint

View full text Add to dashboard Cite

With the increasing number of Internet of Things (IoT) devices, massive amounts of raw data is being generated. The latency, cost, and other challenges in cloud-based IoT data processing have driven the adoption of Edge and Fog computing models, where some data processing tasks are moved closer to data sources. Properly dealing with the flow of such data requires building data pipelines, to control the complete life cycle of data streams from data acquisition at the data source, edge and fog processing, to Cloud side storage and analytics. Data analytics tasks need to be executed dynamically at different distances from the data sources and often on very heterogeneous hardware devices. This can be streamlined by the use of a Serverless (or FaaS) cloud computing model, where tasks are defined as virtual functions, which can be migrated from edge to cloud (and vice versa) and executed in an eventdriven manner on data streams. In this work, we investigate the benefits of building Serverless data pipelines (SDP) for IoT data analytics and evaluate three different approaches for designing SDPs: 1) Off-the-shelf data flow tool (DFT) based, 2) Object storage service (OSS) based and 3) MQTT based. Further, we applied these strategies on three fog applications (Aeneas, PocketSphinx, and custom Video processing application) and evaluated the performance by comparing their processing time (computation time, network communication and disk access time), and resource utilization. Results show that DFT is unsuitable for compute-intensive applications such as video or image processing, whereas OSS is best suitable for this task. However, DFT is nicely fit for bandwidthintensive applications due to the minimum use of network resources. On the other hand, MQTT-based SDP is observed with increase in CPU and Memory usage as the number of users rose, and experienced a drop in data units in the pipeline for PocketSphinx and custom video processing applications, however it performed well for Aeneas which had low size data units.

show abstract

“…W ITH industry sectors such as economics [1], biomedical healthcare [2], [3], or energy [4] increasingly implementing intelligent sensor devices into their process structures, sensor-based applications lead to a growing availability of informative data [5], [6]. For example, a single Boeing 787 flight can generate up to 500GB of unprocessed sensor data [7]. The data captured by these sensors are characterized as time-series data, representing sequences of equally spaced time-stamped data points that are ordered chronologically [8].…”

Section: Introductionmentioning

confidence: 99%

Benchmarking Transfer Learning Strategies in Time-Series Imaging: Recommendations for Analyzing Raw Sensor Data

2022

View full text Add to dashboard Cite

With the growing availability and complexity of time-series sequences, scalable and robust machine learning approaches are required that overcome the sampling challenge of quantitatively sufficient training data. Following the research trend towards the deep learning-based analysis of time-series encoded as images, this study proposes a time-series imaging workflow that overcomes the challenge of quantitatively limited sensor data across domains (i.e., medicine and engineering). After systematically identifying the three relevant dimensions that affect the performance of the deep learning-based analysis of visualized timeseries data, we performed a benchmarking evaluation with a total of 24 unique convolutional neural network models. Following a two-level transfer learning investigation, we reveal that fine-tuning the mid-level features results in the best classification performance. As a result, we present an optimized representation of the VGG16 network, which outperforms previous studies in the field. Our approach is accurate, robust, and manifests internal and external validity. By only using the raw time-series data, our model does not require manual feature engineering, being of high practical relevance. As the post-hoc analysis of our results reveals that our model allows automated extraction of meaningful features based on the trend of the underlying timeseries data, our study also adds to explainable artificial intelligence. Furthermore, our proposed workflow reduces the sequence length of the input data while preserving all information. Especially with the hurdle of long-term dependencies in sequential time-series data, we overcome related work's limitation of the vanishing gradients problem and contribute to the sequential learning theory in artificial intelligence.

show abstract

Designing a Data Management Pipeline for Pervasive Sensor Communication Systems

Cited by 10 publications

References 8 publications

Time Series Analysis of Geotechnical Data From a Sensor Network Controlling the Remoted Pipeline

Time Series Analysis of Geotechnical Data From a Sensor Network Controlling the Remoted Pipeline

Serverless data pipeline approaches for IoT data in fog and cloud computing

Benchmarking Transfer Learning Strategies in Time-Series Imaging: Recommendations for Analyzing Raw Sensor Data

Contact Info

Product

Resources

About