Development and Evaluation of a Big Data Framework for Performance Management in Mobile Networks

Martinez-Mosquera, Diana; Navarrete, Rosa; Luján‐Mora, Sergio

doi:10.1109/access.2020.3045175

Cited by 5 publications

(9 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, for small networks, this approach could be useful, especially for detecting malicious traffic and intrusion attempts. A big-data framework for performance management in mobile networks is proposed in [36]. The proposed framework architecture uses the HDFS file system and focuses on batch processing.…”

Section: Related Workmentioning

confidence: 99%

Big-Data Platform for Performance Monitoring of Telecom-Service-Provider Networks

2022

View full text Add to dashboard Cite

Large telecom-service-provider networks are typically based on complex communications infrastructures comprising millions of network devices. The performance monitoring of such networks is a very demanding and challenging task. A large amount of data is collected and processed during performance monitoring to obtain information that gives insights into the current network performance. Using the obtained information, providers can efficiently detect, locate, and troubleshoot weak spots in the network and improve the overall network performance. Furthermore, the extracted information can be used for planning future network expansions and to support the determination of business-strategy decisions. However, traditional methods for processing and storing data are not applicable because of the enormous amount of collected data. Thus, big-data technologies must be used. In this paper, a big-data platform for the performance monitoring of telecom-service-provider networks is proposed. The proposed platform is capable of collecting, storing, and processing data from millions of devices. Typical challenges and problems in the development and deployment process of the platform, as well as the solutions to overcome them, are presented. The proposed platform is adjusted to HFC (Hybrid Fiber-Coaxial) network and currently operates in the real HFC network, comprising millions of users and devices.

show abstract

Section: Related Workmentioning

confidence: 99%

Big-Data Platform for Performance Monitoring of Telecom-Service-Provider Networks

2022

View full text Add to dashboard Cite

show abstract

“…The mobile network sector is one of the fastest-growing industries around the world. Everybody has witnessed the rapid evolution of its technologies over the last few decades ( Martinez-Mosquera, Navarrete & Lujn-Mora, 2020 ).…”

Section: Related Conceptsmentioning

confidence: 99%

“…These constantly generate performance management data to monitor the network status close to real-time. For this reason, this large amount of data must be queried in the shortest possible time to offer an excellent service to the end user and efficiently prevent or detect outages ( Martinez-Mosquera, Navarrete & Lujn-Mora, 2020 ). These data sets are XML files composed of complex schemas with nested structures and arrays.…”

Section: Introductionmentioning

confidence: 99%

Efficient processing of complex XSD using Hive and Spark

Martinez-Mosquera

Navarrete

Luján‐Mora

2021

PeerJ Computer Science

Self Cite

View full text Add to dashboard Cite

The eXtensible Markup Language (XML) files are widely used by the industry due to their flexibility in representing numerous kinds of data. Multiple applications such as financial records, social networks, and mobile networks use complex XML schemas with nested types, contents, and/or extension bases on existing complex elements or large real-world files. A great number of these files are generated each day and this has influenced the development of Big Data tools for their parsing and reporting, such as Apache Hive and Apache Spark. For these reasons, multiple studies have proposed new techniques and evaluated the processing of XML files with Big Data systems. However, a more usual approach in such works involves the simplest XML schemas, even though, real data sets are composed of complex schemas. Therefore, to shed light on complex XML schema processing for real-life applications with Big Data tools, we present an approach that combines three techniques. This comprises three main methods for parsing XML files: cataloging, deserialization, and positional explode. For cataloging, the elements of the XML schema are mapped into root, arrays, structures, values, and attributes. Based on these elements, the deserialization and positional explode are straightforwardly implemented. To demonstrate the validity of our proposal, we develop a case study by implementing a test environment to illustrate the methods using real data sets provided from performance management of two mobile network vendors. Our main results state the validity of the proposed method for different versions of Apache Hive and Apache Spark, obtain the query execution times for Apache Hive internal and external tables and Apache Spark data frames, and compare the query performance in Apache Hive with that of Apache Spark. Another contribution made is a case study in which a novel solution is proposed for data analysis in the performance management systems of mobile networks.

show abstract

Section: Performance Management Files In Mobile Networkmentioning

confidence: 99%

“…These constantly generate performance management data to monitor the network status close to real-time. For this reason, this large amount of data must be queried in the shortest possible time to offer an excellent service to the end user and efficiently prevent or detect outages (Martinez-Mosquera et al, 2020). These data sets are XML files composed of complex schemas with nested structures and arrays.…”

Section: Introduction 32mentioning

confidence: 99%

Peer Review #1 of "Efficient processing of complex XSD using Hive and Spark (v0.1)"

2021

View full text Add to dashboard Cite

Development and Evaluation of a Big Data Framework for Performance Management in Mobile Networks

Cited by 5 publications

References 21 publications

Big-Data Platform for Performance Monitoring of Telecom-Service-Provider Networks

Big-Data Platform for Performance Monitoring of Telecom-Service-Provider Networks

Efficient processing of complex XSD using Hive and Spark

Peer Review #1 of "Efficient processing of complex XSD using Hive and Spark (v0.1)"

Contact Info

Product

Resources

About