Classification Algorithms for Big Data Analysis, a Map Reduce Approach

Ayma, V. A.; Ferreira, Rodrigo S.; Happ, Patrick Nigri; Oliveira, Dário Augusto Borges; Feitosa, Raul Queiroz; Costa, Gilson Alexandre Ostwald Pedro da; Plaza, Antonio; Gamba, Paolo

doi:10.5194/isprsarchives-xl-3-w2-17-2015

Cited by 33 publications

(11 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Big data analytics and machine learning have several algorithms in common [105] . More security concerns can be addressed through the use of machine learning algorithms such as device fingerprinting.…”

Section: Device Fingerprinting In Wireless Networkmentioning

confidence: 99%

Big data analytics for wireless and wired network design: A survey

et al. 2018

View full text Add to dashboard Cite

a b s t r a c tCurrently, the world is witnessing a mounting avalanche of data due to the increasing number of mobile network subscribers, Internet websites, and online services. This trend is continuing to develop in a quick and diverse manner in the form of big data. Big data analytics can process large amounts of raw data and extract useful, smaller-sized information, which can be used by different parties to make reliable decisions.In this paper, we conduct a survey on the role that big data analytics can play in the design of data communication networks. Integrating the latest advances that employ big data analytics with the networks' control/traffic layers might be the best way to build robust data communication networks with refined performance and intelligent features. First, the survey starts with the introduction of the big data basic concepts, framework, and characteristics. Second, we illustrate the main network design cycle employing big data analytics. This cycle represents the umbrella concept that unifies the surveyed topics. Third, there is a detailed review of the current academic and industrial effort s toward network design using big data analytics. Forth, we identify the challenges confronting the utilization of big data analytics in network design. Finally, we highlight several future research directions. To the best of our knowledge, this is the first survey that addresses the use of big data analytics techniques for the design of a broad range of networks.

show abstract

Section: Device Fingerprinting In Wireless Networkmentioning

confidence: 99%

Big data analytics for wireless and wired network design: A survey

et al. 2018

View full text Add to dashboard Cite

show abstract

“…In this work, the K-Means and the Expectation Maximization (EM) clustering algorithms were implemented as distributed solutions, which are able to exploit the capabilities of cloud computing infrastructure for processing large RS datasets. The solution was developed as part of the InterCloud Data Mining Package, which is a suite of distributed classification methods, previously employed in hyperspectral image analysis (Ayma, 2017) (Ayma, 2015). In this work we extended the functionalities of that package, by making it able to process multispectral images using the aforementioned clustering algorithms.…”

Section: Introductionmentioning

confidence: 99%

Mapping Glacier Changes Using Clustering Techniques on Cloud Computing Infrastructure

Ayma

Beltrán

Happ

et al. 2019

Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci.

View full text Add to dashboard Cite

Climate change and its effects are taking more importance nowadays; and glaciers are one of the most affected ecosystems by that, considering that the energy of Earth's surface and its temperature may be directly related to glacier temporal changes. Then, the comprehension of glaciers behaviour, by its retreating or melting critical conditions, can be achieved by the analysis of Remote Sensing data, but considering the unprecedented volumes of information currently provided by satellites sensors, we can refer to this analysis as a big data problem. Machine learning techniques have the potential to improve the analysis of this type of data; however, most current machine learning algorithms are unable to properly process such huge volumes of data. In the attempt to overcome the computational limitations related to Remote Sensing Big Data analysis, we implemented the K-Means and Expectation Maximization algorithms, as distributed clustering solutions, exploiting the capabilities of cloud computing infrastructure for processing very large datasets. The solution was developed over the InterCloud Data Mining Package, which is a suite of distributed classification methods, previously employed in hyperspectral image analysis. In this work we extended the functionalities of that package, by making it able to process multispectral images using the aforementioned clustering algorithms. To validate our proposal, we analysed the Ausangate glacier, located on the Andes Mountains, in Peru, by mapping the changes in such environment through a multi-temporal Remote Sensing analysis. Our results and conclusions are focused on the thematic accuracy and the computational performance achieved by our proposed solution. Thematic accuracy was assessed by comparing the automatically detected glacier areas by the clustering approaches against the manually selected ground truth data. We compared the computational load involved in executing the clustering processes sequentially and in a distributed fashion, using a local mode and cluster configuration over a cloud computing infrastructure.

show abstract

“…Porém, a versão do InterIMAGE 1.43 apresenta restrição de processamento para classificar imagens com tamanhos superiores a 9 Megapixels (InterIMAGE, 2016), em função do processamento da segmentação e classificação estarem compreendidas adicionalmente a outras grandezas como tamanho da rede semântica e número de objetos a serem processados. Esforços têm sido dispendidos para concluir o Inter-Cloud, uma plataforma open source de interpretação de imagens que suporta a criação de modelos de interpretação e execução de tais modelos na infraestrutura de computação em nuvem (Ayma et al, 2015;Antunes et al, 2016Antunes et al, , 2018a. Entretanto, os usuários precisam considerar os custos relativamente elevados de processamento em nuvem.…”

Section: Introductionunclassified

Quality Analysis of the Segmentation and Object-Based Classification of an Urban GeoEye-1 Satellite Image Performed in the PostgreSQL and InterIMAGE Platforms

et al. 2019

View full text Add to dashboard Cite

Resumo A técnica denominada de classificação baseada em objetos, proposta por Baatz & Shäpe (2000), trata-se de uma abordagem de processamento de imagens em que a unidade primitiva é o objeto, composto de vários pixels. Softwares proprietários como o eCognition® e de open source como o InterIMAGE realizam o processamento de imagens baseadas em objetos considerando o alto grau de relações mútuas e ações em diferentes escalas, como informações de contexto, estrutura semântica e hierárquica. O objetivo principal da pesquisa foi demonstrar e avaliar resultados da integração de sistemas open source com o sistema de classificação denominado InterIMAGE. Os sistemas utilizados nessa pesquisa foram o sistema de gerenciamento de banco de dados objeto-relacional PostgreSQL/PostGIS Raster, biblioteca TerraLib, pacote computacional de sistema de informações geográficas QGIS e linguagem de programação C++. Foi utilizada uma imagem do satélite GeoEye-1 de 2013 de uma área urbana do município de Goianésia no estado de Goiás. Foi desenvolvida uma interface (API-Application Programming Interface) no sistema InterIMAGE para realizar a segmentação multiresolução em ambiente de banco de dados espaciais. A segmentação processada utilizou-se da API com a imagem armazenada no PostgreSQL e em disco rígido, enquanto a classificação foi efetuada somente no InterIMAGE. O índice Kappa foi utilizado para indicação da acurácia dos resultados alcançados na classificação, utilizando-se os parâmetros da segmentação da API, obtendo-se um valor de 0,412. As regras da árvore de decisão devem ser modificadas para realização de novos experimentos visando verificar a influência no processamento da classificação no InterIMAGE. Apesar da ocorrência de algumas confusões temáticas no processo de classificação, demonstrou-se a viabilidade de continuação de desenvolvimento de aplicações de código aberto para o InterIMAGE.

show abstract

Classification Algorithms for Big Data Analysis, a Map Reduce Approach

Cited by 33 publications

References 12 publications

Big data analytics for wireless and wired network design: A survey

Big data analytics for wireless and wired network design: A survey

Mapping Glacier Changes Using Clustering Techniques on Cloud Computing Infrastructure

Quality Analysis of the Segmentation and Object-Based Classification of an Urban GeoEye-1 Satellite Image Performed in the PostgreSQL and InterIMAGE Platforms

Contact Info

Product

Resources

About