Architecture of a Data Portal for Publishing and Delivering Open Data for Atmospheric Measurement

Abstract

Atmospheric data are collected by researchers every day. Campaigns such as GOAmazon 2014/2015 and the Amazon Tall Tower Observatory collect essential data on aerosols, gases, cloud properties, and meteorological parameters in the Brazilian Amazon basin. These data products provide insights and essential information for analyzing and predicting natural processes. However, in Brazil, it is estimated that more than 80% of the scientific data collected are not published due to the lack of web portals that collect and store these data. This makes it difficult, or even impossible, to access and integrate the data, which can result in the loss of significant amounts of information and significantly affect the understanding of the overall data. To address this problem, we propose a data portal architecture and open data deployment that enable Big Data processing, human interaction, and download-oriented approaches with tools that help users catalog, publish and visualize atmospheric data. Thus, we describe the architecture developed, based on the experience of the Atmospheric Radiation Measurement Data Center, which incorporates the principles of FAIR, the infrastructure and content management system for managing scientific data. The portal partial results were tested with environmental data from contaminated areas at the University of São Paulo. Overall, this data portal creates more shared knowledge about atmospheric processes by providing users with access to open environmental data.

Advanced Examination of User Behavior Recognition via Log Dataset Analysis of Web Applications Using Data Mining Techniques

“…They delve into broader data mining, knowledge discovery in databases, and the application of advanced data mining techniques, such as the pincer-search-based approach, for an in-depth understanding of consumer behavior [3,5]. Some studies explore different aspects of data management and processing, such as the development of a new data portal architecture to facilitate the processing and interaction with large datasets [16]. Others employ data mining techniques to expedite the selection process for scholarship recipients using clustering and classification algorithms [17].…”