The Brazilian Virtual Observatory - A New Paradigm for Astronomy

Carvalho, R.

doi:10.6062/jcis.2010.01.03.0022

Cited by 3 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It will also be necessary to explore different technologies of each used storage system, such as Amazon S3, MariaDB, and TimescaleDB, to efficiently extract, transform, and load data. This approach will allow the creation of a flexible and adaptable environment, capable of handling the large volume of data characteristic of astronomy from multiple sources (Kelleher & Tierney, 2018;Allen et al, 2019;Carvalho et al, 2010;Bartos et al, 2017;Rosa, R. R. 2020).…”

Section: The Prototypementioning

confidence: 99%

A Data Lake Prototype Systemfor Multiwavelength (Mwa) Andmultimessenger Astronomy (Mma)

2023

Proceedings of the 16 Th International Conference on ICT, Society and Human Beings (ICT 2023), the 15 Th International Conferen

View full text Add to dashboard Cite

At the beginning of the 21st century, large telescopes are coming into operation, making astronomy a Big Data science like few others. Recently, given this scenario, applied computing to astronomy and space sciences has pointed out major challenges involving data science and data engineering. Faced with such a challenge, this article presents a first prototype of a data lake as a strategic solution for heterogeneous data astronomy such as multi-wavelength astronomy (MWA), also including multi-messenger astronomy (MMA). The proposed prototype presents a three-layer architecture, composed of storage, processing and application layers. The storage layer incorporates S3 for raw data storage, MariaDB for simple data storage (crud) and TimescaleDB for complex data storage. The processing layer includes Laravel for CRUD operations and Python for handling large volumes of data. The application layer uses ReactJS with the Cubes.dev library for analytical data visualization. The solution proposes to deliver, when necessary, agility and concatenation between heterogeneous data (light curve, spectra and images) coming from different silos (instruments and telescopes). Thus, the solution, within the MWA and MMA paradigms, incorporates data cube technology and can be customized for applications other than astronomy, such as space weather monitoring and debriefing in the context of space medicine.

show abstract

Section: The Prototypementioning

confidence: 99%

A Data Lake Prototype Systemfor Multiwavelength (Mwa) Andmultimessenger Astronomy (Mma)

2023

Proceedings of the 16 Th International Conference on ICT, Society and Human Beings (ICT 2023), the 15 Th International Conferen

View full text Add to dashboard Cite

show abstract

“…framework may be thought of as a template for all the Three-Tier scheme we use to find in litterature when dealing with user -database interaction [10]. In Section 2 we discuss two cases of photometric data coming from two different projects and the way we have pictured the schematic files to control the unique routine of storing information into databases: the SPIDER project [11] and the 2DPHOT-UKIDSS [5] project database.…”

Section: Introductionmentioning

confidence: 99%

Organizing Data in Astronomy: in the Way for a Standardized 3 Procedure

Kohl¹,

Carvalho²

2015

JCIS

View full text Add to dashboard Cite

We discuss the world wide fast increasing mass of astronomical data and the importance of taking in account these properly storing and retrieving data procedures, since data is the most important component in the science context, understood as an empirical investigative process. A scheme of a procedure of systematically storing data is presented. We understand the need for organizing data in order to make the interaction astronomer -database easier. We introduce the framework of a universal program for storing data in a structured database, independently of the data type, provided they are in ASCII format. Guidelines to interact with this program is presented.

show abstract