The Astrolabe Project: Identifying and Curating Astronomical ‘Dark Data’ through Development of Cyberinfrastructure Resources

Abstract

As research datasets and analyses grow in complexity, data that could be valuable to other researchers and to support the integrity of published work remain uncurated across disciplines. These data are especially concentrated in the "Long Tail" of funded research, where curation resources and related expertise are often inaccessible. In the domain of astronomy, it is undisputed that uncurated "dark data" exist, but the scope of the problem remains uncertain. The "Astrolabe" Project is a collaboration between University of Arizona researchers, the CyVerse cyberinfrastructure environment, and American Astronomical Society, with a mission to identify and ingest previously-uncurated astronomical data, and to provide a robust computational environment for analysis and sharing of data, as well as services for authors wishing to deposit data associated with publications. Following expert feedback obtained through two workshops held in 2015 and 2016, Astrolabe is funded in part by National Science Foundation. The system is being actively developed within CyVerse, and Astrolabe collaborators are soliciting heterogeneous datasets and potential users for the prototype system. Astrolabe team members are currently working to characterize the properties of uncurated astronomical data, and to develop automated methods for locating potentially-useful data to be targeted for ingest into Astrolabe, while cultivating a user community for the new data management system. *

Dark Data as the New Challenge for Big Data Science and the Introduction of the Scientific Data Officer

“…In a recent paper, Heidorn returns to discussing dark data by showing how small research projects in astronomy produce dark data due to insufficient funding for research data management tasks (Heidorn et al 2018). Admittedly, in this article, Heidorn et al acknowledge dark data to also be found in digital format, and thus closer to our interests.…”

Dados escuros à luz do controle público

“…Acredita-se que estes dados desconhecidos estejam concentrados na chamada "Longa cauda" do governo [Stahlman, Heidorn e Steffen 2018], se caracterizando assim como dados escuros. A "Longa cauda" consiste em um número relativamente pequeno de dados que têm como uma distribuição assimétrica à direita, conforme demonstra a Figura 1 [Heidorn 2008].…”