Challenges for Cluster Analysis in a Virtual Observatory

Djorgovski, S. G.; Brunner, Róbert; Mahabal, A.; Williams, R. D.; Granat, Robert; Stolorz, Paul

doi:10.1007/0-387-21529-8_9

Cited by 9 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are many problems in astronomy that metaheuristic optimization algorithms are applicable to. For example, large scale clustering problems can be approached using metaheuristic algo-rithms [12] [26]. In recent years, these metaheuristic algorithms have seen widespread use, see, for example, [8], [16], [41], [40], and [42].…”

Section: Searching Algorithms For Generating Efficient Lhds With Flexible Sizesmentioning

confidence: 99%

Statistical Analysis of Complex Computer Models in Astronomy

Lukemire

Xiao

Mandal

et al. 2021

Eur. Phys. J. Spec. Top.

View full text Add to dashboard Cite

We introduce statistical techniques required to handle complex computer models with potential applications to astronomy. Computer experiments play a critical role in almost all fields of scientific research and engineering. These computer experiments, or simulators, are often computationally expensive, leading to the use of emulators for rapidly approximating the outcome of the experiment. Gaussian process models, also known as Kriging, are the most common choice of emulator. While emulators offer significant improvements in computation over computer simulators, they require a selection of inputs along with the corresponding outputs of the computer experiment to function well. Thus, it is important to select inputs judiciously for the full computer simulation to construct an accurate emulator. Space-filling designs are efficient when the general response surface of the outcome is unknown, and thus they are a popular choice when selecting simulator inputs for building an emulator. In this tutorial we discuss how to construct these space filling designs, perform the subsequent fitting of the Gaussian process surrogates, and briefly indicate their potential applications to astronomy research.

show abstract

Section: Searching Algorithms For Generating Efficient Lhds With Flexible Sizesmentioning

confidence: 99%

Statistical Analysis of Complex Computer Models in Astronomy

Lukemire

Xiao

Mandal

et al. 2021

Eur. Phys. J. Spec. Top.

View full text Add to dashboard Cite

show abstract

“…The magnitude of the computational challenge for pattern recognition and classification algorithms is suggested by the fact that the VO will contain billions of data points each having hundreds of dimensions [49].…”

Section: Complex Massive Data Setsmentioning

confidence: 99%

The Data Big Bang and the Expanding Digital Universe: High‐Dimensional, Complex and Massive Data Sets in an Inflationary Epoch

Pesenson

McCollum

2010

Advances in Astronomy

View full text Add to dashboard Cite

Recent and forthcoming advances in instrumentation, and giant new surveys, are creating astronomical data sets that are not amenable to the methods of analysis familiar to astronomers. Traditional methods are often inadequate not merely because of the size in bytes of the data sets, but also because of the complexity of modern data sets. Mathematical limitations of familiar algorithms and techniques in dealing with such data sets create a critical need for new paradigms for the representation, analysis and scientific visualization (as opposed to illustrative visualization) of heterogeneous, multiresolution data across application domains. Some of the problems presented by the new data sets have been addressed by other disciplines such as applied mathematics, statistics and machine learning and have been utilized by other sciences such as space-based geosciences. Unfortunately, valuable results pertaining to these problems are mostly to be found in publications outside of astronomy. Here we offer brief overviews of a number of concepts, techniques and developments that are vital to the analysis and visualization of complex datasets and images. One of the goals of this paper is to help bridge the gap between applied mathematics and artificial intelligence on the one side and astronomy on the other.

show abstract

“…We note that the same types of classification techniques are also used for more detailed explorations of large digital sky surveys and other astronomical data sets, especially in searches for outliers in some parameter space, which are often some astrophysically interesting type of objects (e.g., distant quasars) [20,21,22,23,24,37,40]. Resolved sources ("galaxies") in principle contain more morphological information, since all unresolved ones ("stars") by definition look alike.…”

Section: Star-galaxy Image Classification: the Next Generationmentioning

confidence: 99%

Some Pattern Recognition Challenges in Data-Intensive Astronomy

Djorgovski

Donalek

Mahabal

et al. 2006

18th International Conference on Pattern Recognition (ICPR'06)

Self Cite

View full text Add to dashboard Cite

We review some of the recent developments and challenges posed by the data analysis in modern digital sky surveys, which are representative of the information-rich astronomy in the context of Virtual Observatory. Illustrative examples include the problems of an automated star-galaxy classification in complex and heterogeneous panoramic imaging data sets, and an automated, iterative, dynamical classification of transient events detected in synoptic sky surveys. These problems offer good opportunities for productive collaborations between astronomers and applied computer scientists and statisticians, and are representative of the kind of challenges now present in all data-intensive fields. We discuss briefly some emergent types of scalable scientific data analysis systems with a broad applicability.

show abstract

Challenges for Cluster Analysis in a Virtual Observatory

Cited by 9 publications

References 9 publications

Statistical Analysis of Complex Computer Models in Astronomy

Statistical Analysis of Complex Computer Models in Astronomy

The Data Big Bang and the Expanding Digital Universe: High‐Dimensional, Complex and Massive Data Sets in an Inflationary Epoch

Some Pattern Recognition Challenges in Data-Intensive Astronomy

Contact Info

Product

Resources

About