CasJobs and MyDB: A Batch Query Workbench

Li, N.; Thakar, Ani

doi:10.1109/mcse.2008.6

Cited by 60 publications

(43 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…An automated photometric pipeline performed object classification to a magnitude of r ≈22 and measured photometric properties of more than 100 million galaxies. The complete data sample, and many derived catalogs such as the photometric redshift estimates, are publicly available through the CasJobs server (Li & Thakar 2008) 1 . The SDSS is well suited to the analysis presented in this paper due to the enormous number of photometrically selected galaxies with spectroscopic redshifts to use as training, cross-validation and test samples.…”

Section: Datamentioning

confidence: 99%

Feature importance for machine learning redshifts applied to SDSS galaxies

Hoyle

Rau

Zitlau

et al. 2015

Monthly Notices of the Royal Astronomical Society

View full text Add to dashboard Cite

We present an analysis of importance feature selection applied to photometric redshift estimation using the machine learning architecture Decision Trees with the ensemble learning routine Adaboost (hereafter RDF). We select a list of 85 easily measured (or derived) photometric quantities (or 'features') and spectroscopic redshifts for almost two million galaxies from the Sloan Digital Sky Survey Data Release 10. After identifying which features have the most predictive power, we use standard artificial Neural Networks (aNN) to show that the addition of these features, in combination with the standard magnitudes and colours, improves the machine learning redshift estimate by 18% and decreases the catastrophic outlier rate by 32%. We further compare the redshift estimate using RDF with those from two different aNNs, and with photometric redshifts available from the SDSS. We find that the RDF requires orders of magnitude less computation time than the aNNs to obtain a machine learning redshift while reducing both the catastrophic outlier rate by up to 43%, and the redshift error by up to 25%. When compared to the SDSS photometric redshifts, the RDF machine learning redshifts both decreases the standard deviation of residuals scaled by 1/(1+z) by 36% from 0.066 to 0.041, and decreases the fraction of catastrophic outliers by 57% from 2.32% to 0.99%.

show abstract

Section: Datamentioning

confidence: 99%

Feature importance for machine learning redshifts applied to SDSS galaxies

Hoyle

Rau

Zitlau

et al. 2015

Monthly Notices of the Royal Astronomical Society

View full text Add to dashboard Cite

show abstract

“…CASJobs (Li & Thakar 2008) gives each user their own server-side database called MyDB, along with the ability to submit arbitrarily complex SQL queries in batch mode and redirect the output to their MyDB. Users may import their own data to cross-match with the SDSS data.…”

mentioning

confidence: 99%

Sloan Digital Sky Survey IV: Mapping the Milky Way, Nearby Galaxies, and the Distant Universe

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Catalog Archive Server Jobs (CasJobs) is the core of the SciServer infrastructure, providing web-based query management and database services [5]. CasJobs excels when working with large datasets too cumbersome for personal computer-based processing; filtering and aggregations are most efficient when handled by a database designed for such operations.…”

Section: Casjobsmentioning

confidence: 99%

Data integration for materials research

Carey

Budavàri

Daphalapurkar

et al. 2016

Integr Mater Manuf Innov

View full text Add to dashboard Cite

Introduction:A new data science initiative in materials research has been launched at The Johns Hopkins University within the Materials in Extreme Dynamic Environments (MEDE) Collaborative Research Alliance (CRA). Our first goal is to build a solution that facilitates seamless data sharing among MEDE scientists. We expect to shorten the design and development cycle of new materials by providing integrated storage, database, and analysis services, building on proven components of the SciServer project developed at the Institute for Data Intensive Engineering and Science (IDIES). Case description: Here we present our system design and demonstrate the power of our approach through a use-case that enables easy comparison of simulations and measurements. This prototype effort, focusing on boron carbide (BC), brings together multiple materials research elements in the Ceramics group within the MEDE CRA. Discussion and evaluation: The SciServer platform offers single-sign on access to various general purpose data analysis tools familiar to materials scientists in MEDE. During the case study deployment, users appreciated the simple data file upload process, automated database ingestion, and platform applicability to both students of the art and power users. Conclusions: From our case study experience in aggregating data from both simulations and physical experiments, we developed a template workflow from which a user may run a common data comparison task outright or customize to another purpose. Next, we turn to acquiring data from more MEDE groups and expanding the user base to the Metals group.

show abstract

CasJobs and MyDB: A Batch Query Workbench

Cited by 60 publications

References 2 publications

Feature importance for machine learning redshifts applied to SDSS galaxies

Feature importance for machine learning redshifts applied to SDSS galaxies

Sloan Digital Sky Survey IV: Mapping the Milky Way, Nearby Galaxies, and the Distant Universe

Data integration for materials research

Contact Info

Product

Resources

About