Ethical and Responsible Use of AI/ML in the Earth, Space, and Environmental Sciences

Stall, Shelley; Cervone, Guido; Coward, Caroline; Cutcher‐Gershenfeld, Joel; Donaldson, Thomas; Erdmann, Christopher; Hanson, Brooks; Holm, Jeanne; King, John Leslie; Lyon, Laura; MacNamara, Delia Pembrey; McGovern, Amy; McGranaghan, Ryan; Narock, Ayris; Parker, Micaela S.; Peng, Ge; Rao, Yuhan; Ryan, Erin; Sedora, Brian; Shekhar, Shashi; Vrouwenvelder, Kristina T.; Waller, Lance A.; Wirz, Christopher D.; Participants, AGU AI/ML Ethics Workshop

doi:10.22541/essoar.168132856.66485758/v1

Cited by 4 publications

(3 citation statements)

References 13 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nonetheless, we argue the additional rigor, transparency, and reproducibility are well worth the effort. To this end, we have provided an actionable path forward for addressing ethical considerations and goals outlined by recent AGU work on ML ethics in ESS (Stall et al., 2023).…”

Section: Discussionmentioning

confidence: 99%

“…In this paper, we introduce quantitative content analysis (QCA) as an approach to increase the reproducibility and replicability of hand labeling for training supervised ML in ESS applications. With this approach, we provide an actionable path forward for addressing ethical considerations and goals outlined by recent AGU work on ML ethics in ESS (Stall et al., 2023). QCA is a method used in social science to systematically and objectively categorize data using a standardized set of rules, known as a “codebook,” together with assessments of reliability (Coe & Scacco, 2017).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Increasing the Reproducibility and Replicability of Supervised AI/ML in the Earth Systems Science by Leveraging Social Science Methods

Wirz,

Sutter,

Demuth

et al. 2024

Earth and Space Science

Self Cite

View full text Add to dashboard Cite

Artificial intelligence (AI) and machine learning (ML) pose a challenge for achieving science that is both reproducible and replicable. The challenge is compounded in supervised models that depend on manually labeled training data, as they introduce additional decision‐making and processes that require thorough documentation and reporting. We address these limitations by providing an approach to hand labeling training data for supervised ML that integrates quantitative content analysis (QCA)—a method from social science research. The QCA approach provides a rigorous and well‐documented hand labeling procedure to improve the replicability and reproducibility of supervised ML applications in Earth systems science (ESS), as well as the ability to evaluate them. Specifically, the approach requires (a) the articulation and documentation of the exact decision‐making process used for assigning hand labels in a “codebook” and (b) an empirical evaluation of the reliability” of the hand labelers. In this paper, we outline the contributions of QCA to the field, along with an overview of the general approach. We then provide a case study to further demonstrate how this framework has and can be applied when developing supervised ML models for applications in ESS. With this approach, we provide an actionable path forward for addressing ethical considerations and goals outlined by recent AGU work on ML ethics in ESS.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Increasing the Reproducibility and Replicability of Supervised AI/ML in the Earth Systems Science by Leveraging Social Science Methods

Wirz,

Sutter,

Demuth

et al. 2024

Earth and Space Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…As is typical with any new and possibly dissenting perspective, this emerging paradigm continues to ignite a spirited debate, delving into crucial issues such as the reproducibility and trustworthiness of outcomes derived from ML models. More broadly, the discourse encompasses considerations regarding the ethical implications associated with the widespread adoption of AI (Stall et al., 2023).…”

Section: Data‐driven Methods In Sciencementioning

confidence: 99%

Our Vision for JGR: Machine Learning and Computation

Camporeale,

Marino

2024

Journal of Geophysical Research: Machine Learning and Computati

View full text Add to dashboard Cite

This editorial introduces the inaugural issue of the Journal of Geophysical Research: Machine Learning and Computation to the scientific community, elucidating the motivations and vision behind its establishment. The landscape of computational tools for geoscientists has undergone a rapid transformation in the last decade, akin to a new scientific revolution challenging the traditional scientific method. The paradigm shift emphasizes the integration of data‐driven methods and the possibility of predicting and/or reproducing the evolution of natural phenomena with computers as the fourth pillar of scientific discovery, sparking debates on trustworthiness, and ethical implications. The data science revolution is fueled by the convergence of advancements, including the big‐data revolution, GPU market expansion, and significant investments in Artificial Intelligence and high performance computing by both institutional and private players. This transformation has given rise to a trans‐disciplinary community that has investigated a wide range of questions under the lens of machine learning (ML) approaches and has generally advanced the field of computational methods within the broader geosciences community, the core of the American Geophysical Union (AGU) membership. Responding to an unmet demand in the existing worldwide editorial offer, the Journal of Geophysical Research: Machine Learning and Computation aims to serve as an intellectual crucible, fostering collaborations across multiple geophysical disciplines and data scientists. The journal welcomes papers with strong methodological developments that allow for geoscience advancements grounded in specific computational and data‐driven methods, leveraging ML as well as innovative computational strategies, and leading to breakthrough discoveries and original scientific outcomes. Authors are encouraged to balance succinctness in introducing methods with a thorough exploration of the novelty of the work proposed and its future applications placing special emphasis on the connection between the data science approach and the scientific outcome, considering a broad readership. Emphasis on result reproducibility aligns with AGU guidance, inviting active participation from the community in shaping geophysical research in the era of machine learning and computation.

show abstract

Garbage in, garbage out: mitigating risks and maximizing benefits of AI in research

Hanson,

Stall,

Cutcher-Gershenfeld

et al. 2023

Nature

View full text Add to dashboard Cite

Ethical and Responsible Use of AI/ML in the Earth, Space, and Environmental Sciences

Cited by 4 publications

References 13 publications

Increasing the Reproducibility and Replicability of Supervised AI/ML in the Earth Systems Science by Leveraging Social Science Methods

Increasing the Reproducibility and Replicability of Supervised AI/ML in the Earth Systems Science by Leveraging Social Science Methods

Our Vision for JGR: Machine Learning and Computation

Garbage in, garbage out: mitigating risks and maximizing benefits of AI in research

Contact Info

Product

Resources

About