Data Readiness Report

Afzal, Shazia; Rajmohan, C; Kesarwani, Manish; Mehta, Sameep; Patel, Hima

doi:10.1109/smds53860.2021.00016

Cited by 13 publications

(5 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, Gebru et al (2021) advocate that documentation promotes the communication between "dataset consumers and producers". Existing frameworks for the elaboration of documentation include: Datasheets for Datasets (Gebru et al, 2021), Dataset Nutrition Labels (Chmielinski et al, 2022), Data Statements (Bender & Friedman, 2018), Data Readiness Report (Afzal et al, 2021), and Model Cards for Models (Mitchell et al, 2019). Formal data models, like ontologies and controlled vocabularies, can also support AIrelated documentation needs.…”

Section: Living With Bias By Documenting Itmentioning

confidence: 99%

Policy advice and best practices on bias and fairness in AI

Alvarez,

Colmenarejo,

Elobaid

et al. 2024

Ethics Inf Technol

View full text Add to dashboard Cite

The literature addressing bias and fairness in AI models (fair-AI) is growing at a fast pace, making it difficult for novel researchers and practitioners to have a bird’s-eye view picture of the field. In particular, many policy initiatives, standards, and best practices in fair-AI have been proposed for setting principles, procedures, and knowledge bases to guide and operationalize the management of bias and fairness. The first objective of this paper is to concisely survey the state-of-the-art of fair-AI methods and resources, and the main policies on bias in AI, with the aim of providing such a bird’s-eye guidance for both researchers and practitioners. The second objective of the paper is to contribute to the policy advice and best practices state-of-the-art by leveraging from the results of the NoBIAS research project. We present and discuss a few relevant topics organized around the NoBIAS architecture, which is made up of a Legal Layer, focusing on the European Union context, and a Bias Management Layer, focusing on understanding, mitigating, and accounting for bias.

show abstract

Section: Living With Bias By Documenting Itmentioning

confidence: 99%

Policy advice and best practices on bias and fairness in AI

Alvarez,

Colmenarejo,

Elobaid

et al. 2024

Ethics Inf Technol

View full text Add to dashboard Cite

show abstract

“…This information facilitates the evaluation of the suitability of a dataset by data scientists for specific tasks. The Data Readiness Report [1] present a similar proposal, deriving its design from the data readiness framework [6]. On top of the statistical analysis it also defines a set of quality metrics for evaluating datasets' composition.…”

Section: Data Documentation Proposals From the ML Communitymentioning

confidence: 99%

“…This situation has brought recent interest inside the research community about a data-centric cultural shift in the machine learning field 1 . The standardization of data creation processes, the need for formal documentation, and the need for mature tools to adopt best practices are common demands inside the research community.…”

mentioning

confidence: 99%

A domain-specific language for describing machine learning datasets

Giner-Miguelez¹,

Gómez²,

Cabot³

2022

Preprint

View full text Add to dashboard Cite

Datasets play a central role in the training and evaluation of machine learning (ML) models. But they are also the root cause of many undesired model behaviors, such as biased predictions. To overcome this situation, the ML community is proposing a datacentric cultural shift where data issues are given the attention they deserve, and more standard practices around the gathering and processing of datasets start to be discussed and established.So far, these proposals are mostly high-level guidelines described in natural language and, as such, they are difficult to formalize and apply to particular datasets. In this sense, and inspired by these proposals, we define a new domain-specific language (DSL) to precisely describe machine learning datasets in terms of their structure, data provenance, and social concerns. We believe this DSL will facilitate any ML initiative to leverage and benefit from this data-centric shift in ML (e.g., selecting the most appropriate dataset for a new project or better replicating other ML results). The DSL is implemented as a Visual Studio Code plugin, and it has been published under an open source license.

show abstract

“…Challenges include poor and undocumented data collection practices, missing values, inconvenient storage mechanisms, intellectual property, security and privacy -notice the overlap with the interoperability issues above. To help overcome these challenges, several frameworks for data readiness have been developed [122,580,581]. These provide data handling structure and communication tools for ML practitioners, including required implementations such as data profiling and quality analyses as a key step before data enters a ML pipeline, and continuous testing for shifts in the data and environments of deployed models.…”

Section: Data-intensive Science and Computingmentioning

confidence: 99%

Simulation Intelligence: Towards a New Generation of Scientific Methods

Lavin¹,

Zenil²,

Krakauer³

et al. 2021

Preprint

View full text Add to dashboard Cite

The original "Seven Motifs" set forth a roadmap of essential methods for the field of scientific computing, where a motif is an algorithmic method that captures a pattern of computation and data movement. 1 We present the Nine Motifs of Simulation Intelligence, a roadmap for the development and integration of the essential algorithms necessary for a merger of scientific computing, scientific simulation, and artificial intelligence. We call this merger simulation intelligence (SI), for short. We argue the motifs of simulation intelligence are interconnected and interdependent, much like the components within the layers of an operating system. Using this metaphor, we explore the nature of each layer of the simulation intelligence "operating system" stack (SI-stack) and the motifs therein:1. Multi-physics and multi-scale modeling 2. Surrogate modeling and emulation 3. Simulation-based inference 4. Causal modeling and inference 5. Agent-based modeling 6. Probabilistic programming 7. Differentiable programming 8. Open-ended optimization Machine programmingWe believe coordinated efforts between motifs offers immense opportunity to accelerate scientific discovery, from solving inverse problems in synthetic biology and climate science, to directing nuclear energy experiments and predicting emergent behavior in socioeconomic settings. We elaborate on each layer of the SI-stack, detailing the state-of-art methods, presenting examples to highlight challenges and opportunities, and advocating for specific ways to advance the motifs and the synergies from their combinations. Advancing and integrating these technologies can enable a robust and efficient hypothesis-simulation-analysis type of scientific method, which we introduce with several use-cases for human-machine teaming and automated science.

show abstract

Data Readiness Report

Cited by 13 publications

References 10 publications

Policy advice and best practices on bias and fairness in AI

Policy advice and best practices on bias and fairness in AI

A domain-specific language for describing machine learning datasets

Simulation Intelligence: Towards a New Generation of Scientific Methods

Contact Info

Product

Resources

About