Software Engineering for Machine Learning: A Case Study

Amershi, Saleema; Begel, Andrew; Bird, Christian; DeLine, Robert; Gall, Harald C.; Kamar, Ece; Nagappan, Nachiappan; Nushi, Besmira; Zimmermann, Thomas

doi:10.1109/icse-seip.2019.00042

Cited by 658 publications

(499 citation statements)

References 26 publications

Supporting

Mentioning

468

Contrasting

Unclassified

Order By: Relevance

“…Bug Detection in Data. The behaviours of a machine learning system largely depends on data [8]. Bugs in data affect the quality of the generated model, and can be amplified to yield more serious problems over a period a time [45].…”

Section: Testing Componentsmentioning

confidence: 99%

“…According to the work of Amershi et al [8], data testing is especially important and certainly deserves more research efforts on it. Additionally, there are also many opportunities for regression testing, bug report analysis, and bug triage in ML testing.…”

Section: Research Opportunities In ML Testingmentioning

confidence: 99%

“…Machine learning testing poses challenges that arise from the fundamentally different nature and construction of machine learning systems, compared to traditional (relatively more deterministic and less statistically-orientated) software systems. For instance, a machine learning system inherently follows a data-driven programming paradigm where the decision logic is obtained via a training procedure from training data under the machine learning algorithm's architecture [8]. The model's behaviour may evolve over time, in response to the frequent provision of new data [8].…”

Section: Introductionmentioning

confidence: 99%

“…For instance, a machine learning system inherently follows a data-driven programming paradigm where the decision logic is obtained via a training procedure from training data under the machine learning algorithm's architecture [8]. The model's behaviour may evolve over time, in response to the frequent provision of new data [8]. While this is also true of traditional software systems, the core underlying behaviour of a traditional system does not typically change in response to new data, in the way that a machine learning system can.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Machine Learning Testing: Survey, Landscapes and Horizons

Zhang

Harman

Ма

et al. 2022

IIEEE Trans. Software Eng.

453

318

View full text Add to dashboard Cite

This paper provides a comprehensive survey of Machine Learning Testing (ML testing) research. It covers 128 papers on testing properties (e.g., correctness, robustness, and fairness), testing components (e.g., the data, learning program, and framework), testing workflow (e.g., test generation and test evaluation), and application scenarios (e.g., autonomous driving, machine translation). The paper also analyses trends concerning datasets, research trends, and research focus, concluding with research challenges and promising research directions in ML testing. Index Terms-machine learning, software testing, deep neural network, ! • Jie M. Zhang and Mark Harman are with CREST, University College London, United Kingdom. Mark Harman is also with Facebook London.

show abstract

Section: Testing Componentsmentioning

confidence: 99%

Section: Research Opportunities In ML Testingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Machine Learning Testing: Survey, Landscapes and Horizons

Zhang

Harman

Ма

et al. 2022

IIEEE Trans. Software Eng.

453

318

View full text Add to dashboard Cite

show abstract

“…One of large organizations with significant experience in AI development, as reported in [14], conducted a two-phase study with a set of interviews to gather the major topics and a widescale survey about the identified topics, in order to observe their software teams as they develop AI-based applications. They found that various teams have united the new workflow into pre-existing, well-evolved, agile-like SE processes, providing insights about several engineering challenges that organizations may face in building AI-based complex systems [14]: end-to-end pipeline support; data availability, collection, cleaning, and management; education and training; model debugging and interpretability; model evolution, evaluation, and deployment; compliance; varied perceptions.…”

Section: A Ai Development Challenges At Largementioning

confidence: 99%

Requirements Engineering Challenges in Building AI-Based Complex Systems

Belani

Vuković

Car

2019

2019 IEEE 27th International Requirements Engineering Conference Workshops (REW)

View full text Add to dashboard Cite

This paper identifies and tackles the challenges of the requirements engineering discipline when applied to development of AI-based complex systems. Due to their complex behaviour, there is an immanent need for a tailored development process for such systems. However, there is still no widely used and specifically tailored process in place to effectively and efficiently deal with requirements suitable for specifying a software solution that uses machine learning. By analysing the related work from software engineering and artificial intelligence fields, potential contributions have been recognized from agent-based software engineering and goal-oriented requirements engineering research, as well as examples from large product development companies. The challenges have been discussed, with proposals given how and when to tackle them. RE4AI taxonomy has also been outlined, to inform the tailoring of development process.

show abstract

Machine Learning in Lithium‐Ion Battery Cell Production: A Comprehensive Mapping Study

Haghi

Hidalgo

Niri

et al. 2023

Batteries & Supercaps

View full text Add to dashboard Cite

With the global quest for improved sustainability, partially realized through the electrification of the transport and energy sectors, battery cell production has gained ever‐increasing attention. An in‐depth understanding of battery production processes and their interdependence is crucial for accelerating the commercialization of material developments, for example, at the volume predicted to underpin future electric vehicle production. Over the last five years, machine learning approaches have shown significant promise in understanding and optimizing the battery production processes. Based on a systematic mapping study, this comprehensive review details the state‐of‐the‐art applications of machine learning within the domain of lithium‐ion battery cell production and highlights the fundamental aspects, such as product and process parameters and adopted algorithms. The compiled findings derived from multi‐perspective comparisons demonstrate the current capabilities and reveal future research opportunities in this field to further accelerate sustainable battery production.

show abstract

Software Engineering for Machine Learning: A Case Study

Cited by 658 publications

References 26 publications

Machine Learning Testing: Survey, Landscapes and Horizons

Machine Learning Testing: Survey, Landscapes and Horizons

Requirements Engineering Challenges in Building AI-Based Complex Systems

Machine Learning in Lithium‐Ion Battery Cell Production: A Comprehensive Mapping Study

Contact Info

Product

Resources

About