Interpreting cloud computer vision pain-points

Cummaudo, Alex; Vasa, Rajesh; Barnett, Scott; Grundy, John; Abdelrazek, Mohamed

doi:10.1145/3377811.3380404

Cited by 27 publications

(30 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally we concluded that the understanding of how Software Engineering Model Development practices and the adoption of a Machine Learning Workflow in accordance with those practices, more specifically, with the Software Engineering life-cycle is a subject of vital importance for the evolution of Machine Learning/Artificial Intelligence and continuing development of its applications (especially on a large scale), even if further research on the topic is needed. -Provenance tag for data models [1], [2], [13] Documentation and Versioning -Extract metadata from repositories is difficult -Catalog of ML models to support design and maintenance [15] Non-functional Requirements -security -unassured reliability and lacking transparency -Identify parts of the ISO 26262 to be adapted to ML -An approach based on dependability assurances [30], [31] Design and Implementation -APIs look and feel like conventional APIs, but abstract away data-driven behavior -catalog of design patterns for ML development -information to support documentation and design of APIs [6], [32] Evaluation -Testing interpretability, privacy, or efficiency of ML -Proposal of new test semantic -Tests based on quality score [3], [33], [34] Deployment and Maintenance -Lack of support to adapt based on feedback -An approach to support adaptation based on quality gates [34] Software Capability Maturity Model (CMM)…”

Section: Discussionmentioning

confidence: 99%

Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Review

Lorenzoni¹,

Alencar²,

Nascimento³

et al. 2021

Preprint

View full text Add to dashboard Cite

Data scientists often develop machine learning models to solve a variety of problems in the industry and academy but not without facing several challenges in terms of Model Development. The problems regarding Machine Learning Development involves the fact that such professionals do not realize that they usually perform ad-hoc practices that could be improved by the adoption of activities presented in the Software Engineering Development Lifecycle. Of course, since machine learning systems are different from traditional Software systems, some differences in their respective development processes are to be expected. In this context, this paper is an effort to investigate the challenges and practices that emerge during the development of ML models from the software engineering perspective by focusing on understanding how software developers could benefit from applying or adapting the traditional software engineering process to the Machine Learning workflow.

show abstract

Section: Discussionmentioning

confidence: 99%

Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Review

Lorenzoni¹,

Alencar²,

Nascimento³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, due to the rapid development of big data and machine learning, researchers have also started to study the problems and challenges that developers encounter in the development practices of these two types of applications based on the data on Stack Overlow [5,7,51]. For example, studies of deep learning bug characteristics [23], TensorFlow program bugs [52], deep learning deployment [14,19], and cloud computer vision [16] have emerged. These studies are about how to identify or understand the challenges and characteristics related to software development based on the developers' discussions on Stack Overlow.…”

Section: Related Workmentioning

confidence: 99%

Do Developers Really Know How to Use Git Commands? A Large-scale Study Using Stack Overflow

Yang

Zhang

Pan

et al. 2022

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Git, a cross-platform and open-source distributed version control tool, provides strong support for non-linear development and is capable of handling everything from small to large projects with speed and efficiency. It has become an indispensable tool for millions of software developers and is the de facto standard of version control in software development nowadays. However, despite its widespread use, developers still frequently face difficulties when using various Git commands to manage projects and collaborate. To better help developers use Git, it is necessary to understand the issues and difficulties that they may encounter when using Git. Unfortunately, this problem has not yet been comprehensively studied. To fill this knowledge gap, in this paper, we conduct a large-scale study on Stack Overflow, a popular Q&A forum for developers. We extracted and analyzed 80,370 relevant questions from Stack Overflow, and reported the increasing popularity of the Git command questions. By analyzing the questions, we identified the Git commands that are frequently asked and those that are associated with difficult questions on Stack Overflow to help understand the difficulties developers may encounter when using Git commands. In addition, we conducted a survey to understand how developers learn Git commands in practice, showing that self-learning is the primary learning approach. These findings provide a range of actionable implications for researchers, educators, and developers.

show abstract

“…This means that their large training datasets may continuously update the prediction classifiers making the inferences, resulting both in probabilistic and non-deterministic outcomes [11,17]. Critically for software engineers using the services, these non-deterministic aspects have not been sufficiently documented in the service's API documented, which has been shown to confuse developers [9]. A strategy to combat such service changes, which we often observe in traditional software engineering practices, are for such services to be versioned upon substantial change.…”

Section: 'Intelligent' Vs 'Traditional' Web Servicesmentioning

confidence: 99%

“…However, emerging evidence indicates that 'intelligent' services do not communicate changes explicitly [10]. Intelligent services evolve in unpredictable ways, provide no notification to developers and changes are undocumented [9]. To illustrate this, consider fig.…”

Section: Introductionmentioning

confidence: 99%

“…For developers to discover the impact on their applications they need to know the behavioural deviation and the associated impact on the robustness and reliability of their system. Currently, there is no guidance on how to deal with this evolution, nor do developers have an explicit checklist of the likely errors and changes that they must test for [9].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Beware the evolving 'intelligent' web service! An integration architecture tactic to guard AI-first components

Cummaudo¹,

Barnett²,

Vasa³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Intelligent services provide the power of AI to developers via simple RESTful API endpoints, abstracting away many complexities of machine learning. However, most of these intelligent services-such as computer vision-continually learn with time. When the internals within the abstracted 'black box' become hidden and evolve, pitfalls emerge in the robustness of applications that depend on these evolving services. Without adapting the way developers plan and construct projects reliant on intelligent services, significant gaps and risks result in both project planning and development. Therefore, how can software engineers best mitigate software evolution risk moving forward, thereby ensuring that their own applications maintain quality? Our proposal is an architectural tactic designed to improve intelligent service-dependent software robustness. The tactic involves creating an application-specific benchmark dataset baselined against an intelligent service, enabling evolutionary behaviour changes to be mitigated. A technical evaluation of our implementation of this architecture demonstrates how the tactic can identify 1,054 cases of substantial confidence evolution and 2,461 cases of substantial changes to response label sets using a dataset consisting of 331 images that evolve when sent to a service. CCS CONCEPTS• Information systems → Web services; • Software and its engineering → Software evolution; • Hardware → Error detection and error correction; • Computer systems organization → Client-server architectures.

show abstract

Interpreting cloud computer vision pain-points

Cited by 27 publications

References 29 publications

Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Review

Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Review

Do Developers Really Know How to Use Git Commands? A Large-scale Study Using Stack Overflow

Beware the evolving 'intelligent' web service! An integration architecture tactic to guard AI-first components

Contact Info

Product

Resources

About