Towards a Software Engineering Process for Developing Data-Driven Applications

Hesenius, Marc; Schwenzfeier, Nils; Meyer, Ole; Koop, Wilhelm; Gruhn, Volker

doi:10.1109/raise.2019.00014

Cited by 35 publications

(29 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In a similar fashion, by presenting methods for measuring the best practices degree of adoption when investigating the relationship between different groups of practices and assessing/predicting their effects by performing regression models, Serban's [5] article reaches conclusions that are in line with Hesenius et al [4] in a way that there is a set of best practices which is applicable to any ML application development, regardless of the type of data under consideration. Additionally, the author has contributed to the evolution of such practices by presenting a methodology in which each practice is related to its effects and adoption rate.…”

Section: Related Workmentioning

confidence: 87%

“…In contrast to the partial information in works such as Amershi et al [1], authors such as Hesenius et al [4] argue that although there are challenges faced by software engineers when developing data-driven applications, the data dependency of ML/AI applications does not constitute an issue to the adoption of a common integrated Software Engineering (SE) process, upon which the project's overall success would depend. As a consequence, by defining a set of roles (Software Engineer, Data Scientist, Data Domain Expert, and Domain Expert), stages, and responsibilities to structure the necessary work, decisions and documents, the authors provided a structured engineering process that suits all data-driven applications, ultimately filling the gap found in the literature.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Review

Lorenzoni¹,

Alencar²,

Nascimento³

et al. 2021

Preprint

View full text Add to dashboard Cite

Data scientists often develop machine learning models to solve a variety of problems in the industry and academy but not without facing several challenges in terms of Model Development. The problems regarding Machine Learning Development involves the fact that such professionals do not realize that they usually perform ad-hoc practices that could be improved by the adoption of activities presented in the Software Engineering Development Lifecycle. Of course, since machine learning systems are different from traditional Software systems, some differences in their respective development processes are to be expected. In this context, this paper is an effort to investigate the challenges and practices that emerge during the development of ML models from the software engineering perspective by focusing on understanding how software developers could benefit from applying or adapting the traditional software engineering process to the Machine Learning workflow.

show abstract

Section: Related Workmentioning

confidence: 87%

Section: Related Workmentioning

confidence: 99%

Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Review

Lorenzoni¹,

Alencar²,

Nascimento³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…2; [3]) sowie der Einrichtung neuer Rollen und weniger im Erlernen neuer Algorithmen des maschinellen Lernens. Neben der Rolle des Softwareingenieurs und des mittlerweile schon ebenso klassischen Data Scientists werden hierbei etwa auch die Rollen Domain Expert für das branchentypische Geschäftswissen und Data Domain Expert für die Kenntnisse der branchentypischen Daten notwendig werden [3].…”

Section: Bindeglied Methodenkompetenzunclassified

New School of IT: Der aktuelle Spielstand

Bub

Striemer

Gruhn

2020

Wirtsch Inform Manag

Self Cite

View full text Add to dashboard Cite

Rechenleistung und zunehmender Datenmenge, hat eine Reihe von Entwicklungen zur Folge, die die Rolle von IT-Organisationen dauerhaft verändern. In [1] haben wir eine New School of IT beschrieben, die erste Antworten auf die Herausforderungen an IT-Organisationen bietet. Zehn Jahre nach der ersten Formulierung des Begriffes sind durch Cloud Native Computing oder künstliche Intelligenz bereits konkrete Meilensteine in der New School of IT erreicht, es bleiben aber auch eine Reihe von offenen Herausforderungen an Methodenkompetenz und Ausbildung. Wir fassen im Folgenden den Stand der Dinge zusammen.

show abstract

“…This helps the domain expert to obtain an entry point to the methodical field and derive a better understanding of a possible solution space. Thus, the DSA project in its initial stage can be described in terms of domain entities of interest, the DM method to be implemented, and the data assets to be used to create a blueprint for project realization (Brodsky et al 2015;Zschech 2018;Hesenius et al 2019).…”

Section: Description Of the Mapping Problemmentioning

confidence: 99%

Intelligent User Assistance for Automated Data Mining Method Selection

et al. 2020

View full text Add to dashboard Cite

In any data science and analytics project, the task of mapping a domain-specific problem to an adequate set of data mining methods by experts of the field is a crucial step. However, these experts are not always available and data mining novices may be required to perform the task. While there are several research efforts for automated method selection as a means of support, only a few approaches consider the particularities of problems expressed in the natural and domain-specific language of the novice. The study proposes the design of an intelligent assistance system that takes problem descriptions articulated in natural language as an input and offers advice regarding the most suitable class of data mining methods. Following a design science research approach, the paper (i) outlines the problem setting with an exemplary scenario from industrial practice, (ii) derives design requirements, (iii) develops design principles and proposes design features, (iv) develops and implements the IT artifact using several methods such as embeddings, keyword extractions, topic models, and text classifiers, (v) demonstrates and evaluates the implemented prototype based on different classification pipelines, and (vi) discusses the results' practical and theoretical contributions. The best performing classification pipelines show high accuracies when applied to validation data and are capable of creating a suitable mapping that exceeds the performance of joint novice assessments and simpler means of text mining. The research provides a promising foundation for further enhancements, either as a stand-alone intelligent assistance system or as an add-on to already existing data science and analytics platforms.

show abstract

Towards a Software Engineering Process for Developing Data-Driven Applications

Cited by 35 publications

References 10 publications

Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Review

Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Review

New School of IT: Der aktuelle Spielstand

Intelligent User Assistance for Automated Data Mining Method Selection

Contact Info

Product

Resources

About