Automated and Manual Testing in the Development of the Research Software RCE

Abstract

Research software is often developed by individual researchers or small teams in parallel to their research work. The more people and research projects rely on the software in question, the more important it is that software updates implement new features correctly and do not introduce regressions. Thus, developers of research software must balance their limited resources between implementing new features and thoroughly testing any code changes. We present the processes we use for developing the distributed integration framework RCE at DLR. These processes aim to strike a balance between automation and manual testing, reducing the testing overhead while addressing issues as early as possible. We furthermore briefly describe how these testing processes integrate with the surrounding processes for development and releasing.

Теоретичні Засади Тестування Програмного Забезпечення

“…Ручне тестування є фундаментальним і часто необхідним аспектом тестування програмного забезпечення, що пропонує такі переваги, як гнучкість, детальне розуміння досвіду користувача та здатність адаптуватися до мінливих вимог [4]. Кращим випадком тестування є варіант коли наявна вимога тестувати програмне забезпечення з точки зору користувача чи клієнта і програмне забезпечення перевіряється попередньо вручну перед виконанням будь-якого автоматизованого тестування [5].…”

Facilitating Test-Driven Development via Domain-Specific Languages in Computational Science Software Engineering

“…In contrast to computational science, in the software development industry, DSLs are already used for software testing on a daily basis (Micallef & Colombo, 2015). While the specific implementations are not directly designed for computational science, mainstream technologies such as Cucumber or Gherkin (Wynne, Hellesoy, & Tooke, 2017) can be used to define test suites and test cases to validate high-level aspects (Mischke, Schaffert, Schneider, & Weinert, 2022).…”

Orchestrating Tool Chains for Model-based Systems Engineering with RCE

“…The development cycle is oriented on semantic versioning [18], meaning there are major, minor, and patch releases on a regular basis. Each release undergoes a quality management procedure containing, among other things, continuous integration, automated tests, and manual testing [10].…”