Human recoverability index: A TraceLab experiment

Abstract

Abstract-It has been generally accepted that not all trace links in a given requirements traceability matrix are equal -both human analysts and automated methods are good at spotting some links, but have blind spots for some other. One way to choose automated techniques for inclusion in assisted tracing processes (i.e., the tracing processes that combine the expertise of a human analyst and special-purpose tracing software) is to select the techniques that tend to discover more links that are hard for human analysts to observe and establish on their own. This paper proposes a new measure of performance of a tracing method: human recoverability index-based recall. In the presence of knowledge about the difficulty of link recovery by human analysts, this measure rewards methods that are able to recover such links over methods that tend to recover the same links as the human analysts. We describe a TraceLab experiment we designed to evaluate automated trace recovery methods based on this measure and provide a case study of the use of this experiment to profile and evaluate different automated tracing techniques.

Configuring and Assembling Information Retrieval Based Solutions for Software Engineering Tasks

“…That being said, we tried to implement as many of the techniques that we could in order to show the efficacy and usefulness of TraceLab as a research tool in the domain of software evolution and maintenance. We are continuously working on driving new research projects with TraceLab [3,29,44,50,78,101,155,183] and encourage others to do so as well.…”