Experimental Evaluation of FMCheck: A Replication Study

Abstract

Software Product Lines are usually specified using feature models. A hierarchically arranged set of features with different relationships among them represents a feature model. However, there is a lack of techniques to support the detection of semantic defects in feature models. In this context, it was recently developed FMCheck, a checklist-based inspection technique to support the detection of defects in feature models. The results of a first study conducted by FMCheck's developers indicated its feasibility (more effective) when compared to ad-hoc techniques. This paper reports the replication accomplished by an independent research group following a different experimental design but using the same artifacts. The obtained results strengthened the previous findings, indicating that FMCheck is more effective than ad-hoc inspections. However, additional replications should be performed with different experimental designs to understand better the influence of the artifacts inspected over such findings.