Measurement and metrology requirements for empirical studies in software engineering

Abran, Alain; Sellami, Asma

doi:10.1109/step.2002.1267631

Cited by 5 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…2 See footnote 1. 3 The test suites are provided as a single Eclipse workspace containing four projects. They are available as one Eclipse workspace at https://github.com/GRISE-UPM/TestSuitesMeasurement/tree/master/test suites.…”

Section: Characteristics Of the Mr And Bsk Test Suitesmentioning

confidence: 99%

“…Measure theory has been the target of a substantial amount of research in SE. In turn, metrology has been overlooked (with few exceptions such as [3,4]), giving rise to the absence of a methodological background to perform the comparisons of measurements. To fill this gap, we introduce specialized terminology in the following Sections and, later, the comparison methods themselves.…”

Section: Fundamental Conceptsmentioning

confidence: 99%

“…There are diverse strategies to assess inter-rater agreement 14 . For our research problem, the right one is the Inter-class Correlation Coefficient (ICC) Model 3 (mixed factorial design), also known as ICC (3,1) in some statistical packages such as SPSS ® [35,56].…”

Section: Fig 2 Interpretation Of the Bland-altman Plotmentioning

confidence: 99%

See 2 more Smart Citations

Test cases as a measurement instrument in experimentation

Dieste¹,

Uyaguari²,

Vegas³

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: Test suites are frequently used to quantify relevant software attributes, such as quality or productivity. Problem: We have detected that the same response variable, measured using different test suites, yields different experiment results. Aims: Assess to which extent differences in test case construction influence measurement accuracy and experimental outcomes. Method: Two industry experiments have been measured using two different test suites, one generated using an ad-hoc method and another using equivalence partitioning. The accuracy of the measures has been studied using standard procedures, such as ISO 5725, Bland-Altman and Interclass Correlation Coefficients. Results: There are differences in the values of the response variables up to ±60%, depending on the test suite (ad-hoc vs. equivalence partitioning) used. Conclusions: The disclosure of datasets and analysis code is insufficient to ensure the reproducibility of SE experiments. Experimenters should disclose all experimental materials needed to perform independent measurement and re-analysis.

show abstract

Section: Characteristics Of the Mr And Bsk Test Suitesmentioning

confidence: 99%

Section: Fundamental Conceptsmentioning

confidence: 99%

See 1 more Smart Citation

Test cases as a measurement instrument in experimentation

Dieste¹,

Uyaguari²,

Vegas³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Using both the ISO set of metrology concepts model [3,4] and the measurement process model [2,20,21], we can analyze the current status of the field of software measurements as documented in the SWEBOK Guide.…”

Section: Measurement Steps and Metrology Concepts Within Swebokmentioning

confidence: 99%

“…Using a detailed inventory of the measurement-related statements appearing in the ten SWEBOK chapters, these statements were analyzed in terms of measurement concepts, and then mapped into both the set of metrology concepts presented in Section 4 and to the measurement process model presented in Section 5. Table 5 lists, for each of the ten chapters of SWEBOK, which metrology concepts and measurement steps are addressed whenever a measurement-related statement appears in the SWEBOK Guide [4]. From Table 5, it can be observed that a large majority of the measurement-related concepts mentioned in SWEBOK are listed in the category of concepts related to the exploitation of the measurement results.…”

Section: Measurement Steps and Metrology Concepts Within Swebokmentioning

confidence: 99%