Case studies for method and tool evaluation

Kitchenham, Barbara; Pickard, Lesley M.; Pfleeger, Shari Lawrence

doi:10.1109/52.391832

Cited by 447 publications

(303 citation statements)

References 5 publications

Supporting

Mentioning

245

Contrasting

Unclassified

Order By: Relevance

“…However, because the corporate, team, and project characteristics are unique to each case study, comparisons and generalizations of case study results are difficult and are subject to questions of external validity [14]. Nevertheless, case studies are particularly important for industrial evaluation of software engineering methods and tools [13]. Researchers become more confident in a theory when similar findings emerge in different contexts [13].…”

Section: Case Study Researchmentioning

confidence: 99%

“…Nevertheless, case studies are particularly important for industrial evaluation of software engineering methods and tools [13]. Researchers become more confident in a theory when similar findings emerge in different contexts [13]. By performing multiple case studies and/or experiments and recording the context variables of each case study, researchers can build up evidence through a family of experiments.…”

Section: Case Study Researchmentioning

confidence: 99%

“…Before conducting any case study, it is important to select a sample that is representative of the company or organization as a whole in order to reduce threats to external validity. Kitchenham notes that project selection is not always a decision that can be made by the researcher, since a participating company may only commit one or two teams/projects to the research [13]. One can select based on project age, programming language, development methodology, risk factors, etc.…”

Section: Team and Project Selectionmentioning

confidence: 99%

“…Case studies can be viewed as "research in the typical" [8,13]. As opposed to formal experiments, which often have a narrow focus and an emphasis on controlling context variables, case studies in software engineering test theories and collect data through observation of a project in an unmodified setting [25].…”

Section: Case Study Researchmentioning

confidence: 99%

See 3 more Smart Citations

Motivations and measurements in an agile case study

Layman

Williams

Cunningham

2006

Journal of Systems Architecture

View full text Add to dashboard Cite

With the recent emergence of agile software development technologies, the software community is awaiting sound, empirical investigation of the impacts of agile practices in a live setting. One means of conducting such research is through industrial case studies. However, there are a number of influencing factors that contribute to the success of such a case study. In this paper, we describe a case study performed at Sabre Airline Solutions evaluating the effects of adopting Extreme Programming (XP) practices with a team that had characteristically plan-driven risk factors. We compare the team's business-related results (productivity and quality) to two published sources of industry averages. Our case study found that the Sabre team yielded above-average post-release quality and average to above-average productivity.We discuss our experience in conducting this case study, including specifics of how data was collected, the rationale behind our process of data collection, and what obstacles were encountered during the case study. We also identify four factors that potentially impact the outcome of industrial case studies: availability of data, tool support, co-operative personnel and project status. We believe that recognizing and planning for these factors is essential to conducting industrial case studies, and that this information will be helpful to researchers and practitioners alike.

show abstract

Section: Case Study Researchmentioning

confidence: 99%

Section: Case Study Researchmentioning

confidence: 99%

Section: Team and Project Selectionmentioning

confidence: 99%

Section: Case Study Researchmentioning

confidence: 99%

See 2 more Smart Citations

Motivations and measurements in an agile case study

Layman

Williams

Cunningham

2006

Journal of Systems Architecture

View full text Add to dashboard Cite

show abstract

“…Test-Driven Development (TDD) is one of XP's best practices, and possibly the key practice (Beck, 1999a;Jeffries et al, 2001;Auer and Miller, 2002). When a team practices TDD, they write automated tests before they write or change their code, to ensure that their changes cause the new tests to move from failing to passing and to add to the team's test assets.…”

mentioning

confidence: 99%

A multiple comparative study of test-with development product changes and their effects on team speed and product quality

Bannerman

Martin

2010

Empir Software Eng

View full text Add to dashboard Cite

Researchers have typically studied the effects of Test-First Development (TFD), compared to Test-Last Development (TLD), across groups or projects, and for relatively short durations. We define Test-With Development (TWD) as more general than the fine-grained step of TFD, but also in contrast to the large-grained phase of TLD. With this definition, we performed a multiple comparative study to explore and describe TWD product changes, and the effects of those changes on two attributes related to team speed and two attributes related to product quality, within six long-term open-source projects. Our results indicate that when developers exercised some of their changes with automated tests, on average they made significantly larger changes over time while significantly reducing their product's complexity. And, when they exercised all of their changes with tests, on average they made significantly smaller changes over time. We interpret these results to indicate that practicing TWD supports faster simplification of a product. Therefore, we conclude that teams that need to simplify their product can benefit from practicing TWD.

show abstract

Framework for tool evaluation for a maintenance environment

Vierimaa

Taramaa

Puustinen

et al. 1998

J. Softw. Maint: Res. Pract.

View full text Add to dashboard Cite

Tool evaluation forms an essential part of tool development. Unfortunately, there are only a few solutions for this activity. As part of the AMES project, tool evaluation was implemented as part of a more comprehensive framework, used with the development and application of several AMES tools for application maintenance. These included an application understanding tool‐set, a disabbreviation tool, a reverse‐engineering tool and an impact analysis tool‐set. In addition, the requirement for applicability both to on‐board space software and to tool software has directed our approach in setting requirements for tools evaluation. The central element of the evaluation process was an AMES‐developed framework including evaluation criteria and the use of a goal/question/metrics (GQM) based approach. The criteria have produced detailed information about each tool. The final result can be regarded as a combination of the score for each criterion and explicit metrics data. In addition, descriptive information about applications was used in tool evaluation. © 1998 John Wiley & Sons, Ltd.

show abstract

Case studies for method and tool evaluation

Cited by 447 publications

References 5 publications

Motivations and measurements in an agile case study

Motivations and measurements in an agile case study

A multiple comparative study of test-with development product changes and their effects on team speed and product quality

Framework for tool evaluation for a maintenance environment

Contact Info

Product

Resources

About