Combining self-reported and automatic data to improve programming effort measurement

Hochstein, Lorin; Basili, Victor R.; Zelkowitz, Marvin V.; Hollingsworth, Jeffrey K.; Carver, Jeff

doi:10.1145/1081706.1081762

Cited by 18 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 5 summarizes the number of times each technology has been applied to each programming problem. More details are given in [Ho05] and [Al07]. For example, we can use this data to partially answer an earlier stated hypothesis (Hyp.…”

Section: Programming Model Speedup On 8 Processorsmentioning

confidence: 89%

“…After conducting a series of tests using variations on these techniques, we settled on a hybrid approach that combines diaries with an instrumented programming environment that captures a time-stamped record of all compiler invocations (including capture of source code), all programs invoked by the subject as a shell command, and interactions with supported editors. Elsewhere [Ho05], we describe the details of how we gather this information and convert it into a record of programmer effort.…”

Section: Embarrassingly Parallelmentioning

confidence: 99%

“…By understanding how, when, and the kind of defects that appear in HPC codes, tools and techniques can be developed to mitigate these risks to improve the overall workflow. As we have shown [Ho05], automatically determining workflow is not precise, so we are working on a mixture of process activity (e.g., coding, compiling, executing) with source code analysis techniques. The process of defect analysis we are building consists of the following main activities:…”

Section: Defect Studiesmentioning

confidence: 99%

See 2 more Smart Citations

Experiments to Understand HPC Time to Development (Final report for Department of Energy contract DE-FG02-04ER25633) Report DOE/ER/25633-1

Basili¹,

Zelkowitz²

2007

View full text Add to dashboard Cite

In order to understand how high performance computing (HPC) programs are developed, a series of experiments, using students in graduate level HPC classes and various research centers, were conducted at various locations in the US. In this report, we discuss this research, give some of the early results of those experiments, and describe a web-based Experiment Manager we are developing that allows us to run studies more easily and consistently at universities and laboratories, allowing us to generate results that more accurately reflect the process of building HPC programs.

show abstract

Section: Programming Model Speedup On 8 Processorsmentioning

confidence: 89%

Section: Embarrassingly Parallelmentioning

confidence: 99%

Section: Defect Studiesmentioning

confidence: 99%

See 1 more Smart Citation

Experiments to Understand HPC Time to Development (Final report for Department of Energy contract DE-FG02-04ER25633) Report DOE/ER/25633-1

Basili¹,

Zelkowitz²

2007

View full text Add to dashboard Cite

show abstract

“…On the pragmatic side, activities provide useful input for predicting development cost and effort [5,6] or the resolution of bugs [8]. Several authors studied how commit messages can be leveraged to predict types of source code changes based on predefined sets of terms for both commercial [14] and Open Source projects [4].…”

Section: Related Workmentioning

confidence: 99%

From work to word: How do software developers describe their work?

Maalej

Happel

2009

2009 6th IEEE International Working Conference on Mining Software Repositories

View full text Add to dashboard Cite

Developers take notes about their work sessions, either to remember the work status and share it with collaborators, or because employers explicitly require this for project management matters. We report on an exploratory study which aims at understanding how software developers describe their work. We analyzed more than 750,000 work descriptions of about 2,000 professionals taken over 8 years in three settings. We observed several similarities in the content and time meta-data of work descriptions. Most frequent terms, such as top-30 performed activities, are used consistently. Particular templates such as "ACTION concerning ARTIFACT because of CAUSE" occur frequently. Developers described sessions that last 30-120 min. 4-16 times a day. Maintaining diaries seems to consume between 3-6% of the total work time, and in 10% of the sessions, developers did not describe their work in sufficient detail. We argue that our results make the first step towards automatically generating work diaries for software developers.

show abstract

“…Our analyses show that there are some discrepancies between these measures and the selfreported logs that subjects also kept, although the results point to the instrumented data as being the more accurate source. Our analysis and process for reconciling these differences has been described in some detail elsewhere [7].…”

Section: Threats To Validitymentioning

confidence: 99%

Empirical study design in the area of high-performance computing (HPC)

Shull

Carver²,

Hochstein³

et al. 2005

2005 International Symposium on Empirical Software Engineering, 2005.

View full text Add to dashboard Cite

The development of High-Performance Computing (HPC) programs is crucial to progress in many fields of scientific endeavor. We have run initial studies of the productivity of HPC developers and of techniques for improving that productivity, which have not previously been the subject of significant study. Because of key differences between development for HPC and for more conventional software engineering applications, this work has required the tailoring of experimental designs and protocols.A major contribution of our work is to begin to quantify the code development process in a specialized area that has previously not been extensively studied. Specifically, we present an analysis of the domain of High-Performance Computing for the aspects that would impact experimental design; show how those aspects are reflected in experimental design for this specific area; and demonstrate how we are using such experimental designs to build up a body of knowledge specific to the domain. Results to date build confidence in our approach by showing that there are no significant differences across studies comparing subjects with similar experience tackling similar problems, while there are significant differences in performance and effort among the different parallel models applied.

show abstract

Combining self-reported and automatic data to improve programming effort measurement

Cited by 18 publications

References 14 publications

Experiments to Understand HPC Time to Development (Final report for Department of Energy contract DE-FG02-04ER25633) Report DOE/ER/25633-1

Experiments to Understand HPC Time to Development (Final report for Department of Energy contract DE-FG02-04ER25633) Report DOE/ER/25633-1

From work to word: How do software developers describe their work?

Empirical study design in the area of high-performance computing (HPC)

Contact Info

Product

Resources

About