Usability evaluation for enterprise SOA APIs

Beaton, Jack; Myers, Brad A.; Stylos, Jeffrey; Jeong, Sae Young; Xie, Yingyu

doi:10.1145/1370916.1370924

Cited by 26 publications

(19 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…An (old) overview of the project is [1]. There are many papers about the Whyline [2,3], the study of hard-to-answer questions [4,5], the visualization tool to help with reverse engineering [6], some studies of APIs [7][8][9][10], and tools to help with APIs [11][12][13][14][15].…”

Section: For More Informationmentioning

confidence: 99%

Improving program comprehension by answering questions (keynote)

Myers

2013

2013 21st International Conference on Program Comprehension (ICPC)

Self Cite

View full text Add to dashboard Cite

Abstract-My Natural Programming Project is working on making software development easier to learn, more effective, and less error prone. An important focus over the last few years has been to discover what are the hard-to-answer questions that developers ask while they are trying to comprehend their programs, and then to develop tools to help answer those questions. For example, when studying programmers working on everyday bugs, we found that they continuously ask "Why" and "Why Not" questions as they try to comprehend what happened. We developed the "Whyline" debugging tool, which allows programmers to directly ask these questions of their programs and get a visualization of the answers. In a small lab study, Whyline increased productivity by a factor of about two. We studied professional programmers trying to understand unfamiliar code and identified over 100 questions they identified as hard-to-answer. In particular, we saw that programmers frequently had specific questions about the feasible execution paths, so we developed a new visualization tool to directly present this information. When trying to use unfamiliar APIs, such as the Java SDK and the SAP eSOA APIs, we discovered some common patterns that make programmers up to 10 times slower in finding and understanding how to use the appropriate methods, so we developed new tools to assist them. This talk will provide an overview of our studies and resulting tools that address program comprehension issues.

show abstract

Section: For More Informationmentioning

confidence: 99%

Improving program comprehension by answering questions (keynote)

Myers

2013

2013 21st International Conference on Program Comprehension (ICPC)

Self Cite

View full text Add to dashboard Cite

show abstract

“…In a similar way, Heer et al [9] analyzed the usability of their prefuse toolkit. An interesting alteration of this approach was proposed by Beaton et al [2]. Participants first have to write in pseudo code what they would expect in the API for a certain task and then perform the real task using the API.…”

Section: Challenges For the Evaluation Of An Apimentioning

confidence: 99%

Using concept maps to evaluate the usability of APIs

Gerken

Jetter

Reiterer

2010

CHI '10 Extended Abstracts on Human Factors in Computing Systems

View full text Add to dashboard Cite

Application programming interfaces (APIs) are the interfaces to existing code structures, such as widgets, frameworks, or toolkits. Therefore, they very much do have an impact on the quality of the resulting system. So ensuring that developers can make the most out of them is an important challenge. However standard usability evaluation methods as known from HCI have limitations in grasping the interaction between developer and API -the GUI, which makes this interaction obvious, is missing. In this paper we present a longitudinal approach using concept maps and a question diary to make this interaction visible and study the usability of an API over time. KeywordsAPI usability, longitudinal evaluation, concept maps. ACM Classification Keywords H5.2. [Information Interfaces and Presentation]: UserInterfaces. General TermsExperimentation, Human Factors. IntroductionDeveloping a software system nowadays hardly means programming everything from scratch. Instead, developers can often rely on existing widgets, Copyright is held by the author/owner(s). harald.reiterer@uni-konstanz.de frameworks, libraries, or software development toolkits that provide existing code structure for reuse. To access these, application programming interfaces are provided (APIs) and while there may be many different kinds of APIs they all serve the same purpose, as Daughtry et al. [4] described it: "they each provide a programmatic user-interface to a module of code". As with any kind of interface, some of them are more usable than others and so in recent years, the study of the usability of APIs has been emphasized by more and more researchers [e.g. 4,5]. We can identify two main goals in such studies. One is to analyze the usage of APIs on a more general level in order to derive design principles for the creation of new APIs or the modification of existing ones. The second is to evaluate the usability of one specific API, preferably during its development process as part of a user-centered iterative lifecycle. In this paper we will primarily focus on the latter part by presenting a longitudinal evaluation method that can be used to assess the barriers developers come across when trying to use an API as well as their evolution over time. Challenges for the Evaluation of an APIEvaluating an API is quite different from standard usability evaluation. The most important aspect is the missing GUI, so using and interacting with an API is much more subtle than using a standard software application and therefore more difficult to observe and analyze. Accordingly, it is not straight forward to define wrong doings or errors during the observation of users since there are many ways to reach a goal. From a methodological point of view, the most common approaches are lab based usability tests in combination with thinking aloud. So for example Klemmer et al. [10] presented such a usability study with seven participants using the Papier-Mâché Toolkit for developing tangible user interfaces. Participants were first introduced to the toolkit and th...

show abstract

“…Our previous research has shown that programming using eSOA APIs is not simple if APIs are providing access to powerful business functionality [1,2]. Some barriers we identified included long names for services, different behaviors of services due to different business behavior, parameters of the services as hierarchies of objects with complex dependencies driven by internal, not exposed, business logic, and lack of example code [1,2].…”

Section: Introductionmentioning

confidence: 99%

“…Some barriers we identified included long names for services, different behaviors of services due to different business behavior, parameters of the services as hierarchies of objects with complex dependencies driven by internal, not exposed, business logic, and lack of example code [1,2]. The current paper presents the results of a new user study of the usability of the online documentation provided by SAP for their SOA product.…”

Section: Introductionmentioning

confidence: 99%

Improving Documentation for eSOA APIs through User Studies

Jeong

Xie

Beaton

et al. 2009

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. All software today is written using libraries, toolkits, frameworks and other application programming interfaces (APIs). We performed a user study of the online documentation a large and complex API for Enterprise ServiceOriented Architecture (eSOA), which identified many issues and recommendations for making API documentation easier to use. eSOA is an appropriate testbed because the target user groups range from high-level business experts who do not have significant programming expertise (and thus are endparticipant developers), to professional programmers. Our study showed that the participants' background influenced how they navigated the documentation. Lack of familiarity with business terminology was a barrier we observed for developers without business application experience. Participants with business software experience had difficulty differentiating similarly named services. Both groups avoided areas of the documentation that had an inconsistent visual design. A new design for the documentation that supports flexible navigation strategies seem to be required to support the wide range of users for eSOA. This paper summarizes our study and provides recommendations for future documentation for developers.

show abstract

Usability evaluation for enterprise SOA APIs

Cited by 26 publications

References 9 publications

Improving program comprehension by answering questions (keynote)

Improving program comprehension by answering questions (keynote)

Using concept maps to evaluate the usability of APIs

Improving Documentation for eSOA APIs through User Studies

Contact Info

Product

Resources

About