ArguLens: Anatomy of Community Opinions On Usability Issues Using Argumentation Models

Wang, Wenting; Arya, Deeksha M.; Novielli, Nicole; Cheng, Jinghui; Guo, Jin

doi:10.1145/3313831.3376218

Cited by 29 publications

(30 citation statements)

References 51 publications

(55 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These studies further demonstrated the complexity of issue discussions in the OSS context. Recognizing this complexity, Wang et al [5] proposed the use of an argumentation model, as well as the corresponding automated technique, to help extract and consolidate the diverse topics and opinions that the OSS community members post in ITSs. Our work is built upon these previous studies to further investigate the roles of emotions in the already complex OSS issue discussions.…”

Section: A Oss Issue Discussionmentioning

confidence: 99%

“…On the one hand, OSS communities' heterogeneous nature provides diverse perspectives that help a healthy advancement of the project. On the other hand, consolidating the varied perspectives in a large amount of community-generated requests and discussions is a challenging issue [5]. This challenge is particularly exacerbated by the complex emotions and affective states from the community participants manifested in the issues [6].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Impacts of Sentiments and Tones in Community-Generated Issue Discussions

Sanei¹,

Cheng²,

Adams³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The diverse community members who contribute to the discussions on issue tracking systems of open-source software projects often exhibit complex affective states such as sentiments and tones. These affective states can significantly influence the effectiveness of the issue discussions in elaborating the initial ideas into actionable tasks that the development teams need to address. In this paper, we present an extended empirical study to investigate the impacts of sentiments and tones in community-generated issue discussions. We created and validated a large dataset of sentiments and tones in the issues posts and comments created by diverse community members in three popular open source projects. Our analysis results drew a complex picture of the relationships between, on the one hand, the sentiments and tones in the issue discussions, and on the other hand, various discussion and development-related measures such as the discussion length and the issue resolution time. We also found that when factors such as the issue poster roles and the issue types were controlled, sentiments and tones had varied associations with the measures. Insights gained from these findings can support open source community members in making and moderating effective issue discussions and guide the design of tools to better support community engagement.

show abstract

Section: A Oss Issue Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

The Impacts of Sentiments and Tones in Community-Generated Issue Discussions

Sanei¹,

Cheng²,

Adams³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Currently, many OSS usability practices and discussions take place in an ad-hoc manner and, especially on OSS platforms such as GitHub (www.github.com), within a repository's issue tracking system [2,13,15]. Cheng and Guo [2] found usability specific issue threads on GitHub to be lengthy and contain over-generalized assumptions.…”

Section: D5mentioning

confidence: 99%

“…Due to its affinity for supporting various asynchronous activities, GitHub has enabled a vibrant community for remote collaboration and development of OSS projects. While there are many tools that can potentially be used for monitoring usability interests on GitHub OSS's, such as Issue tracking, there is currently no mature method for end user community collaboration [13,15,16]. Consequently, if any design decisions are made through input from end users, there are difficulties with properly communicating these decisions to other team members due to a lack of design artifacts clearly documenting the process.…”

mentioning

confidence: 99%

Facilitating Asynchronous Participatory Design of Open Source Software: Bringing End Users into the Loop

Jazlyn

Cheng

Guo

2021

Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

As open source software (OSS) becomes increasingly mature and popular, there are significant challenges with properly accounting for usability concerns for the diverse end users. Participatory design, where multiple stakeholders collaborate on iterating the design, can be an efficient way to address the usability concerns for OSS projects. However, barriers such as a code-centric mindset and insufficient tool support often prevent OSS teams from effectively including end users in participatory design methods. This paper proposes preliminary contributions to this problem through the user-centered exploration of (1) a set of design guidelines that capture the needs of OSS participatory design tools, (2) two personas that represent the characteristics of OSS designers and end users, and (3) a low-fidelity prototype tool for end user involvement in OSS projects. This work paves the road for future studies about tool design that would eventually help improve OSS usability. CCS CONCEPTS• Human-centered computing → Participatory design; Computer supported cooperative work; • Software and its engineering → Open source model.

show abstract

“…There are also works that tackled all the levels mentioned by Wambsganss et al (2020) . In Wang et al (2020 ), the authors deal with online discussions about usability on issue-tracking systems of open-source projects. Since a large number of issues and comments with different perspectives are posted daily, the contributors of projects face a major challenge in digesting the rich information embedded in the issue-tracking systems to determine the actual user needs and consolidate the diverse feedback.…”

Section: Background Knowledge and Related Workmentioning

confidence: 99%

Developing a Conversational Agent’s Capability to Identify Structural Wrongness in Arguments Based on Toulmin’s Model of Arguments

Mirzababaei

Pammer-Schindler

2021

Front. Artif. Intell.

View full text Add to dashboard Cite

This article discusses the usefulness of Toulmin’s model of arguments as structuring an assessment of different types of wrongness in an argument. We discuss the usability of the model within a conversational agent that aims to support users to develop a good argument. Within the article, we present a study and the development of classifiers that identify the existence of structural components in a good argument, namely a claim, a warrant (underlying understanding), and evidence. Based on a dataset (three sub-datasets with 100, 1,026, 211 responses in each) in which users argue about the intelligence or non-intelligence of entities, we have developed classifiers for these components: The existence and direction (positive/negative) of claims can be detected a weighted average F1 score over all classes (positive/negative/unknown) of 0.91. The existence of a warrant (with warrant/without warrant) can be detected with a weighted F1 score over all classes of 0.88. The existence of evidence (with evidence/without evidence) can be detected with a weighted average F1 score of 0.80. We argue that these scores are high enough to be of use within a conditional dialogue structure based on Bloom’s taxonomy of learning; and show by argument an example conditional dialogue structure that allows us to conduct coherent learning conversations. While in our described experiments, we show how Toulmin’s model of arguments can be used to identify structural problems with argumentation, we also discuss how Toulmin’s model of arguments could be used in conjunction with content-wise assessment of the correctness especially of the evidence component to identify more complex types of wrongness in arguments, where argument components are not well aligned. Owing to having progress in argument mining and conversational agents, the next challenges could be the developing agents that support learning argumentation. These agents could identify more complex type of wrongness in arguments that result from wrong connections between argumentation components.

show abstract

ArguLens: Anatomy of Community Opinions On Usability Issues Using Argumentation Models

Abstract: facilitated the review of lengthy issues. ArguLens provides the direction of designing valuable tools for high-level reasoning and effective discussion about usability.

Cited by 29 publications

References 51 publications

The Impacts of Sentiments and Tones in Community-Generated Issue Discussions

The Impacts of Sentiments and Tones in Community-Generated Issue Discussions

Facilitating Asynchronous Participatory Design of Open Source Software: Bringing End Users into the Loop

Developing a Conversational Agent’s Capability to Identify Structural Wrongness in Arguments Based on Toulmin’s Model of Arguments

Contact Info

Product

Resources

About