Supporting cognitive walkthrough with video data

Gabrielli, Silvia; Mirabella, Valeria; Kimani, Stephen; Catarci, Tiziana

doi:10.1145/1085777.1085791

Cited by 10 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In one study, experts who conducted cognitive walkthroughs supplemented with video found as many problems as experts who conducted a cognitive walkthrough in situ. Both the video and in situ groups found more problems than experts who conducted a traditional cognitive walkthrough (Gabrielli et al, 2005). This suggests that the drawbacks of prospective usability evaluation may be mitigated using experienced participants and augmenting the evaluation with visual stimuli.…”

Section: Prospective Usability Evaluationmentioning

confidence: 96%

Developing and Validating Measurement Scales During Pandemic Conditions: A Case Study with the Scale for Habitat Usability

Robertson

Munson

Stephenson

et al. 2022

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

At NASA, habitat evaluations often employ subjective measures. Some measures are frequently used, well-established tools, whereas others are homegrown measures tailored to specific projects. The variety of measures used makes evaluation comparisons across projects difficult. Additionally, some of these measures are burdensome, may be too specialized, or may require an expert to use and interpret, limiting their utility. Taken together, these drawbacks suggest the need for a new measurement tool. To that purpose, a team at NASA worked on developing a new scale for measuring habitat usability, the Scale for Habitat Usability (SHU). The SHU is intended to be a quick, multi-faceted measure for evaluating habitat usability across the development lifecycle. However, like many research projects, the development of the SHU faced setbacks due to the COVID-19 pandemic. Pandemic prevention protocols precluded in-person data collection, forcing the team to take some non-traditional approaches to scale development. This paper reports the steps the team took to complete the project.

show abstract

Section: Prospective Usability Evaluationmentioning

confidence: 96%

Developing and Validating Measurement Scales During Pandemic Conditions: A Case Study with the Scale for Habitat Usability

Robertson

Munson

Stephenson

et al. 2022

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

show abstract

“…Watching Others Using Video has been used to evaluate workload (Hale & Long, 2017), support cognitive walkthroughs (Gabrielli et al, 2005), and execute heuristic evaluations of robot interactions (Weiss et al, 2010). Though advantageous, previous work found that Watching Others Using Video results in positively biased usability ratings compared to Use-Then-Measure (Robertson & Kortum, 2019).…”

Section: Watching Others Using Videomentioning

confidence: 99%

Rating Product Satisfaction by Watching Others Using Video

Shanklin

Kortum

Robertson

2022

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

During the COVID-19 pandemic, usability practitioners and researchers had to find new approaches to product testing. In-person, contact-intensive product testing became a safety concern, resulting in the need for more remote testing practices. An underexplored and promising method to capture subjective usability measures is Watching Others Using Video, wherein users rate a product after watching videos of others using it. This can have broad application, but previous research found this method yields positively biased usability ratings relative to post-use ratings. This study explored potential factors (e.g., success, error, and failure) that may impact how users perceive satisfaction using this method. To do so, participants were shown videos with different product interactions while systematically varying the factors of interest. Additionally, the effects of the number of errors and error recovery versus failure were explored. Participants watched different videos of the following products being used and rated them using the After-Scenario Questionnaire (ASQ): a website, electric can opener, digital timer. Results found inflated satisfaction ratings across products, however the effect did not reach statistical significance for the website. There also was no observable effect of increasing errors or showing failures. This may be attributed to poor error detection or negligible error severities. Further research is needed for Watching Others Using Video to be accurately implemented as a viable testing method.

show abstract

“…In the recent past, video data has been used to support an expert-based evaluation of an e-learning course that was delivered on a mobile device [28].…”

Section: Evaluation Challenges In Mobile Computingmentioning

confidence: 99%

A unified methodology for the evaluation of accessibility and usability of mobile applications

Billi

Burzagli

Catarci

et al. 2010

Univ Access Inf Soc

Self Cite

View full text Add to dashboard Cite

This article reports a unified methodology developed to evaluate the accessibility and usability of mobile computing applications, which is intended to guarantee universal access as far as possible. As a basis for the methodology, this paper presents an analysis of the accessibility guidelines, conducted to take into account the specificity of mobile systems, as well as a set of usability heuristics, specifically devised for mobile computing. Finally, it presents the results of the application of the proposed methodology to applications that have been semiautomatically developed by the MAIS Designer, a new design tool that provides applications suited to different mobile devices

show abstract

Supporting cognitive walkthrough with video data

Cited by 10 publications

References 5 publications

Developing and Validating Measurement Scales During Pandemic Conditions: A Case Study with the Scale for Habitat Usability

Developing and Validating Measurement Scales During Pandemic Conditions: A Case Study with the Scale for Habitat Usability

Rating Product Satisfaction by Watching Others Using Video

A unified methodology for the evaluation of accessibility and usability of mobile applications

Contact Info

Product

Resources

About