Streamlining the Design Process: Running Fewer Subjects

Virzi, Robert A.

doi:10.1177/154193129003400411

Cited by 72 publications

(51 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Nielsen (1993Nielsen ( , 1994; Lewis (1994) and Virzi (1990Virzi ( , 1992 argue that five or fewer subjects are sufficient to identify most of the usability problems found in a system. Virzi (1990Virzi ( , 1992 showed that this could account for around 80% of the problems. However, other studies recommended using more subjects.…”

Section: Recruitment Of Subjectsmentioning

confidence: 99%

An overview of semantic search evaluation initiatives

Elbedweihy

Wrigley

Clough

et al. 2015

Journal of Web Semantics

View full text Add to dashboard Cite

Section: Recruitment Of Subjectsmentioning

confidence: 99%

An overview of semantic search evaluation initiatives

Elbedweihy

Wrigley

Clough

et al. 2015

Journal of Web Semantics

View full text Add to dashboard Cite

“…The first studies in this area mostly focused on determining the cost-benefit of web interface analysis by estimating the return on investment to justify the cost of usability assessment [24]. In line with this aim, researchers in the 1990s proposed a specific rule of thumb arising from the results of Virzi [23,25] and Nielsen [26][27][28][29]. This rule, known as the five-user assumption, proposes a one-size-fits-all solution in which five users are considered enough for reliable usability testing.…”

Section: Usability Testing Of Medical Devices: Estimating the Sample mentioning

confidence: 99%

“…The estimation of the final number of users for an evaluation sample can be calculated by inserting the p-value into the following well-known error distribution formula [22,23,25,29,31]:…”

Section: Usability Testing Of Medical Devices: Estimating the Sample mentioning

confidence: 99%

How many testers are needed to assure the usability of medical devices?

Borsci

Macredie

Martin

2014

Expert Review of Medical Devices

View full text Add to dashboard Cite

A note on versions:The version presented here may differ from the published version or from the version of record. If you wish to cite this item you are advised to consult the publisher's version. Please see the repository url above for details on accessing the published version and note that access may require a subscription.  The assessment of usability and its place in the management of product safety is an increasingly important aspect of medical device development. Medical device manufacturers, especially in small companies, do not have sufficient expertise or knowledge about usability and human factors and it is hard for them suitably to address all of the steps required by relevant standards. Despite a rigorous framework for designing and assessing a product, an increasing number of medical devices are recalled each year because of safety concerns. There is substantial evidence that testing a device with mandated sample sizes could lead manufacturers to evaluate a product without having real control over the reliability of the assessment. A new approach to usability data management applied in this paper, called the 'Grounded Procedure', drives manufacturers to estimate the sample size needed to identify a given proportion of interaction problems and to inform critical product decisions. Using the Grounded Procedure could enable manufacturers to increase the usability and the safety of their medical devices and help practitioners to check the representativeness of the evaluation cohorts, to analyse significance of specific usability problems, and to rethink the user selection criteria for validation testing. Summary 3Before releasing a product, manufacturers have to follow a regulatory framework and meet standards, producing reliable evidence that the device presents low levels of risk in use. There is, though, a gap between the needs of the manufacturers to conduct usability testing whilst managing their costs, and the requirements of authorities for representative evaluation data. A key issue here is the number of users that should complete this evaluation to provide confidence in a product's safety. This paper reviews the FDA's indication that a sample composed of 15 participants per major group (or a minimum of 25 users) should be enough to identify 90-97% of the usability problems and argues that a more nuanced approach to determining sample size (which would also fit well with the FDA's own concerns expressed in [1]) would be beneficial. The paper will show that there is no a priori cohort size that can guarantee a reliable assessment, a point stressed by the FDA in the appendices to its guidance, but that manufacturers can terminate the assessment when appropriate by using a specific approach -illustrated in this paper through a case study -called the 'Grounded Procedure'.4

show abstract

“…According to [17], [31], it is possible, with 4 or 5 participants, to detect nearly 80 to 85% of the utilisability problems. Hence, for this first evaluation of the IAS in laboratory, the population is made up of five subjects.…”

Section: Population Implied In the Evaluationmentioning

confidence: 99%

Evaluation of an Information Assistance System Based on an Agent-based Architecture in Transportation Domain: First Results

Trabelsi

Ezzeddine

2013

INT J COMPUT COMMUN

View full text Add to dashboard Cite

Abstract:The evaluation of interactive systems is a wide and rich research domain: many methods, criteria and tools are available. In this article, we are focused on agent-based interactive systems. We first describe the agent oriented architecture used in our researches. Then we propose an evaluation approach based on three complementary techniques: assistance evaluation tool, questionnaire, and verbalization. The validation of our approach occurred within the framework of a project involving an industrial partner which is running the current urban transport network (tramway and bus) in the town of Valenciennes, France. The main results of the evaluation of an agent-based Information Assistance System (IAS) are presented. This evaluation is based on two scenarios: normal or distrupted running mode. This evaluation has been conducted in laboratory.

show abstract

Streamlining the Design Process: Running Fewer Subjects

Cited by 72 publications

References 3 publications

An overview of semantic search evaluation initiatives

An overview of semantic search evaluation initiatives

How many testers are needed to assure the usability of medical devices?

Evaluation of an Information Assistance System Based on an Agent-based Architecture in Transportation Domain: First Results

Contact Info

Product

Resources

About