ECOVAL: Ecological Validity of Cues and Representative Design in User Experience Evaluations

Kieffer, Suzanne

doi:10.17705/1thci.00093

Cited by 17 publications

(6 citation statements)

References 18 publications

(33 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Affordances' complex and situational nature (Volkoff & Strong, 2013;Volkoff & Strong, 2018) explains why the consequences of a user's interaction with an artifact in the usage world may differ from what designers envision in the design world (Hassenzahl, 2003;Kieffer, 2017). Therefore, it is crucial to collect information about the user experience of designed artifacts in the usage world (Djamasbi, 2014;Albert & Tullis, 2013).…”

Section: Figure 3 Ux-driven Innovation (Uxdi) Conceptualization In Traditional Settingsmentioning

confidence: 99%

User Experience-driven Innovation in Smart and Connected Worlds

Djamasbi¹,

Strong²

2019

THCI

View full text Add to dashboard Cite

In our fast-paced digital economy, expectations for improved user experiences (UX) increasingly drive innovation. Thus, companies must fully grasp users' points of view when they design innovative technologies that can successfully compete in a crowded global market. These technologies must not only satisfy users' expectations but also empower them and improve their quality of life. To address this challenge in our rapidly evolving and globally expanding digital economy, we need new theories and models for technology design-ones that incorporate UX. In this paper, we address this need by developing conceptual models for UX-driven innovation. We explain how these models can enable innovative, responsive technologies that meet users' needs in real time. These models also facilitate the production of new theories that are discovered via accessing the rich, real-time data sets that our increasingly smart and connected worlds create.

show abstract

Section: Figure 3 Ux-driven Innovation (Uxdi) Conceptualization In Traditional Settingsmentioning

confidence: 99%

User Experience-driven Innovation in Smart and Connected Worlds

Djamasbi¹,

Strong²

2019

THCI

View full text Add to dashboard Cite

show abstract

“…We used real-life, detailed vignette descriptions of carers’ circumstances and these were able to prompt responses that mimic the decision-making processes of assessments. This makes it possible to draw conclusions that have ecological validity [19] and can be more realistically generalised to inform practice and policy. There is no ‘gold standard’ by which to judge the reliability of assessments, particularly for assessments that have a legislative basis, such as these carer assessments in the UK.…”

Section: Discussionmentioning

confidence: 99%

Judgements about carer assessments for carers of people with dementia: case vignette study

Clarkson,

Mills,

Butt

et al. 2024

Preprint

View full text Add to dashboard Cite

Objectives UK carer assessments, in primary and social care, intend to discover what carers need in their caring roles and more widely. Evidence points to these not being configured sufficiently around carers of people with dementia, with potentially their breadth of needs not being recognised. We evaluated the extent of agreement, between carers of people with dementia, primary care, and social care professionals, on their recommendations from assessing carers’ needs in a range of circumstances. It is intended for findings to be taken forward as recommendations for policy and practice. Methods Comparison of judgements, between carers, primary and social care professionals, on whether real-life circumstances in 9 anonymised case vignettes necessitated a range of 14 services to support carers appropriately. Participants were 6 carers of people with dementia, 7 primary care staff, and 2 social care staff. We presented participants with each vignette and asked them to make binary judgements of whether they would recommend a range of services in each case. Percentage agreement and Fleiss’ kappa coefficients measured the level of agreement amongst multiple carers, primary and social care staff and overall. These agreements were then compared. Results Carers agreed in their judgements more than primary or social care professionals. The overall level of agreement from judgements made by all participants, however, was ‘slight’ with variability between participant groups and overall. The need for First Language Support in some cases was recognised, an improvement from previous evidence. Conclusions Case vignettes are useful for investigating judgements concerning these carers’ needs, so raising issues for policy and practice. It is essential for carer assessments to be more reliable in recommending services based on need to ensure less variability, depending on assessor and carers circumstances.

show abstract

“…The methodology is grounded in the interdisciplinary research methods compendium of HCI (see an overview in [ 55 ] and follows an equal-status sequential multi-phase mixed-methods approach [ 99 ] with qualitative methods complementing quantitative results for each of the three research phases: (quan qual) (quan qual) (quan qual). Within the available “scientific design space of HCI” [ 87 ], a method was tailored to investigate the research questions with a balanced goal of internal validity and external validity [ 50 ]. Internal validity was accounted for through an experimental design, and external validity was considered by the definition of a concrete and realistic usage scenario, by the participation of expert practitioners from the construction domain (see Sect. "…”

Section: Methodsmentioning

confidence: 99%

Effects of reliability indicators on usage, acceptance and preference of predictive process management decision support systems

et al. 2022

View full text Add to dashboard Cite

Despite the growing availability of data, simulation technologies, and predictive analytics, it is not yet clear whether and under which conditions users will trust Decision Support Systems (DSS). DSS are designed to support users in making more informed decisions in specialized tasks through more accurate predictions and recommendations. This mixed-methods user study contributes to the research on trust calibration by analyzing the potential effects of integrated reliability indication in DSS user interfaces for process management in first-time usage situations characterized by uncertainty. Ten experts specialized in digital tools for construction were asked to test and assess two versions of a DSS in a renovation project scenario. We found that while users stated that they need full access to all information to make their own decisions, reliability indication in DSS tends to make users more willing to make preliminary decisions, with users adapting their confidence and reliance to the indicated reliability. Reliability indication in DSS also increases subjective usefulness and system reliability. Based on these findings, it is recommended that for the design of reliability indication practitioners consider displaying a combination of reliability information at several granularity levels in DSS user interfaces, including visualizations, such as a traffic light system, and to also provide explanations for the reliability information. Further research directions towards achieving trustworthy decision support in complex environments are proposed.

show abstract

ECOVAL: Ecological Validity of Cues and Representative Design in User Experience Evaluations

Cited by 17 publications

References 18 publications

User Experience-driven Innovation in Smart and Connected Worlds

User Experience-driven Innovation in Smart and Connected Worlds

Judgements about carer assessments for carers of people with dementia: case vignette study

Effects of reliability indicators on usage, acceptance and preference of predictive process management decision support systems

Contact Info

Product

Resources

About