An Acceptability Predictor for Websites

Abstract

Abstract.User acceptance is a high priority for website design and implementation.Two significant, but largely separate, approaches to acceptability are: First, the Web Accessibility Initiative (WAI) has explored the measurement of technical features of a website to gauge its accessibility. Second, human judgments about acceptability are obtained from intended users or experts. The present work explores the important question of how best to combine these two methods. Experiment One required new users to explore automatic website evaluation systems. They found two of four systems difficult or impossible to use and system outputs difficult to understand. Experiment Two combines formal properties and user judgments, using an automatic system to predict user judgments from formal website properties. A simple system was able to predict user judgments within 91% accuracy. Clearly, user judgments about websites can be predicted reliably, a result of value to designers.

Aggregating predictions from experts: A review of statistical methods, experiments, and applications

“…Political articles focused on predicting presidential outcomes, a categorical target (Graefe, 2015, 2018; Graefe, Armstrong, Jones Jr, & Cuzán, 2014a, 2014b; Hurley & Lior, 2002; Morgan, 2014). Risk‐related targets were continuous and categorical: the probability of structural damage, nuclear fallout, occupational hazards, and balancing power load (Adams et al, 2009; Baecke et al, 2017; Brito et al, 2012; Brito & Griffiths, 2016; Cabello et al, 2012; Craig et al, 2001; Hathout et al, 2016; Jana et al, 2019; Kläs et al, 2010; Mu & Xianming, 1999; Neves & Frangopol, 2008; Ren‐jun & Xian‐zhong, 2002; Wang et al, 2008; Zio, 1996; Zio & Apostolakis, 1997). Public health papers predicted continuous targets over time, like forecasting carcinogenic risk (Evans et al, 1994) and US mortality rates (Alho, 1992).…”

“…These fields focus on decision makers and their ability to make predictions from data that cannot easily be collected and fed to a statistical model. Other areas of active research in expert opinion are quality assurance (Kläs et al, 2010), politics (Cai, Lin, Han, Liu, & Zhang, 2016; Graefe, 2015, 2018; Graefe, Armstrong, Jones Jr, & Cuzán, 2014a, 2014b; Hanea, McBride, Burgman, & Wintle, 2018; Satopää, Jensen, Mellers, Tetlock, & Ungar, 2014; Wang & Zhang, 2018), economics (Huang et al, 2016; Mak et al, 1996; Shin et al, 2013), engineering (Brito et al, 2012; Brito & Griffiths, 2016; Craig et al, 2001; Hathout, Vuillet, Peyras, Carvajal, & Diab, 2016; Jin, Lu, & Gan, 2007; Neves & Frangopol, 2008; Ren‐jun & Xian‐zhong, 2002; Tartakovsky, 2007; Wang, Du, & Cao, 2008; Zio, 1996), sports (Gu, Saaty, & Whitaker, 2016), sociology (Adams, White, & Ceylan, 2009; Cabello et al, 2012), meteorological (Abramson, Brown, Edwards, Murphy, & Winkler, 1996), ecological (Borsuk, 2004; Cooke et al, 2014; Failing et al, 2004; Johnson et al, 2018), environmental science (Li, Liu, & Yang, 2012; Mantyka‐Pringle, Martin, Moffatt, Linke, & Rhodes, 2014; Morales‐Nápoles et al, 2017; Zio & Apostolakis, 1997), and public health (Alho, 1992; Evans et al, 1994; Jana, Pramanik, Sahoo, & Mukherjee, 2019; Kurowicka, Bucura, Cooke, & Havelaar, 2010). The diversity and breadth of applications underscore the importance of expert opinion in a wide variety of disciplines.…”

Aggregating expert-elicited data for prediction: A scoping review of statistical methods, experiments, and applications v1

“…Authors more often proposed combining expert-generated point estimates compared to predictive distributions. A diverse set of models were proposed to combine point estimates: regression models (linear regression, logistic regression, ARIMA, exponential smoothing), simple averaging, and neural networks (Cabello et al, 2012;Adams et al, 2009;Mak et al, 1996;Graefe et al, 2014b;, and fuzzy logic Petrovic et al (2006); Kabak and Ulengin (2008); Jana et al (2019); Ren-jun and Xian-zhong (2002). Authors that combined predictive densities focused on simpler combination models.…”

“…These fields focus on decision makers and their ability to make predictions from data that cannot easily be collected and fed to a statistical model. Other areas of active research in expert opinion are quality assurance , politics ; Graefe et al (2014a); Graefe (2015Graefe ( , 2018; Cai et al (2016); Wang and Zhang (2018); Satopää et al (2014); Graefe et al (2014b), economics Shin et al (2013); Huang et al (2016); Mak et al (1996), engineering Craig et al (2001; Tartakovsky (2007); Neves and Frangopol (2008); Zio (1996a); Brito and Griffiths (2016a); Jin et al (2007); Wang et al (2008); ; ; Ren-jun and Xianzhong (2002), sports Gu et al (2016), sociology Cabello et al (2012); Adams et al (2009), meteorological Abramson et al (1996), ecological Johnson et al (2018; Borsuk (2004);Failing et al (2004); , environmental science ; Mantyka-Pringle et al (2014); Li et al (2012); Zio and Apostolakis (1997), and public health Alho (1992); Evans et al (1994b); Jana et al (2019); Kurowicka et al (2010). The diversity and breadth of applications underscore the importance of expert opinion in a wide variety of disciplines.…”

Development of a web-based recipe database for use in promotion of the Mediterranean diet among older adults in the UK

“…Participants were also encouraged to submit to the website their own recipes. After 2 weeks participants returned for an assessment session in which they assessed the acceptability of the website using a standardised questionnaire ( 1 ) , and for group feedback discussion about the users' experience.…”