Assessing Competencies Needed to Engage With Digital Health Services: Development of the eHealth Literacy Assessment Toolkit

Abstract

BackgroundTo achieve full potential in user-oriented eHealth projects, we need to ensure a match between the eHealth technology and the user’s eHealth literacy, described as knowledge and skills. However, there is a lack of multifaceted eHealth literacy assessment tools suitable for screening purposes.ObjectiveThe objective of our study was to develop and validate an eHealth literacy assessment toolkit (eHLA) that assesses individuals’ health literacy and digital literacy using a mix of existing and newly developed scales.MethodsFrom 2011 to 2015, scales were continuously tested and developed in an iterative process, which led to 7 tools being included in the validation study. The eHLA validation version consisted of 4 health-related tools (tool 1: “functional health literacy,” tool 2: “health literacy self-assessment,” tool 3: “familiarity with health and health care,” and tool 4: “knowledge of health and disease”) and 3 digitally-related tools (tool 5: “technology familiarity,” tool 6: “technology confidence,” and tool 7: “incentives for engaging with technology”) that were tested in 475 respondents from a general population sample and an outpatient clinic. Statistical analyses examined floor and ceiling effects, interitem correlations, item-total correlations, and Cronbach coefficient alpha (CCA). Rasch models (RM) examined the fit of data. Tools were reduced in items to secure robust tools fit for screening purposes. Reductions were made based on psychometrics, face validity, and content validity.ResultsTool 1 was not reduced in items; it consequently consists of 10 items. The overall fit to the RM was acceptable (Anderson conditional likelihood ratio, CLR=10.8; df=9; P=.29), and CCA was .67. Tool 2 was reduced from 20 to 9 items. The overall fit to a log-linear RM was acceptable (Anderson CLR=78.4, df=45, P=.002), and CCA was .85. Tool 3 was reduced from 23 to 5 items. The final version showed excellent fit to a log-linear RM (Anderson CLR=47.7, df=40, P=.19), and CCA was .90. Tool 4 was reduced from 12 to 6 items. The fit to a log-linear RM was acceptable (Anderson CLR=42.1, df=18, P=.001), and CCA was .59. Tool 5 was reduced from 20 to 6 items. The fit to the RM was acceptable (Anderson CLR=30.3, df=17, P=.02), and CCA was .94. Tool 6 was reduced from 5 to 4 items. The fit to a log-linear RM taking local dependency (LD) into account was acceptable (Anderson CLR=26.1, df=21, P=.20), and CCA was .91. Tool 7 was reduced from 6 to 4 items. The fit to a log-linear RM taking LD and differential item functioning into account was acceptable (Anderson CLR=23.0, df=29, P=.78), and CCA was .90.ConclusionsThe eHLA consists of 7 short, robust scales that assess individual’s knowledge and skills related to digital literacy and health literacy.

Health literacy, digital literacy and eHealth literacy in Danish nursing students at entry and graduate level: a cross sectional study

“…The eHLA toolkit was developed in the period of 2011 to 2015 where the scales were continuously tested and developed in an iterative process to ensure content and face validity [21]. The toolkit consists of 44 items grouped into 7 tools.…”

“…Tools 3,4 and 7 have been developed from scratch. All tools have been thoroughly explored and validated using modern test theory [21]. The instrument is developed in Danish.…”

“…The introduction of multi-facetted instruments to measure HL, DL, and eHL creates new opportunities to obtain better insight into nursing students' competences. In this study, we used the HLQ [20], eHealth Literacy Assessment (eHLA) toolkit [21], and eHealth Literacy Questionnaire (eHLQ) [22] to measure HL, DL and eHL respectively. The HLQ was selected as it recently has been used in a global initiative to measure HL in various regions of the world, primarily among nursing students [23].…”

Measures of eHealth Literacy: Options for the Malaysian Population

“…At the same time that the eHLQ was being developed, another eHealth literacy instrument was also being produced by Karnoe et al (2018) -the eHealth Literacy Assessment toolkit (eHLA). This instrument is discussed in the next section.…”

“…The eHealth Literacy Assessment Toolkit (eHLA) by Karnoe et al (2018) was published only a short while after the eHLQ by Kayser et al (2018). Although both instruments are intended to measure eHealth literacy and therefore share similar domains, the development of the eHLA began before the eHLF was introduced and before the eHLQ was initiated (see Karnoe & Kayser, 2016).…”