Initial assessment of reliability of a self-administered web-based neuropsychological test battery

Hansen, Tor Ivar; Lehn, Hanne; Evensmoen, Hallvard Røe; Håberg, Asta

doi:10.1016/j.chb.2016.05.025

Cited by 30 publications

(22 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To allow the comparison with test-retest reliabilities commonly reported in the literature [ 3 , 5 , 18 , 42 ], bivariate coefficients were computed to measure the strength of the linear association of outcome measures across test settings. Spearman rank correlations are reported because of the nonnormal distribution of data.…”

Section: Methodsmentioning

confidence: 99%

“…Computerized testing platforms and widespread access to fast and affordable internet has the potential to bring neuropsychological assessment into people’s homes [ 2 - 4 ]. Web-based neuropsychological assessments could help to meet increasing demands in clinical and cohort studies [ 3 , 5 ]: providing access to large samples, allowing fine-grained phenotyping of complex clinical conditions, facilitating access to patients and participants in remote areas or those with mobility problems, enhancing coordination of data collection across multiple sites, assisting in monitoring of patients with chronic or progressive neurological diseases, and enabling cost-effective screening for clinical trials.…”

Section: Introductionmentioning

confidence: 99%

“…Web-based automated assessments are inexpensive, are quick to conduct, and provide fewer restrictions on timing and location [ 2 , 5 - 7 ]. Evidence suggests that broadly targeted web-based assessments allow the recruitment of samples that are reasonably representative in terms of personality and adjustment characteristics and are more diverse than traditionally recruited samples in terms of geographical location, gender, and socioeconomic status [ 7 ].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Comparing Web-Based and Lab-Based Cognitive Assessment Using the Cambridge Neuropsychological Test Automated Battery: A Within-Subjects Counterbalanced Study

Backx¹,

Skirrow²,

Dente³

et al. 2020

J Med Internet Res

View full text Add to dashboard Cite

Background Computerized assessments are already used to derive accurate and reliable measures of cognitive function. Web-based cognitive assessment could improve the accessibility and flexibility of research and clinical assessment, widen participation, and promote research recruitment while simultaneously reducing costs. However, differences in context may influence task performance. Objective This study aims to determine the comparability of an unsupervised, web-based administration of the Cambridge Neuropsychological Test Automated Battery (CANTAB) against a typical in-person lab-based assessment, using a within-subjects counterbalanced design. The study aims to test (1) reliability, quantifying the relationship between measurements across settings using correlational approaches; (2) equivalence, the extent to which test results in different settings produce similar overall results; and (3) agreement, by quantifying acceptable limits to bias and differences between measurement environments. Methods A total of 51 healthy adults (32 women and 19 men; mean age 36.8, SD 15.6 years) completed 2 testing sessions, which were completed on average 1 week apart (SD 4.5 days). Assessments included equivalent tests of emotion recognition (emotion recognition task [ERT]), visual recognition (pattern recognition memory [PRM]), episodic memory (paired associate learning [PAL]), working memory and spatial planning (spatial working memory [SWM] and one touch stockings of Cambridge), and sustained attention (rapid visual information processing [RVP]). Participants were randomly allocated to one of the two groups, either assessed in-person in the laboratory first (n=33) or with unsupervised web-based assessments on their personal computing systems first (n=18). Performance indices (errors, correct trials, and response sensitivity) and median reaction times were extracted. Intraclass and bivariate correlations examined intersetting reliability, linear mixed models and Bayesian paired sample t tests tested for equivalence, and Bland-Altman plots examined agreement. Results Intraclass correlation (ICC) coefficients ranged from ρ=0.23-0.67, with high correlations in 3 performance indices (from PAL, SWM, and RVP tasks; ρ≥0.60). High ICC values were also seen for reaction time measures from 2 tasks (PRM and ERT tasks; ρ≥0.60). However, reaction times were slower during web-based assessments, which undermined both equivalence and agreement for reaction time measures. Performance indices did not differ between assessment settings and generally showed satisfactory agreement. Conclusions Our findings support the comparability of CANTAB performance indices (errors, correct trials, and response sensitivity) in unsupervised, web-based assessments with in-person and laboratory tests. Reaction times are not as easily translatable from in-person to web-based testing, likely due to variations in computer hardware. The results underline the importance of examining more than one index to ascertain comparability, as high correlations can present in the context of systematic differences, which are a product of differences between measurement environments. Further work is now needed to examine web-based assessments in clinical populations and in larger samples to improve sensitivity for detecting subtler differences between test settings.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Comparing Web-Based and Lab-Based Cognitive Assessment Using the Cambridge Neuropsychological Test Automated Battery: A Within-Subjects Counterbalanced Study

Backx¹,

Skirrow²,

Dente³

et al. 2020

J Med Internet Res

View full text Add to dashboard Cite

show abstract

“…Following the rise of computerized cognitive testing over the past decades [ 1 , 2 ], online cognitive testing is now also increasingly applied in both research and clinical practice [ 3 ]. Online testing shares its main advantages with computerized testing: standardization and precise (multiple) response measurements [ 4 - 6 ] but adds to that the advantages associated with self-administered testing: flexibility (in time and location) and cost-efficiency [ 1 , 7 - 11 ].…”

Section: Introductionmentioning

confidence: 99%

Online Self-Administered Cognitive Testing Using the Amsterdam Cognition Scan: Establishing Psychometric Properties and Normative Data

Feenstra¹,

Vermeulen²,

Murre³

et al. 2018

J Med Internet Res

View full text Add to dashboard Cite

BackgroundOnline tests enable efficient self-administered assessments and consequently facilitate large-scale data collection for many fields of research. The Amsterdam Cognition Scan is a new online neuropsychological test battery that measures a broad variety of cognitive functions.ObjectiveThe aims of this study were to evaluate the psychometric properties of the Amsterdam Cognition Scan and to establish regression-based normative data.MethodsThe Amsterdam Cognition Scan was self-administrated twice from home—with an interval of 6 weeks—by 248 healthy Dutch-speaking adults aged 18 to 81 years.ResultsTest-retest reliability was moderate to high and comparable with that of equivalent traditional tests (intraclass correlation coefficients: .45 to .80; .83 for the Amsterdam Cognition Scan total score). Multiple regression analyses indicated that (1) participants’ age negatively influenced all (12) cognitive measures, (2) gender was associated with performance on six measures, and (3) education level was positively associated with performance on four measures. In addition, we observed influences of tested computer skills and of self-reported amount of computer use on cognitive performance. Demographic characteristics that proved to influence Amsterdam Cognition Scan test performance were included in regression-based predictive formulas to establish demographically adjusted normative data.ConclusionsInitial results from a healthy adult sample indicate that the Amsterdam Cognition Scan has high usability and can give reliable measures of various generic cognitive ability areas. For future use, the influence of computer skills and experience should be further studied, and for repeated measurements, computer configuration should be consistent. The reported normative data allow for initial interpretation of Amsterdam Cognition Scan performances.

show abstract

“…Following the rise of computerized cognitive testing over the past decades [1,2], online cognitive testing is now also increasingly applied in both research and clinical practice [3]. Online testing shares its main advantages with computerized testing: standardization and precise (multiple) response measurements [4][5][6]; but adds to that the advantages associated with self-administered testing: flexibility (in time and location) and cost-efficiency [1,[7][8][9][10][11].…”

Section: Introductionmentioning

confidence: 99%

Online Self-Administered Cognitive Testing Using the Amsterdam Cognition Scan: Establishing Psychometric Properties and Normative Data (Preprint)

Feenstra¹,

Vermeulen²,

Murre³

et al. 2017

Preprint

View full text Add to dashboard Cite

BackgroundOnline tests enable efficient self-administered assessments and consequently facilitate large-scale data collection for many fields of research. The Amsterdam Cognition Scan (ACS) is a new online neuropsychological test battery which measures a broad variety of cognitive functions. ObjectiveThe aims of the present study were to evaluate the psychometric properties of the ACS and to establish regression-based normative data. MethodsThe ACS was self-administrated twice from home -with an interval of 6 weeks -by 248 healthy Dutch speaking adults aged 18 to 81 years. ResultsTest-retest reliability was moderate to high and comparable to that of equivalent traditional tests (intraclass correlation coefficients: .45 to .80, and .83 for the ACS total score). Multiple regression analyses indicated that 1) participants' age negatively influenced all (12) cognitive measures; 2) gender was associated with performance on 6 measures; and 3) education level was positively associated with performance on 4 measures. In addition, we observed an age-and gender-specific influence of computer skills on cognitive performance. Demographic characteristics that proved to influence ACS test performance were included in regression-based predictive formulas to establish demographically adjusted normative data. ConclusionsThe ACS is a promising new online tool to efficiently assess cognitive functioning for research purposes.Results from a healthy adult sample indicate that the ACS has high usability and can give reliable measures of various generic cognitive ability areas. For repeated measurements, computer configuration should be consistent. Combined with these psychometric properties, our normative data allow for initial interpretation of ACS performances.

show abstract

Initial assessment of reliability of a self-administered web-based neuropsychological test battery

Cited by 30 publications

References 31 publications

Comparing Web-Based and Lab-Based Cognitive Assessment Using the Cambridge Neuropsychological Test Automated Battery: A Within-Subjects Counterbalanced Study

Comparing Web-Based and Lab-Based Cognitive Assessment Using the Cambridge Neuropsychological Test Automated Battery: A Within-Subjects Counterbalanced Study

Online Self-Administered Cognitive Testing Using the Amsterdam Cognition Scan: Establishing Psychometric Properties and Normative Data

Online Self-Administered Cognitive Testing Using the Amsterdam Cognition Scan: Establishing Psychometric Properties and Normative Data (Preprint)

Contact Info

Product

Resources

About