Empowering Researchers to Query Medical Data and Biospecimens by Ensuring Appropriate Usability of a Feasibility Tool: Evaluation Study

Abstract

Background The Aligning Biobanking and Data Integration Centers Efficiently project aims to harmonize technologies and governance structures of German university hospitals and their biobanks to facilitate searching for patient data and biospecimens. The central element will be a feasibility tool for researchers to query the availability of samples and data to determine the feasibility of their study project. Objective The objectives of the study were as follows: an evaluation of the overall user interface usability of the feasibility tool, the identification of critical usability issues, comprehensibility of the underlying ontology operability, and analysis of user feedback on additional functionalities. From these, recommendations for quality-of-use optimization, focusing on more intuitive usability, were derived. Methods To achieve the study goal, an exploratory usability test consisting of 2 main parts was conducted. In the first part, the thinking aloud method (test participants express their thoughts aloud throughout their use of the tool) was complemented by a quantitative questionnaire. In the second part, the interview method was combined with supplementary mock-ups to collect users’ opinions on possible additional features. Results The study cohort rated global usability of the feasibility tool based on the System Usability Scale with a good score of 81.25. The tasks assigned posed certain challenges. No participant was able to solve all tasks correctly. A detailed analysis showed that this was mostly because of minor issues. This impression was confirmed by the recorded statements, which described the tool as intuitive and user friendly. The feedback also provided useful insights regarding which critical usability problems occur and need to be addressed promptly. Conclusions The findings indicate that the prototype of the Aligning Biobanking and Data Integration Centers Efficiently feasibility tool is headed in the right direction. Nevertheless, we see potential for optimization primarily in the display of the search functions, the unambiguous distinguishability of criteria, and the visibility of their associated classification system. Overall, it can be stated that the combination of different tools used to evaluate the feasibility tool provided a comprehensive picture of its usability.

Towards a Bavarian Oncology Real World Data Research Platform

“…Finally, the stepwise defined high-level BZKF IT strategy framed all work focusing on clinical oncology data (within this subproject) to serve as a joined link between all IT and data gathering subprojects within the BZKF. As already pursued within earlier projects for establishing harmonized multicenter IT networks [13][14][15], we adapted the strategy of data-driven development accompanied by stakeholder workshops while aiming at a detailed requirements analysis built on experiences from oncology researchers within the BZKF network. The stakeholder workshop focused on discussing the most important types of queries and how we can accurately represent them, bringing together a diverse group of oncology researchers.…”

Adaptability of Existing Feasibility Tools for Clinical Study Research Data Platforms

“…No openly available source code could also be found for the "Electronic Health Records Systems for Clinical Research (EHR4CR)" project [15]. The tools "cBioPortal" [14], "Biology-Related Information Storage Kit (BRISK)" [14], "COVID-19 Exchange Data Platform (CODEX)" [16], "integrated clinical omics database (iCOD)" [14], "integrating data for analysis, anonymization and sharing (iDASH)" [14], "Informatics for Integrating Biology and the Bedside (i2b2) Query & Analysis Tool" [16,17], "Medical-Blocks" [18], "tranSMART" [6,16] and "Medex" [19] are central research data platforms or platforms built around a single database. Although in principle possible to adopt, we consider the effort required to customize them to be very high and have therefore not included them in the further evaluation.…”

“…Finally the "COVID-Curated and the Open aNalysis aNd rEsearCh platform (CO-CONNECT)" [23] was excluded as the project terminated in October 2022 and there is no indication for further provision or development. The remaining tools are "OHDSI's ATLAS" [16], the "Clinical Communication Platform (CCP)" [24], the "Research health data portal (FDPG)" [16], the "DZHK Feasibility Explorer (FE)" [6], "MedCo" [25], the "Sample Locator" [16,26] and the "Shared Health Research Informatics (SHRINE)" Network [27].…”

“…Only SHRINE has been excluded here, based on the in-depth description from the developers of MedCo: SHRINE provides an ad-hoc mechanism for obfuscating query results and for locking-out investigators after a certain number of queries, whereas MedCo features a mechanism that achieves differential privacy [25]. [10,16] YES [28] NO [10] NO [16] CCP [24] YES [24] N/A N/A FDPG [16] YES [16] N/A YES [16] FE [6] YES [6]) N/A N/A MedCo [25] YES [25] N/A YES [25] Sample Locator [16,26] YES YES [10] YES [10,26] SHRINE [27] NO [25] N/A N/A…”

“…According to a comparative usability study, it turned out that ATLAS has a poor usability score for feasibility tests, as it has been designed as an expert tool [16]. M4 is rarely assessed in the literature and not assessable without knowledge about the ground truth, i.e.…”