FAIRness through automation: development of an automated medical data integration infrastructure for FAIR health data in a maximum care university hospital

Abstract

Background Secondary use of routine medical data is key to large-scale clinical and health services research. In a maximum care hospital, the volume of data generated exceeds the limits of big data on a daily basis. This so-called “real world data” are essential to complement knowledge and results from clinical trials. Furthermore, big data may help in establishing precision medicine. However, manual data extraction and annotation workflows to transfer routine data into research data would be complex and inefficient. Generally, best practices for managing research data focus on data output rather than the entire data journey from primary sources to analysis. To eventually make routinely collected data usable and available for research, many hurdles have to be overcome. In this work, we present the implementation of an automated framework for timely processing of clinical care data including free texts and genetic data (non-structured data) and centralized storage as Findable, Accessible, Interoperable, Reusable (FAIR) research data in a maximum care university hospital. Methods We identify data processing workflows necessary to operate a medical research data service unit in a maximum care hospital. We decompose structurally equal tasks into elementary sub-processes and propose a framework for general data processing. We base our processes on open-source software-components and, where necessary, custom-built generic tools. Results We demonstrate the application of our proposed framework in practice by describing its use in our Medical Data Integration Center (MeDIC). Our microservices-based and fully open-source data processing automation framework incorporates a complete recording of data management and manipulation activities. The prototype implementation also includes a metadata schema for data provenance and a process validation concept. All requirements of a MeDIC are orchestrated within the proposed framework: Data input from many heterogeneous sources, pseudonymization and harmonization, integration in a data warehouse and finally possibilities for extraction or aggregation of data for research purposes according to data protection requirements. Conclusion Though the framework is not a panacea for bringing routine-based research data into compliance with FAIR principles, it provides a much-needed possibility to process data in a fully automated, traceable, and reproducible manner.

Research collaboration data platform ensuring general data protection

“…The MeDIC receives the data from the clinical source systems in a particular network segment within PatLAN. In this step of the operation the data is handled by an ETL (extract, transform, load) process for data minimization and transformation 35 . This means that only the minimized data is stored in the MeDIC.…”

Geoinformática y salud: revisión, caracterización y análisis de plataformas SIG basadas en WEB para vigilancia en salud pública

“…Investigadores y desarrolladores de plataformas para vigilancia en salud pública deben afrontar la difícil tarea de recopilar y actualizar constantemente el vasto flujo de información generado por los entornos sanitarios contemporáneos. Adicionalmente, los procesos tradicionales de recopilación manual de datos en los hospitales son notablemente ineficientes, con alto consumo de tiempo y recursos humanos (Parciak et al, 2023). Esta ineficiencia persiste incluso cuando los datos de salud se almacenan en bases de datos.…”

Automated Metadata Transformation in a Medical Data Integration Center: Implementation of an Algorithm and Standardized Quality Analysis

“…The UMG-MeDIC combines data from a maximum care hospital. Metadata and corresponding data from the clinical primary and secondary systems of the Medical Center are stored in a formalized manner within a data warehouse (DWH), using dedicated processes of extraction, transformation, and loading (ETL) (Parciak et al 2023).…”