Quality assessment of real-world data repositories across the data life cycle: A literature review

Liaw, Teng; Guo, Jason Guan Nan; Ansari, Sameera; Jonnagaddala, Jitendra; Godinho, Myron Anthony; Borelli, Alder Jose; Lusignan, Simon de; Capurro, Daniel; Liyanage, Harshana; Bhattal, Navreet; Bennett, Vicki; Chan, Jaclyn; Kahn, Michael G.

doi:10.1093/jamia/ocaa340

Cited by 56 publications

(42 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This framework is well established and is used in a variety of scientific work. 24 25 26 27 28 29 30 31 To avoid linguistic misunderstandings, we considered it necessary and sensible to use this uniform harmonized DQA terminology in our present work.…”

Section: Methodsmentioning

confidence: 99%

Linking a Consortium-Wide Data Quality Assessment Tool with the MIRACUM Metadata Repository

et al. 2021

View full text Add to dashboard Cite

Background Many research initiatives aim at using data from electronic health records (EHRs) in observational studies. Participating sites of the German Medical Informatics Initiative (MII) established data integration centers to integrate EHR data within research data repositories to support local and federated analyses. To address concerns regarding possible data quality (DQ) issues of hospital routine data compared with data specifically collected for scientific purposes, we have previously presented a data quality assessment (DQA) tool providing a standardized approach to assess DQ of the research data repositories at the MIRACUM consortium's partner sites. Objectives Major limitations of the former approach included manual interpretation of the results and hard coding of analyses, making their expansion to new data elements and databases time-consuming and error prone. We here present an enhanced version of the DQA tool by linking it to common data element definitions stored in a metadata repository (MDR), adopting the harmonized DQA framework from Kahn et al and its application within the MIRACUM consortium. Methods Data quality checks were consequently aligned to a harmonized DQA terminology. Database-specific information were systematically identified and represented in an MDR. Furthermore, a structured representation of logical relations between data elements was developed to model plausibility-statements in the MDR. Results The MIRACUM DQA tool was linked to data element definitions stored in a consortium-wide MDR. Additional databases used within MIRACUM were linked to the DQ checks by extending the respective data elements in the MDR with the required information. The evaluation of DQ checks was automated. An adaptable software implementation is provided with the R package DQAstats. Conclusion The enhancements of the DQA tool facilitate the future integration of new data elements and make the tool scalable to other databases and data models. It has been provided to all ten MIRACUM partners and was successfully deployed and integrated into their respective data integration center infrastructure.

show abstract

Section: Methodsmentioning

confidence: 99%

Linking a Consortium-Wide Data Quality Assessment Tool with the MIRACUM Metadata Repository

et al. 2021

View full text Add to dashboard Cite

show abstract

“…Engaging a diverse range of citizens from various walks of life in data capture and creation can better ensure that data is representative of the society it claims to represent. Being a form of public participation, participatory knowledge co-creation is also contingent on core values and a code of ethics (International Association for Public Participation, 2020), upon which the trustworthiness, (and thus, the quality) of the data depends (Liaw et al, 2020). Without these prerequisites, virtuous cycles can spin out of control and turn vicious, with unintended consequences: the rise of 'fake news' has highlighted how easily untrustworthy information can fuel digital participation that erodes social capital (de Z uñiga et al, 2017).…”

Section: Data Trustworthinessmentioning

confidence: 99%

Knowledge co-creation in participatory policy and practice: Building community through data-driven direct democracy

et al. 2021

Self Cite

View full text Add to dashboard Cite

Engaging citizens with digital technology to co-create data, information and knowledge has widely become an important strategy for informing the policy response to COVID-19 and the ‘infodemic’ of misinformation in cyberspace. This move towards digital citizen participation aligns well with the United Nations’ agenda to encourage the use of digital tools to enable data-driven, direct democracy. From data capture to information generation, and knowledge co-creation, every stage of the data lifecycle bears important considerations to inform policy and practice. Drawing on evidence of participatory policy and practice during COVID-19, we outline a framework for citizen ‘e-participation’ in knowledge co-creation across every stage of the policy cycle. We explore how coupling the generation of information with that of social capital can provide opportunities to collectively build trust in institutions, accelerate recovery and facilitate the ‘e-society’. We outline the key aspects of realising this vision of data-driven direct democracy by discussing several examples. Sustaining participatory knowledge co-creation beyond COVID-19 requires that local organisations and institutions (e.g. academia, health and welfare, government, business) incorporate adaptive learning mechanisms into their operational and governance structures, their integrated service models, as well as employing emerging social innovations.

show abstract

“…3 Centre for Clinical Trials, University of Münster, Münster, Germany. 4 Institute of Community Medicine, University Medicine of Greifswald, Greifswald, Germany.…”

Section: Supplementary Informationmentioning

confidence: 99%

“…Standardizing and reusing such metadata definitions has two major advantages. First, it yields harmonized data sets that allow data exchange between institutions [3,4] and facilitate data analyses, such as multi-site phenotyping [5] or machine learning [6]. Second, medical documentation does not have to be developed from scratch reducing costs [7].…”

Section: Introductionmentioning

confidence: 99%

Pragmatic MDR: a metadata repository with bottom-up standardization of medical metadata through reuse

Hegselmann

Storck

Geßner

et al. 2021

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background The variety of medical documentation often leads to incompatible data elements that impede data integration between institutions. A common approach to standardize and distribute metadata definitions are ISO/IEC 11179 norm-compliant metadata repositories with top-down standardization. To the best of our knowledge, however, it is not yet common practice to reuse the content of publicly accessible metadata repositories for creation of case report forms or routine documentation. We suggest an alternative concept called pragmatic metadata repository, which enables a community-driven bottom-up approach for agreeing on data collection models. A pragmatic metadata repository collects real-world documentation and considers frequent metadata definitions as high quality with potential for reuse. Methods We implemented a pragmatic metadata repository proof of concept application and filled it with medical forms from the Portal of Medical Data Models. We applied this prototype in two use cases to demonstrate its capabilities for reusing metadata: first, integration into a study editor for the suggestion of data elements and, second, metadata synchronization between two institutions. Moreover, we evaluated the emergence of bottom-up standards in the prototype and two medical data managers assessed their quality for 24 medical concepts. Results The resulting prototype contained 466,569 unique metadata definitions. Integration into the study editor led to a reuse of 1836 items and item groups. During the metadata synchronization, semantic codes of 4608 data elements were transferred. Our evaluation revealed that for less complex medical concepts weak bottom-up standards could be established. However, more diverse disease-related concepts showed no convergence of data elements due to an enormous heterogeneity of metadata. The survey showed fair agreement (Kalpha = 0.50, 95% CI 0.43–0.56) for good item quality of bottom-up standards. Conclusions We demonstrated the feasibility of the pragmatic metadata repository concept for medical documentation. Applications of the prototype in two use cases suggest that it facilitates the reuse of data elements. Our evaluation showed that bottom-up standardization based on a large collection of real-world metadata can yield useful results. The proposed concept shall not replace existing top-down approaches, rather it complements them by showing what is commonly used in the community to guide other researchers.

show abstract

Quality assessment of real-world data repositories across the data life cycle: A literature review

Cited by 56 publications

References 15 publications

Linking a Consortium-Wide Data Quality Assessment Tool with the MIRACUM Metadata Repository

Linking a Consortium-Wide Data Quality Assessment Tool with the MIRACUM Metadata Repository

Knowledge co-creation in participatory policy and practice: Building community through data-driven direct democracy

Pragmatic MDR: a metadata repository with bottom-up standardization of medical metadata through reuse

Contact Info

Product

Resources

About