The Real-World Data Challenges Radar: A Review on the Challenges and Risks regarding the Use of Real-World Data

Grimberg, Frank; Asprion, Petra Maria; Schneider, Bettina; Miho, Enkelejda; Babrak, Lmar; Habbabeh, Ali

doi:10.1159/000516178

Cited by 16 publications

(22 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many challenges [ 33 ] and information parameters to guide RWD appropriateness for research questions have been proposed [ 34 ]. However, a framework that translates information (eg, quantifying parameters such as the importance of longitudinal data, accessibility, and publication as a metric that indicates quality [ 34 ]) into an evaluation framework is still missing.…”

Section: Resultsmentioning

confidence: 99%

“…Regulatory authorities such as the FDA or the EMA face challenges when it comes to consideration of RWE generated from RWD in regulatory decision-making and drug approval [ 2 , 11 ]. Some of the challenges [ 33 ] the agencies must overcome are related to ensuring the quality of data and providing frameworks for consideration. Without guiding regulations, the currently expanding use of RWD in studies [ 26 , 39 ] fails to follow industry standards.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

RWD-Cockpit: Application for Quality Assessment of Real-world Data

Babrak¹,

Smakaj²,

Agac³

et al. 2022

JMIR Form Res

Self Cite

View full text Add to dashboard Cite

Background Digital technologies are transforming the health care system. A large part of information is generated as real-world data (RWD). Data from electronic health records and digital biomarkers have the potential to reveal associations between the benefits and adverse events of medicines, establish new patient-stratification principles, expose unknown disease correlations, and inform on preventive measures. The impact for health care payers and providers, the biopharmaceutical industry, and governments is massive in terms of health outcomes, quality of care, and cost. However, a framework to assess the preliminary quality of RWD is missing, thus hindering the conduct of population-based observational studies to support regulatory decision-making and real-world evidence. Objective To address the need to qualify RWD, we aimed to build a web application as a tool to translate characterization of some quality parameters of RWD into a metric and propose a standard framework for evaluating the quality of the RWD. Methods The RWD-Cockpit systematically scores data sets based on proposed quality metrics and customizable variables chosen by the user. Sleep RWD generated de novo and publicly available data sets were used to validate the usability and applicability of the web application. The RWD quality score is based on the evaluation of 7 variables: manageability specifies access and publication status; complexity defines univariate, multivariate, and longitudinal data; sample size indicates the size of the sample or samples; privacy and liability stipulates privacy rules; accessibility specifies how the data set can be accessed and to what granularity; periodicity specifies how often the data set is updated; and standardization specifies whether the data set adheres to any specific technical or metadata standard. These variables are associated with several descriptors that define specific characteristics of the data set. Results To address the need to qualify RWD, we built the RWD-Cockpit web application, which proposes a framework and applies a common standard for a preliminary evaluation of RWD quality across data sets—molecular, phenotypical, and social—and proposes a standard that can be further personalized by the community retaining an internal standard. Applied to 2 different case studies—de novo–generated sleep data and publicly available data sets—the RWD-Cockpit could identify and provide researchers with variables that might increase quality. Conclusions The results from the application of the framework of RWD metrics implemented in the RWD-Cockpit application suggests that multiple data sets can be preliminarily evaluated in terms of quality using the proposed metrics. The output scores—quality identifiers—provide a first quality assessment for the use of RWD. Although extensive challenges remain to be addressed to set RWD quality standards, our proposal can serve as an initial blueprint for community efforts in the characterization of RWD quality for regulated settings.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

RWD-Cockpit: Application for Quality Assessment of Real-world Data

Babrak¹,

Smakaj²,

Agac³

et al. 2022

JMIR Form Res

Self Cite

View full text Add to dashboard Cite

show abstract

“…We observed widespread receptiveness among our respondents to the idea of sharing their patients' anonymized or pseudonymized electronic health data with others inside or outside their country for research purposes, no matter whether that would mean that these data would then be transferred to an external repository or not. This finding demonstrates that, although acquiring access to data from hospitals and clinics is usually cited as a major hurdle to overcome for stakeholders seeking to conduct RWD studies (Cave et al, 2019;Rudrapatna and Butte, 2020;Grimberg et al, 2021), the clinicians employed by those institutions are willing to deliver those data directly or to at least allow them to be analyzed, under certain conditions. Here again, it should be highlighted that investigators belonging to the EORTC network are in all likelihood more inclined to contribute to research, so this observation may not extend to oncologists in general.…”

Section: Discussionmentioning

confidence: 99%

“…Here, observational RWD can offer valuable insights into the performance of these products in larger groups of patients ( Agarwala et al, 2018 ; Miksad et al, 2019 ), as long as the quality of the data is sufficiently high. Since RWD of this type originate from routine clinical practice and not from research activities, they can be incomplete, inconsistent and subject to misclassification bias ( Blacketer et al, 2021 ; Grimberg et al, 2021 ). Consequently, it is essential to evaluate the completeness, conformance and plausibility of the data by performing quality checks through the application of appropriate tools ( Kahn et al, 2016 ; Blacketer et al, 2021 ).…”

Section: Discussionmentioning

confidence: 99%

How do cancer clinicians perceive real-world data and the evidence derived therefrom? Findings from an international survey of the European Organisation for Research and Treatment of Cancer

et al. 2022

View full text Add to dashboard Cite

Background: The role of real-world evidence (RWE) in the development of anticancer therapies has been gradually growing over time. Regulators, payers and health technology assessment agencies, spurred by the rise of the precision medicine model, are increasingly incorporating RWE into their decision-making regarding the authorization and reimbursement of novel antineoplastic treatments. However, it remains unclear how this trend is viewed by clinicians in the field. This study aimed to investigate the opinions of these stakeholders with respect to RWE and its suitability for informing regulatory, reimbursement-related and clinical decisions in oncology.Methods: An online survey was disseminated to clinicians belonging to the network of the European Organisation for Research and Treatment of Cancer between May and July 2021.Results: In total, 557 clinicians across 30 different countries participated in the survey, representing 13 distinct cancer domains. Despite seeing the methodological challenges associated with its interpretation as difficult to overcome, the respondents mostly (75.0%) perceived RWE positively, and believed such evidence could be relatively strong, depending on the designs and data sources of the studies from which it is produced. Few (4.6%) saw a future expansion of its influence on decision-makers as a negative evolution. Furthermore, nearly all (94.0%) participants were open to the idea of sharing anonymized or pseudonymized electronic health data of their patients with external parties for research purposes. Nevertheless, most clinicians (77.0%) still considered randomized controlled trials (RCTs) to be the gold standard for generating clinical evidence in oncology, and a plurality (49.2%) thought that RWE cannot fully address the knowledge gaps that remain after a new antitumor intervention has entered the market. Moreover, a majority of respondents (50.7%) expressed that they relied more heavily on RCT-derived evidence than on RWE for their own decision-making.Conclusion: While cancer clinicians have positive opinions about RWE and want to contribute to its generation, they also continue to hold RCTs in high regard as sources of actionable evidence.

show abstract

“…Real-world data (RWD), and its analogue in the search for medical evidence (real-world evidence) [ 1 , 2 ], represent novel approaches. They make it possible for large numbers of professionals who agree to participate to contribute anonymised epidemiological data collected from their patients to a systematised database that organises, hierarchises and facilitates analysis of that data.…”

Section: Introductionmentioning

confidence: 99%

Developing a real-world database for oncology: a descriptive analysis of breast cancer in Argentina

Streich¹,

Villalba²,

Cid³

et al. 2022

ecancer

View full text Add to dashboard Cite

Introduction:Registries based on Real-World Data (RWD) are those obtained outside of systematised and randomised clinical trials. They allow the collection of information from a large number of patients and enable the participation of a significant number of professionals. PrecisaXperta is a web platform developed for this purpose with more than 2 years of operation, parameterised for oncology. Its design allows the construction of an epidemiological database in real time and exportable for processing.Objective: To describe the characteristics and operation of this online data recording tool, explain how it was developed and analyse the quality of the information recorded, taking as an example the data obtained for breast cancer.Materials and methods: Physicians, computer scientists and data science analysts participated in the development. Patient data, history, educational level, diagnosis, staging, molecular markers, quality of life, types of treatments, progression and response, imaging, complications, adverse events are some of the fields included. Data treatment in terms of encryption, anonymisation, protection and validation is also explained. The selected breast cancer data for description were processed with medium-level statistical programmes, since the number required to apply Big Data engines is not yet available.Results: From a total of 6,892 solid tumours, 1,892 were breast cancer and 1,654 were selected that complied with a data set minimum elaborated ad hoc. Cases from 13 provinces showed a geolocation bias according to the place of practice of the professionals in the collaborative network. The predominant lack of data was detected in molecular markers (ki67) and correlativity in some lines of treatment. Inconsistencies in dates and therapeutic schemes were also detected. Data curation made it possible to exclude them. The age of the patients was 55.3 ± 11.88 years. At the time of diagnosis, the predominance was in stage I: 36.48% and II 30.06%, with positive hormone receptors in 1,424 (89.96%) cases. The predominant treatments were hormonal (61.54%) and target directed with 30.85% for HER2(+) and 39.14% for HER2(−) accompanied in most cases (85.9%) by some period of chemotherapy. Immunotherapy was much less represented (0.36%). Data were processed, homogenised, pooled and presented and made accessible in a form suitable for application to RWD analyses.Conclusions: PrecisaXperta fulfils this purpose of systematising the information to facilitate its loading with its simple and intuitive interface. From the analysis of the data obtained in breast cancer, it is clear that some fields should be mandatory in order to improve the quality of the information. The results describing the registered breast cancers give us a surface view of the affected population and prepare us to design future studies when we have local Big Data. This type of development, with continuous improvements and online results, will allow with its dissemination, that the participating professionals have information of what happen...

show abstract

The Real-World Data Challenges Radar: A Review on the Challenges and Risks regarding the Use of Real-World Data

Cited by 16 publications

References 12 publications

RWD-Cockpit: Application for Quality Assessment of Real-world Data

RWD-Cockpit: Application for Quality Assessment of Real-world Data

How do cancer clinicians perceive real-world data and the evidence derived therefrom? Findings from an international survey of the European Organisation for Research and Treatment of Cancer

Developing a real-world database for oncology: a descriptive analysis of breast cancer in Argentina

Contact Info

Product

Resources

About