On the Utility of Synthetic Data

Hittmeir, Markus; Ekelhart, Andreas; Mayer, Rudolf

doi:10.1145/3339252.3339281

Cited by 65 publications

(40 citation statements)

References 8 publications

(7 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While real behavior data are valuable and sensitive, in many domains, this type of data is unavailable, which makes the usage of synthetic data solutions attractive (Nikolenko, 2019). In most cases, the models trained on the synthetic data are almost as effective as models trained on the real data (Hittmeir et al, 2019). Synthetic data approaches are popular in healthcare, because of the obvious concerns for data sensitivity and privacy, where the models trained on synthetic data show only small decreases in accuracy when compared to the models trained on the real data (Rankin et al, 2020).…”

Section: The Potential Of Synthetic Data Application In Cdtsmentioning

confidence: 99%

Exploring city digital twins as policy tools: A task-based approach to generating synthetic data on urban mobility

Papyshev

Yarime

2021

Data & Policy

View full text Add to dashboard Cite

This article discusses the technology of city digital twins (CDTs) and its potential applications in the policymaking context. The article analyzes the history of the development of the concept of digital twins and how it is now being adopted on a city-scale. One of the most advanced projects in the field—Virtual Singapore—is discussed in detail to determine the scope of its potential domains of application and highlight challenges associated with it. Concerns related to data privacy, availability, and its applicability for predictive simulations are analyzed, and potential usage of synthetic data is proposed as a way to address these challenges. The authors argue that despite the abundance of urban data, the historical data are not always applicable for predictions about the events for which there does not exist any data, as well as discuss the potential privacy challenges of the usage of micro-level individual mobility data in CDTs. A task-based approach to urban mobility data generation is proposed in the last section of the article. This approach suggests that city authorities can establish services responsible for asking people to conduct certain activities in an urban environment in order to create data for possible policy interventions for which there does not exist useful historical data. This approach can help in addressing the challenges associated with the availability of data without raising privacy concerns, as the data generated through this approach will not represent any real individual in society.

show abstract

Section: The Potential Of Synthetic Data Application In Cdtsmentioning

confidence: 99%

Exploring city digital twins as policy tools: A task-based approach to generating synthetic data on urban mobility

Papyshev

Yarime

2021

Data & Policy

View full text Add to dashboard Cite

show abstract

“…Therefore, it is important to evaluate N3C synthetic data in a manner that can inform users with a wide range of intended use cases and definitions for synthetic data fitness for use. [25] The utility of synthetic health data has been evaluated in other work [15,19,20,[26][27][28][29][30] outside of N3C which applied a variety of the ways one can validate synthetic data. [31] However, N3C synthetic data utility has only been evaluated once before.…”

Section: Background and Significancementioning

confidence: 99%

Demonstrating an approach for evaluating synthetic geospatial and temporal epidemiologic data utility: Results from analyzing >1.8 million SARS-CoV-2 tests in the United States National COVID Cohort Collaborative (N3C)

Thomas

Foraker

Zamstein³

et al. 2021

Preprint

View full text Add to dashboard Cite

ObjectiveTo evaluate whether synthetic data derived from a national COVID-19 data set could be used for geospatial and temporal epidemic analyses.Materials and MethodsUsing an original data set (n=1,854,968 SARS-CoV-2 tests) and its synthetic derivative, we compared key indicators of COVID-19 community spread through analysis of aggregate and zip-code level epidemic curves, patient characteristics and outcomes, distribution of tests by zip code, and indicator counts stratified by month and zip code. Similarity between the data was statistically and qualitatively evaluated.ResultsIn general, synthetic data closely matched original data for epidemic curves, patient characteristics, and outcomes. Synthetic data suppressed labels of zip codes with few total tests (mean=2.9±2.4; max=16 tests; 66% reduction of unique zip codes). Epidemic curves and monthly indicator counts were similar between synthetic and original data in a random sample of the most tested (top 1%; n=171) and for all unsuppressed zip codes (n=5,819), respectively. In small sample sizes, synthetic data utility was notably decreased.DiscussionAnalyses on the population-level and of densely-tested zip codes (which contained most of the data) were similar between original and synthetically-derived data sets. Analyses of sparsely-tested populations were less similar and had more data suppression.ConclusionIn general, synthetic data were successfully used to analyze geospatial and temporal trends. Analyses using small sample sizes or populations were limited, in part due to purposeful data label suppression -an attribute disclosure countermeasure. Users should consider data fitness for use in these cases.

show abstract

“…In evaluating multivariate relationships, Rankin et al, 22 Yale et al, 8 Wang et al, 29 Bourou et al, 25 Hittmeir et al, 39 Dankar et al, 41 and Rashidian et al 32 visually compared the Pairwise Pearson Correlation (PPC) matrices to assess whether correlations between attributes of RD are maintained in STD. Additionally, principal component analysis transformation has been used by Yale et al 8 to compare the dimensional properties of STD and RD.…”

Section: Introductionmentioning

confidence: 99%

Synthetic Tabular Data Evaluation in the Health Domain Covering Resemblance, Utility, and Privacy Dimensions

et al. 2023

View full text Add to dashboard Cite

Background Synthetic tabular data generation is a potentially valuable technology with great promise for data augmentation and privacy preservation. However, prior to adoption, an empirical assessment of generated synthetic tabular data is required across dimensions relevant to the target application to determine its efficacy. A lack of standardized and objective evaluation and benchmarking strategy for synthetic tabular data in the health domain has been found in the literature. Objective The aim of this paper is to identify key dimensions, per dimension metrics, and methods for evaluating synthetic tabular data generated with different techniques and configurations for health domain application development and to provide a strategy to orchestrate them. Methods Based on the literature, the resemblance, utility, and privacy dimensions have been prioritized, and a collection of metrics and methods for their evaluation are orchestrated into a complete evaluation pipeline. This way, a guided and comparative assessment of generated synthetic tabular data can be done, categorizing its quality into three categories (“Excellent,” “Good,” and “Poor”). Six health care-related datasets and four synthetic tabular data generation approaches have been chosen to conduct an analysis and evaluation to verify the utility of the proposed evaluation pipeline. Results The synthetic tabular data generated with the four selected approaches has maintained resemblance, utility, and privacy for most datasets and synthetic tabular data generation approach combination. In several datasets, some approaches have outperformed others, while in other datasets, more than one approach has yielded the same performance. Conclusion The results have shown that the proposed pipeline can effectively be used to evaluate and benchmark the synthetic tabular data generated by various synthetic tabular data generation approaches. Therefore, this pipeline can support the scientific community in selecting the most suitable synthetic tabular data generation approaches for their data and application of interest.

show abstract

On the Utility of Synthetic Data

Cited by 65 publications

References 8 publications

Exploring city digital twins as policy tools: A task-based approach to generating synthetic data on urban mobility

Exploring city digital twins as policy tools: A task-based approach to generating synthetic data on urban mobility

Demonstrating an approach for evaluating synthetic geospatial and temporal epidemiologic data utility: Results from analyzing >1.8 million SARS-CoV-2 tests in the United States National COVID Cohort Collaborative (N3C)

Synthetic Tabular Data Evaluation in the Health Domain Covering Resemblance, Utility, and Privacy Dimensions

Contact Info

Product

Resources

About