On Utility and Privacy in Synthetic Genomic Data

Oprisanu, Bristena; Ganev, Georgy; Cristofaro, Emiliano De

doi:10.48550/arxiv.2102.03314

Cited by 6 publications

(7 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…But it is not accurate enough to allow these tasks in many of the real-world experimental scenarios that we tested, even though the datasets we are using are relatively small and with a limited number of features. This confirms some of the more recent cautionary tales found in the synthetic data literature Oprisanu et al (2021). • PrivBayes vs. PATE-CTGAN: Comparing the two synthetic methods, PrivBayes is better than PATE-CTGAN for medium and high epsilon bounds, especially in the Adult dataset, but worse than PATE-CTGAN for low epsilon bounds.…”

Section: Real-world Datasupporting

confidence: 79%

“…1. Measures that capture the empirical probability of successful identification attacks assuming a motivated intruder with access to certain information (Reiter and Mitra, 2009;Stadler et al, 2020;Oprisanu et al, 2021;Hayes et al, 2018;Hilprecht et al, 2019); 2. Measures that quantify properties of the released data which are a proxy for privacy-for example, k-anonymity (Sweeney, 2002;Wagner and Eckhoff, 2018;Yale et al, 2019); 3.…”

Section: Privacymentioning

confidence: 99%

See 1 more Smart Citation

Faking feature importance: A cautionary tale on the use of differentially-private synthetic data

Giles¹,

Hosseini²,

Mingas³

et al. 2022

Preprint

View full text Add to dashboard Cite

Synthetic datasets are often presented as a silver-bullet solution to the problem of privacy-preserving data publishing. However, for many applications, synthetic data has been shown to have limited utility when used to train predictive models. One promising potential application of these data is in the exploratory phase of the machine learning workflow, which involves understanding, engineering and selecting features. This phase often involves considerable time, and depends on the availability of data. There would be substantial value in synthetic data that permitted these steps to be carried out while, for example, data access was being negotiated, or with fewer information governance restrictions. This paper presents an empirical analysis of the agreement between the feature importance obtained from raw and from synthetic data, on a range of artificially generated and realworld datasets (where feature importance represents how useful each feature is when predicting a the outcome). We employ two differentially-private methods to produce synthetic data, and apply various utility measures to quantify the agreement in feature importance as this varies with the level of privacy. Our results indicate that synthetic data can sometimes preserve several representations of the ranking of feature importance in simple settings but their performance is not consistent and depends upon a number of factors. Particular caution should be exercised in more nuanced real-world settings, where synthetic data can lead to differences in ranked feature importance that could alter key modelling decisions. This work has important implications for developing synthetic versions of highly sensitive data sets in fields such as finance and healthcare. * Equal contribution. Listing order is alphabetical.

show abstract

Section: Real-world Datasupporting

confidence: 79%

Section: Privacymentioning

confidence: 99%

Faking feature importance: A cautionary tale on the use of differentially-private synthetic data

Giles¹,

Hosseini²,

Mingas³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The synthetic patients are generated by a simulator [19] grounded in clinical-genetic knowledge. Furthermore, training on simulated data mitigates concerns regarding privacy breaches, in which specific individuals can be identified from the training data [55,56]. Hence, it is possible to publicly release the fully trained SHEPHERD without privacy concerns.…”

Section: Discussionmentioning

confidence: 99%

Few shot learning for phenotype-driven diagnosis of patients with rare genetic diseases

Alsentzer

Kobren

et al. 2022

Preprint

View full text Add to dashboard Cite

There are more than 7,000 rare diseases, some of which affect 3,500 or fewer patients in the US. Due to clinicians' limited experience with such diseases and the considerable heterogeneity of their clinical presentations, many patients with rare genetic diseases remain undiagnosed. While artificial intelligence has demonstrated success in assisting diagnosis, its success is usually contingent on the availability of large labeled datasets. Here, we present SHEPHERD, a deep learning approach for multi-faceted rare disease diagnosis. SHEPHERD is guided by existing knowledge of diseases, phenotypes, and genes to learn novel connections between a patient's clinico-genetic information and phenotype and gene relationships. We train SHEPHERD exclusively on simulated patients and evaluate on a cohort of 465 patients representing 299 diseases (79% of genes and 83% of diseases are represented in only a single patient) in the Undiagnosed Diseases Network. SHEPHERD excels at several diagnostic facets: performing causal gene discovery (causal genes are predicted at rank = 3.52 on average), retrieving "patients-like-me" with the same gene or disease, and providing interpretable characterizations of novel disease presentations. SHEPHERD demonstrates the potential of artificial intelligence to accelerate the diagnosis of rare disease patients and has implications for the use of deep learning on medical datasets with very few labels.

show abstract

“…[3] introduces the framework of differential testing, similar to the notion of DP, which deems a dataset anonymized when the inference accuracy from the synthetic data is about the same whether the user's record is included in the original dataset or not. In the medical domain, [43] and [18,33] evaluate privacy vs. utility of generative models and synthetic datasets, respectively. The first two works rely on the notion of membership inference to quantify the privacy risks, the last one on equivalence classes and intra-record distances.…”

Section: Related Workmentioning

confidence: 99%

A Unified Framework for Quantifying Privacy Risk in Synthetic Data

Giomi¹,

Boenisch

Wehmeyer³

et al. 2023

PoPETs

View full text Add to dashboard Cite

Synthetic data is often presented as a method for sharing sensitive information in a privacy-preserving manner by reproducing the global statistical properties of the original data without dis closing sensitive information about any individual. In practice, as with other anonymization methods, synthetic data cannot entirely eliminate privacy risks. These residual privacy risks need instead to be ex-post uncovered and assessed. However, quantifying the actual privacy risks of any synthetic dataset is a hard task, given the multitude of facets of data privacy. We present Anonymeter, a statistical framework to jointly quantify different types of privacy risks in synthetic tabular datasets. We equip this framework with attack-based evaluations for the singling out, linkability, and inference risks, which are the three key indicators of factual anonymization according to data protection regulations, such as the European General Data Protection Regulation (GDPR). To the best of our knowledge, we are the first to introduce a coherent and legally aligned evaluation of these three privacy risks for synthetic data, as well as to design privacy attacks which model directly the singling out and linkability risks. We demonstrate the effectiveness of our methods by conducting an extensive set of experiments that measure the privacy risks of data with deliberately inserted privacy leakages, and of synthetic data generated with and without differential privacy. Our results highlight that the three privacy risks reported by our framework scale linearly with the amount of privacy leakage in the data. Furthermore, we observe that synthetic data exhibits the lowest vulnerability against linkability, indicating one-to-one relationships between real and synthetic data records are not preserved. Finally, with a quantitative comparison we demonstrate that Anonymeter outperforms existing synthetic data privacy evaluation frameworks both in terms of detecting privacy leaks, as well as computation speed. To contribute to a privacy-conscious usage of synthetic data, we publish Anonymeter as an open-source library (https://github.com/statice/anonymeter).

show abstract

On Utility and Privacy in Synthetic Genomic Data

Cited by 6 publications

References 42 publications

Faking feature importance: A cautionary tale on the use of differentially-private synthetic data

Faking feature importance: A cautionary tale on the use of differentially-private synthetic data

Few shot learning for phenotype-driven diagnosis of patients with rare genetic diseases

A Unified Framework for Quantifying Privacy Risk in Synthetic Data

Contact Info

Product

Resources

About