NCI Pilot 3 - Synthetic Data Generation Report

Gonçalves, Alim P. C.; Sales, Ana Paula; Ray, Priyadip; Soper, Braden

doi:10.2172/1430997

Cited by 3 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The bean dataset has 17 measurements (floating point) related to 7 bean types. We selected 10 measures at random to make the dataset dimensionality compatible with the other datasets giving this set of variables: area (1); minor axis (5); eccentricity (6); convex area (7); equivalent diameter (8); extent (9); solidarity (10); roundness (11); shape factor 3 (15), and shape factor 4 (16). The parenthetical references give the variable number as listed in the respective resource (see [23]).…”

Section: Data and Measurementsmentioning

confidence: 99%

“…Synthetic data applications in health-related research use a variety of techniques. Some methods are used for generating samples from large populations [16][17][18][19]. These approaches include hidden Markov models [18], techniques that reconstruct time series data coupled with sampling the empirical probability density function of the relevant variables [16], and methods that estimate probability density functions (pdfs) from the data, not accounting for variable correlation [19].…”

Section: Introductionmentioning

confidence: 99%

“…Some methods are used for generating samples from large populations [16][17][18][19]. These approaches include hidden Markov models [18], techniques that reconstruct time series data coupled with sampling the empirical probability density function of the relevant variables [16], and methods that estimate probability density functions (pdfs) from the data, not accounting for variable correlation [19]. Other work used moment matching to generate synthetic data but does not consider relative frequencies in the comparison analysis [20].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Techniques to Produce and Evaluate Realistic Multivariate Synthetic Data

Heine

Fowler

Berglund

et al. 2021

Preprint

View full text Add to dashboard Cite

BackgroundProper data modeling in biomedical research requires sufficient data for exploration and reproducibility purposes. A limited sample size can inhibit objective performance evaluation.ObjectiveWe are developing a synthetic population (SP) generation technique to address the limited sample size condition. We show how to estimate a multivariate empirical probability density function (pdf) by converting the task to multiple one-dimensional (1D) pdf estimations.MethodsKernel density estimation (KDE) in 1D was used to construct univariate maps that converted the input variables (X) to normally distributed variables (Y). Principal component analysis (PCA) was used to transform the variables in Y to the uncoupled representation (T), where the univariate pdfs were assumed normal with specified variances. A standard random number generator was used to create synthetic variables with specified variances in T. Applying the inverse PCA transform to the synthetic variables in T produced the SP in Y. Applying the inverse maps produced the respective SP in X. Multiple tests were developed to compare univariate and multivariate pdfs and covariance matrices between the input (sample) and synthetic samples. Three datasets were investigated (n = 667) each with 10 input variables.ResultsFor all three datasets, both the univariate (in X, Y, and T) and multivariate (in X, Y, and T) tests showed that the univariate and multivariate pdfs from synthetic samples were statistically similar to their pdfs from the respective samples. Application of several tests for multivariate normality indicated that the SPs in Y were approximately normal. Covariance matrix comparisons (in X and Y) also indicated the same similarity.ConclusionsThe work demonstrates how to generate multivariate synthetic data that matches the real input data by converting the input into multiple 1D problems. The work also shows that it is possible to convert a multivariate input pdf to a form that approximates a multivariate normal, although the technique is not dependent upon this finding. Further studies are required to evaluate the generalizability of the approach.

show abstract

Section: Data and Measurementsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Techniques to Produce and Evaluate Realistic Multivariate Synthetic Data

Heine

Fowler

Berglund

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Empirically-derived synthetic populations to mitigate small sample sizes

Fowler¹,

Berglund²,

Schell³

et al. 2020

Journal of Biomedical Informatics

View full text Add to dashboard Cite

Techniques to produce and evaluate realistic multivariate synthetic data

Heine

Fowler

Berglund

et al. 2023

Sci Rep

View full text Add to dashboard Cite

Data modeling requires a sufficient sample size for reproducibility. A small sample size can inhibit model evaluation. A synthetic data generation technique addressing this small sample size problem is evaluated: from the space of arbitrarily distributed samples, a subgroup (class) has a latent multivariate normal characteristic; synthetic data can be generated from this class with univariate kernel density estimation (KDE); and synthetic samples are statistically like their respective samples. Three samples (n = 667) were investigated with 10 input variables (X). KDE was used to augment the sample size in X. Maps produced univariate normal variables in Y. Principal component analysis in Y produced uncorrelated variables in T, where the probability density functions were approximated as normal and characterized; synthetic data was generated with normally distributed univariate random variables in T. Reversing each step produced synthetic data in Y and X. All samples were approximately multivariate normal in Y, permitting the generation of synthetic data. Probability density function and covariance comparisons showed similarity between samples and synthetic samples. A class of samples has a latent normal characteristic. For such samples, this approach offers a solution to the small sample size problem. Further studies are required to understand this latent class.

show abstract

NCI Pilot 3 - Synthetic Data Generation Report

Cited by 3 publications

References 0 publications

Techniques to Produce and Evaluate Realistic Multivariate Synthetic Data

Techniques to Produce and Evaluate Realistic Multivariate Synthetic Data

Empirically-derived synthetic populations to mitigate small sample sizes

Techniques to produce and evaluate realistic multivariate synthetic data

Contact Info

Product

Resources

About