Skip to contents

Perform generation of synthetic datasets for an empirical distribution

Source: R/sg-vpop-est.R
sg_vpop_est.Rd

The function operates in two modes:

  • Fixed seed mode (when seed is specified): Generates a single dataset using the provided seed

  • Search mode (when seed = NA): Iteratively searches for nds datasets that meet the correlation difference threshold

Usage

sg_vpop_est(
  data_i,
  nobj = NA,
  id_col = NULL,
  minnumlev = 3,
  npop = 1,
  excl_col = NULL,
  seed = 123,
  seed_umap = 42,
  palette = NULL,
  diag_plots = FALSE,
  remove_duplicates = TRUE,
  noise_level = 0.1,
  nds = 1,
  tg_corrdif = 0.1
)

Arguments

data_i

data frame. Input data frame containing the original dataset to be synthesized (required)

nobj

integer. Specify the exact number of rows to generate in the synthetic dataset. When provided, overrides npop (optional, default: NA)

id_col

Character string. Specify the name of the identifier column to exclude from synthesis. Default: NULL.

minnumlev

integer. Threshold; numeric variables with <= minnumlev unique values are converted to factors (optional, default: 3)

npop

Integer. Number of population replicates. Default is 1.

excl_col

Character vector. Contains column names to exclude from synthesis. Default: NULL

seed

integer. Random seed for synthetic data generation. If provided (not NA, not NULL), generates a single dataset with this seed (fixed seed mode). If NA or NULL uses search mode to find nds datasets meeting correlation threshold (optional, default: 123)

seed_umap

integer. Random seed for UMAP algorithm reproducibility (optional, default: 42)

palette

character vector. Contains color codes (hex format) for custom plot color schemes. If provided, should contain at least 2 colors. Used for histograms, bar plots, and UMAP visualizations (optional, default: c("#3a6eba", "#efdd3c", "#1a1866", "#f2b93b"))

diag_plots

logical flag. If TRUE, generates diagnostic plots and UMAP visualizations (optional, default: TRUE)

remove_duplicates

logical flag. If TRUE, automatically removes exact duplicates between original and synthetic data by adding controlled noise (optional, default: TRUE)

noise_level

numeric. Proportion of standard deviation to use when adding noise to continuous variables for duplicate removal. E.g., 0.10 means 10% of SD (optional, default: 0.10)

nds

integer. Number of synthetic datasets to generate in search mode. Ignored in fixed seed mode (optional, default: 1)

tg_corrdif

numeric. Target maximum absolute correlation difference threshold for dataset selection in search mode. Ignored in fixed seed mode (optional, default: 0.1)

Value

Output format depends on mode:

  • In fixed seed mode (when seed is specified), returns a single list with:

    • datagen - Synthetic dataset

    • seed - Random seed used to generate this dataset (integer or NA)

    • exact_dupl_check - Logical indicating if exact duplicates exist between original and synthetic data

    • dplot_umap - ggplot object with combined UMAP visualization comparing original and synthetic data (or NULL if diag_plots=FALSE)

    • ks_test - Tibble with Kolmogorov-Smirnov results for continuous variables (variable, formatted p.value, status, d_statistic); when ties are present, a warning is emitted that the test may be approximate (or NULL if no continuous variables)

    • jsd_res - Tibble with per-variable Jensen-Shannon divergence (JSD) for categorical variables (variable, jsd, n_levels), with weighted mean stored in attribute jsd_weighted_mean (or NULL if no categorical variables)

    • corr_diff_mean - Mean absolute difference between original and synthetic correlation matrices (numeric or NULL if no continuous variables)

    • corr_diff_max - Maximum absolute difference between original and synthetic correlation matrices (numeric or NULL if no continuous variables)

    • dplot_corr_diff - ggplot heatmap object showing correlation difference matrix (Synthetic - Original) (or NULL if diag_plots=FALSE or no continuous variables)

    • dplot_cont - List of ggplot histograms for continuous variables (or NULL if diag_plots=FALSE or no continuous variables)

    • dplot_cat - List of ggplot barplots for categorical variables (or NULL if diag_plots=FALSE or no categorical variables)

  • In search mode (when seed = NA), returns a list of such lists, one per generated dataset.

Examples

# \donttest{
library(dplyr)

# Generate example dataset
data <- data.frame(
  id = 1:150,
  ALB = rnorm(150, mean = 3.5, sd = 0.5),
  ALT = rnorm(150, mean = 50, sd = 20),
  SEX = factor(sample(c("M", "F"), 150, replace = TRUE)),
  RACE = factor(sample(c("White", "Black", "Asian", "Other"), 150, replace = TRUE))
)

# EXAMPLE 1: Fixed seed mode - generate single dataset with specified seed
output_fixed <- sg_vpop_est(data_i = data,
                            id_col = "id",
                            seed = 123,        # Fixed seed mode
                            seed_umap = 40,
                            diag_plots = TRUE)
#> Error: cannot open file '/tmp/RtmpLedq7K/temp_libpath8b6abff614d/SimuRg/R/SimuRg.rdb': No such file or directory

# Access the dataset and its diagnostics
print(head(output_fixed[[1]]$datagen, 10))
#> Error: object 'output_fixed' not found
print(output_fixed[[1]]$ks_test)
#> Error: object 'output_fixed' not found
print(output_fixed[[1]]$seed)           # Will be 123
#> Error: object 'output_fixed' not found
print(output_fixed[[1]]$corr_diff_max)
#> Error: object 'output_fixed' not found
print(output_fixed[[1]]$dplot_umap)
#> Error: object 'output_fixed' not found

# EXAMPLE 2: Search mode - generate 3 datasets meeting correlation threshold
output_search <- sg_vpop_est(data_i = data,
                             id_col = "id",
                             seed = NA,        # Search mode (default)
                             seed_umap = 40,
                             diag_plots = TRUE,
                             nds = 3,          # Generate 3 datasets
                             tg_corrdif = 0.1) # Target correlation difference
#> Warning: restarting interrupted promise evaluation
#> Error: cannot open file '/tmp/RtmpLedq7K/temp_libpath8b6abff614d/SimuRg/R/SimuRg.rdb': No such file or directory

# Compare metrics across all datasets
sapply(output_search, function(x) x$corr_diff_max)
#> Error: object 'output_search' not found
sapply(output_search, function(x) x$jsd_res)
#> Error: object 'output_search' not found
sapply(output_search, function(x) x$seed)  # Different seeds for each dataset
#> Error: object 'output_search' not found

# }