Bayesian Sample Size and Precision Considerations for Risk Prediction Models

Performs Bayesian sample size, precision, and value-of-information analysis for external validation of existing multi-variable prediction models using the approach proposed by Sadatsafavi and colleagues (2026) .

Bayespmtools

The goal of Bayespmtools is to enable Bayesian sample size and precision calculations for external validation of risk prediction models.

For the details of the methodology, please refer to the accompanying paper: https://doi.org/10.1002/sim.70389

#Specify evidence:
evidence <- list(
  prev ~ beta(116, 155),           # Outcome prevalence
  cstat ~ beta(3628, 1139),        # C-statistic
  cal_mean ~ norm(-0.009, 0.125),  # Mean calibration error
  cal_slp ~ norm(0.995, 0.024)     # Calibration slope
)

#Specifying targets
#eciw=x indicates desired expected CI Width of x.
#qciw=c(a,b) indicates desired assurance CI Width of b at assurance level a.
#voi.nb indicates targeting a given EVSI/EVPI ratio
targets <- list(
  eciw.cstat = 0.1,             # Expected CI width for c-statistic
  eciw.cal_oe = 0.22,           # Expected CI width for O/E ratio
  eciw.cal_slp = 0.30,          # Expected CI width for calibration slope
  qciw.cal_slp = c(0.90, 0.35), # Quantile of CI width for calibration slope (assurance)
  voi.nb = 0.90                 # EVSI/EVPI ratio
)

library(bayespmtools)

#Main function call
res <- bpm_valsamp(
  evidence = evidence,
  targets = targets,
  n_sim = 1000,           # Number of Monte Carlo simulations
  threshold = 0.2         # Risk threshold for net benefit calculations
)
#> Processing evidence...
#> Generating Monte Carlo sample...
#> Imputing correlation, based on effective sample size: 272 ...
#> Inferring calibration intercept...
#> Computing CI sample size...
#> Computing se/sp...
#> VoI / NB assurance...

print(res$results)
#>   eciw.cstat  eciw.cal_oe eciw.cal_slp qciw.cal_slp       voi.nb 
#>          361          435         1060          879          644