Making Biomedical ML Reproducible
The confluence of machine learning (ML) data-driven approaches and increased computational power, alongside access to the wealth of electronic health records (EHRs) and other emergent types of data (e.g., omics, imaging, mHealth), are accelerating the development of biomedical predictive models. Such models range from traditional statistical approaches (e.g., regression) through to more advanced deep learning techniques (e.g., convolutional neural networks, CNNs), and span different tasks (e.g., biomarker/pathway discovery, diagnostic, prognostic, etc.). Two issues have become evident: 1) as there are no comprehensive standards to support the dissemination of these models, scientific reproducibility (vs. replicability) is problematic, given challenges in interpretation and implementation; and 2) as new models are put forth, methods to assess differences in performance, as well as insights into external validity (i.e., transportability), are necessary. Tools moving beyond data sharing and model “executables” are needed, capturing the information needed to fully reproduce a model and its evaluation. The objective of this R01 is the development of <strong>PREMIERE (PREdictive Model Index and Exchange REpository)</strong>, an informatics standard supporting the requisite information for scientific reproducibility for statistical and ML-based biomedical predictive models.