Clinical Trial Duration Forecasting: A Validation-First Case Study
What time-based validation reveals about oncology trial duration forecasting using registry data
🚀 Live App:
Clinical Trial Duration Forecaster
💻 Source Code:
GitHub repository
On a random split, registry fields explained about a quarter of the variation in trial duration. Tested on later trials, they explained almost none—and that finding became the point.
Earlier versions of this post reported a random-split R² of 0.84 and a time-based R² of 0.04, and attributed the gap to temporal drift. Both figures were wrong. A date-parsing bug (lubridate::ymd() read month-only registry dates such as “2005-08” as 2020-05-08) mis-dated 609 trials, dropped 975 others, and produced the spurious 0.84. A follow-up check also found 74 Phase 2/3 trials counted twice. The data have been rebuilt and the models retrained; the numbers below are the corrected ones. The conclusion—the registry features and models tested here did not support reliable forecasts for later trials—still holds. The size of the random-split result does not.
1. Framing the problem
Forecasting clinical trial timelines is an appealing problem in pharmaceutical analytics. Timelines influence portfolio planning, resourcing, and cross-functional expectations across R&D organizations. At the same time, experienced trial teams recognize that duration is shaped by factors that are difficult to observe early—protocol complexity, enrollment dynamics, site performance, and evolving standards of care.
This project approaches trial duration forecasting as a validation problem first, rather than an optimization exercise. The objective is not to produce an operational forecasting tool, but to evaluate what level of prospective signal is realistically supported by publicly available registry data.
2. What question this analysis tests
The core question is intentionally narrow:
Can ClinicalTrials.gov registry features support prospective forecasting of oncology trial duration?
To answer this, I built an end-to-end modeling pipeline and Shiny dashboard using completed oncology treatment trials. The target variable is registry-defined duration (Start Date → Completion Date), which may include extended follow-up and should not be interpreted as a direct proxy for last-patient–last-visit (LPLV); predictors are limited to registry fields (phase, sponsor type, enrollment, sites, countries, and registration-to-start lag). Several of these are final reported values rather than what was known at study start, which, if anything, flatters the model. That includes registration-to-start lag: 55% of modeled trials were first registered after they started, so their lag was not known at study start either. A check shows how much this matters: refitting with phase and sponsor only, baseline characteristics nominally fixed at study start, drops R² from 0.24 to 0.09 on a random split and from 0.03 to −0.09 on later trials, worse than predicting the average. Most of the signal comes from inputs known only in hindsight.
The emphasis is on aligning validation strategy with intended use, rather than maximizing retrospective fit.
3. What a random split shows
A conventional random train/test split shows a modest retrospective signal:
- R² ≈ 0.24 (mean of five random splits)
- Median absolute error ≈ 21 months
That is modest, but it suggests registry fields carry some signal. However, random splits mix trials from the same years into training and test sets, so era-specific patterns are shared between them. The split measures how well the model describes history, not how well it forecasts new trials. For a forward-looking use case, this validation strategy is inappropriate.
At this stage, the model was answering the wrong question.
4. What changes under time-based validation
When the validation strategy was changed to resemble the intended use—training on trials started by 2015 (n = 2,079) and testing on trials started 2016–2018 (n = 378)—almost all of that signal disappeared. The trials, features, recipe and tuning grid are the same in every row; only the split, or the test trials it is scored on, changes:
| Validation Approach | Test R² | Median Absolute Error |
|---|---|---|
| Random split | 0.24 | 21 months |
| Random split, test trials ≤ 112 months | 0.13 | 18 months |
| Time-based split | 0.03 | 15 months |
R² here is 1 − SSE/SST on log duration. The middle row scores the random split only on trials that fit the same window as the time-based test set (explained below). The squared correlation between predicted and actual is 0.24, 0.16 and 0.11 for the three rows.
Three things explain the result:
- Registry feature limitations: Key drivers of duration—protocol complexity, endpoint strategy, enrollment velocity, and operational execution—are largely absent from ClinicalTrials.gov. What signal the registry fields do carry is weak and partly specific to the era the model was trained on.
- Right-truncation of the test window: Only completed trials are included, so a trial started in 2016–2018 had to finish by the January 2026 data pull (the longest ran 112 months), while earlier trials could run up to 15 years. Long trials are missing, so the test set runs shorter (on average about 13% shorter than predicted) and slightly narrower than the training data, and both lower R². Scoring the random split on the same window cuts its R² from 0.24 to 0.13. As a sensitivity check, that suggests the window accounts for a large part of the gap (40–80% across five splits). It illustrates the window’s effect rather than decomposing the gap: the comparisons also differ in cohort, training data and follow-up, and one 112-month cap is not each trial’s own follow-up limit.
- Later trials only look shorter: among completed trials, median duration is 64 months for trials started by 2015 and 52 months for 2016–2018, but among the earlier trials no longer than 112 months the median is 56. A censoring-aware check (below) points to the truncation rather than a change in practice. It also explains a puzzle in the table: errors in months get smaller as R² falls, because the missing long trials are the ones a model misses by the most months. Neither metric is immune to the window; the middle row is the fairer comparison.
A censoring-aware check
Dropping unfinished trials is the root of the truncation problem, so I pulled every Phase 2/3 cancer treatment trial started 1990–2018, of any status, and let unfinished trials enter as censored observations (20,562 trials after exclusions). Counting the trials still running, the Kaplan–Meier median duration is 68 months for 1990–2015 starts and 73 months for 2016–2018 starts. On this measure, later trials are not shorter.
That check censors terminated trials at their stop date, which assumes termination says nothing about when a trial would have finished. Two alternatives give the same direction. Counting termination as the end of the study, the medians are 58 and 60 months. Treating termination as a competing event, half of the trials have completed by 76 and 84 months. Later trials are not shorter under any of the three.
With phase and sponsor only, trained on 1990–2015 starts and tested on 2016–2018 starts, a Weibull survival model predicts a median of 76 months against an actual 73; the same features fitted to completed trials only predict 60. Ranking ability is equally weak for both (concordance 0.57). Handling censoring removes most of a 13-month bias in the overall median, but matching a median is aggregate agreement, not individual calibration: these registry features do not say which trials will run long.
Against simple benchmarks
Does the model beat rules a planner could apply by hand? On the time-based split, the median absolute error is 15.2 months for the model, against 18.6 for predicting the training average, 19.8 for the phase median and 19.3 for phase-by-sponsor medians. The model wins by a few months, so the registry fields carry some signal. Much of that margin comes from final enrollment, sites and countries, which are known only in hindsight: with phase and sponsor alone, the model’s median error (18.3 months) is about the same as the constant’s.
What the time-based split is, and is not
The time-based split measures whether patterns from earlier start-year cohorts carry over to later ones, using records retrieved in 2026. It is not a historical backtest: a model built in 2016 could not have known the eventual end dates of every trial started before then, and registry fields are updated after the fact. A stricter test would limit outcomes and predictors to what the registry showed at each forecast date, which needs archived snapshots I do not have.
Under these conditions, a low R² on later trials is not a modeling failure. The time-based number is, if anything, a pessimistic estimate because of the truncated window, but even the most generous comparison leaves registry fields explaining a small fraction of the variation.
The result is also robust to how the cohort was defined. The oncology cohort comes from a keyword-based registry query, and about 8% of modeled trials (203 of 2,457) list no cancer term in their conditions field. Refitting without them moves test R² from 0.030 to 0.018 and leaves the conclusions unchanged.
5. Why this matters in pharma analytics
In pharmaceutical decision-making, forecasts are used prospectively: to inform planning, resource allocation, and expectations. Validation strategies that mix past and future data can materially overstate confidence and lead to incorrect conclusions.
This case study demonstrates that how a model is validated matters more than which algorithm is chosen. All four algorithms tried were statistically tied in cross-validation. A model that describes history modestly but fails under time-based testing may still be useful diagnostically—but it should not be positioned as a forecasting tool.
From an analytics governance perspective, this distinction is critical.
6. What this project does—and does not—claim
This project demonstrates:
- How to design validation strategies that align with prospective use
- How the choice of split changes the answer, and why right-truncation must be accounted for when comparing eras
- How registry-only features constrain forecasting power
It explicitly does not claim:
- That oncology trial duration is predictable from registry data alone
- That this model should be used for operational timeline planning
- That algorithmic complexity would resolve the observed limitations
The results are conditional on the data sources and features used.
7. What would be required for planning-grade forecasting
If the objective were operational forecasting, additional data would be required, including:
- Protocol complexity and amendment history
- Endpoint structure and follow-up requirements
- Enrollment trajectory and screen-failure dynamics
- Site and geography-level performance history
- Competitive and standard-of-care context
These factors are typically available only through internal systems, not public registries.
Dashboard Preview
The dashboard is structured to make validation behavior visible and to discourage overinterpretation of point predictions.
Executive Brief Surfaces key metrics (R², MAE, sample size) with appropriate caveats about prospective use limitations. 
Model Sandbox Shows how the fitted model responds to trial characteristics, positioned as educational, not operational. Each estimate comes with the historical test-error range: how far 2016–2018 test trials landed from their estimates, pooled across all test trials. Its coverage for a new trial has not been established, and the upper end is likely too low, but it makes the model’s uncertainty visible. 
Feature Explorer Visualizes Elastic Net coefficients and feature distributions to understand which registry variables the model weighted. 
Model Performance Diagnostic plots comparing actual vs. predicted duration, with error breakdown by phase and sponsor type. 
Methods & Data Documents the data pipeline, inclusion criteria, and validation methodology—including the key lesson on why time-based validation matters. 
Appendix: Methodology & Build Notes
Analytical Pipeline
- Data acquisition – ClinicalTrials.gov API (oncology interventional trials)
- Cohort filtering – Treatment purpose, Phase 1/2 through Phase 3, completed studies
- Feature engineering – Log transforms, ratios, categorical encoding
- Validation design – Time-based split by start year (cross-cohort transfer; not a historical backtest)
- Modeling – Elastic Net regression via tidymodels
- Diagnostics – Error analysis and residual inspection
Model specification
- Algorithm: Elastic Net (glmnet)
- Target: Log-transformed duration (months)
- Cross-validation: 10-fold CV within training data only
- Baseline comparisons: Linear regression, Random Forest, XGBoost
- Selection criterion: One-standard-error rule on cross-validated RMSE; all four models were within one standard error of the best, and Elastic Net was kept by a stated preference for a regularized linear model (at the small penalty chosen, it is nearly identical to linear regression). The 2016–2018 test set was not used for model selection
- Random vs time-based comparison: Same cohort, recipe and grid; five random splits, each also scored on a truncation-matched subset, vs the time-based split
- Robustness check: Refit excluding trials with no oncology term in their conditions field (test R² 0.030 → 0.018)
Technical stack
- Language: R
- Framework: Shiny (modular architecture)
- UI: shiny.semantic (Appsilon)
- Modeling: tidymodels
- Visualization: ggplot2, ggiraph, reactable
- Deployment: shinyapps.io
References
- ClinicalTrials.gov – U.S. National Library of Medicine
- Tidymodels documentation – https://www.tidymodels.org/
- Appsilon shiny.semantic – https://appsilon.github.io/shiny.semantic/
- Wong, C.H., Siah, K.W., & Lo, A.W. (2019). Estimation of clinical trial success rates and related parameters. Biostatistics, 20(2), 273–286.
Closing reflection
The most important outcome of this project is not a performance metric, but a methodological lesson: honest validation protects decision-makers from false confidence.
In trial analytics, recognizing the limits of available data is as important as extracting signal from it.
Updated October 2026: corrected a date-parsing bug that invalidated the original headline numbers (see the correction note at the top); removed duplicate Phase 2/3 records; rebuilt the data, retrained the models, and replaced the temporal-drift explanation with the right-truncation analysis, including a truncation-matched random split. Also added the cohort robustness check, the random-vs-time-based comparison, the pre-start-features check, a censoring-aware survival check on a full all-status pull, and the forecaster’s likely range. Later in October: narrowed the conclusions to the features and models tested, added simple benchmarks, a check on how terminated trials are treated, and regression checks; relabeled the forecaster as a model sandbox. After a second review: dropped registration lag from the reduced-feature and survival checks, since most trials were registered after they started.
Citation
@online{ponce2026,
author = {Ponce, Steven},
title = {Clinical {Trial} {Duration} {Forecasting:} {A}
{Validation-First} {Case} {Study}},
date = {2026-01-19},
url = {https://stevenponce.netlify.app/projects/standalone_visualizations/sa_2026-01-19.html},
langid = {en}
}