Validation of a calibrated prediction model for response to growth hormone treatment in an independent cohort

BACKGROUND: Prediction models, e.g. for prediction of response to growth hormone treatment, need validation in appropriate independent cohorts, comparing predicted and observed outcomes. In a previous validation of a model for predicting the first-year response to growth hormone treatment in children with idiopathic growth hormone deficiency, overfitting was observed. We modified the prediction formula and now report validation of this modified model. PATIENTS AND METHODS: The modified and original prediction models were applied to a group of patients selected from Lilly's GeNeSIS database using the same inclusion and exclusion criteria as for the original model. For both prediction methods, observed first-year height velocity was plotted vs. predicted height velocity in a calibration plot. For a valid prediction, the regression line should correspond to the line of identity (observed outcome is equal to predicted outcome); the regression lines for each prediction model were tested for significant differences from this line of identity. RESULTS: The number of patients fulfilling the criteria was 226. The regression line in the calibration plot of the modified model was not significantly different from the line of identity (p = 0.43), in contrast to the original model (p < 0.001). For the modified model the mean (SD) prediction error was -0.11 (2.05) cm/year and for the original model 0.28 (2.11) cm/year. CONCLUSION: The modified prediction method, obtained after calibration of the original model, performs well in an independent patient sample and gives more accurate predictions than the original model.