30 October 2017 - Predictive validation (also known as fourth-order validation) involves the comparison between model outputs and observed data collected after the initial analysis of a model.
Given that the data might be collected years after the development of the model, this type of model performance evaluation is the rarest form of validation applied to cost-effectiveness models. In a review of model performance evaluation across 81 published cardiovascular cost-effectiveness models, predictive validity was not reported in any of the reviewed papers.
This is not a surprising finding—funding for the development and analysis of cost-effectiveness models and for the collection of prospective data rarely cover the period beyond the primary analysis of a model. However, predictive validation appears to be in short supply even when models and funding decisions are re-visited. NICE has re-appraised a range of technologies between 3 and 8 years after the publication of original guidance. The updated guidance for these appraisals makes no reference to the predictive validation of the original models.