Prediction models are research-based tools that can play a vital role in disease management. From predicting disease development to patient prognosis, these models are remarkable tools to translate scientific literature into medical practice. The main goal of prediction models is to risk-stratify patients to support tailored clinical decision-making and enhance patient outcomes. In addition to supporting clinical decision making, risk prediction models help facilitate the allocation of scarce resources.
Recent years have seen the publication of several medical prediction models, but the implementation of such models in clinical practice is easier said than done. Effective implementation mandates the creation, validation, and integration of functional risk calculators.
To facilitate these steps altogether, a team of researchers from the Netherlands developed “Evidencio”, an online prediction platform.
All About the “Evidencio” Platform
“Evidencio” is a centralized cloud-based platform accommodating over 1000 functional risk calculators hosted in a public library. It is a platform for clinical prediction models and calculators in which the users can add models based on their own (institutional) data.
Through Evidencio, a cardiovascular algorithm ecosystem can be created to facilitate the creation, validation, integration as well as the certification process of risk calculators for clinical applications.
“Evidencio”- CV algorithm ecosystem incorporating creation, validation, integration and certification of risk calculators for clinical application.
Standardized Creation of Functional Risk Calculators
The past few years have witnessed a steep rise in the number of cardiovascular algorithms being published. Though encouraging, this raises concerns about scalability and efficiency. Additionally, the variation in the reporting of prediction models can hamper quality, transparency ultimately impeding the integration in the clinical workflow. Standardized creation of functional risk calculators is a potential solution to this challenge.
In the “Evidencio” Platform, researchers can use the online tools to develop, host and share fully functional risk calculators within minutes and free of cost. The platform supports a large variety of model types including point scores, logistic regression models, survival functions, and also machine learning algorithms.
The platform offers a user-friendly interface, which is automatically created on completion of the model creation process. Once the relevant clinical data is added by the user, risk estimates tailored towards an individual patient are provided. In addition, context information can be added to support result interpretation.
Semi-Automated Model Validation
Though strongly suggested, external model validation of cardiovascular prediction models prior to clinical application is often not performed. This limits the usability of prediction models. A systemic review by Damen et al (2016), found that only 36% of 363 evaluated cardiovascular models were externally validated. Of these, only 19% were validated by independent investigators. As a proposed solution to this problem, an online validation module was developed.
With the online module, researchers and physicians can easily validate model performance in the target population. Local, anonymized patient data can be used. After providing the supporting model parameters and outcome data, the platform calculates relevant output based on model performance. This includes (but is not limited to) model discrimination, model calibration, and calculation of decision curves.
The online validation module described above can also be used to compare multiple cardiovascular prediction models simultaneously. This head-to-head comparison can enable the selection of the best performing prediction model as per the target population or patient subgroup.
Integration of Prediction Models in the Clinical Workflow
The prediction models hosted on the “Evidencio” platform are created in a standardized way, using standardized building blocks. Thus, the single integration of the IT infrastructure allows for direct access to hundreds of prediction models simultaneously. This allows the integration of prediction models hosted on the platform directly into the workflow of an electronic health record system or third-party application. This integration can be game-changing in facilitating the successful adoption of risk calculators in clinical practice.
In Europe, medical prediction models require “CE certification” prior to clinical application as per the Medical Device Regulations (MDR 2017/745), in which each calculator requires separate certification. Standardized creation, validation, and integration of the model on the “Evidencio” platform can help reduce the burden of this expensive certification process. It is estimated that up to 80% of the underlying code of each model on the “Evidencio” platform is interchangeable – this means that just around 20% of each model must be re-evaluated instead of 100%. This saves time and expenses.
Digital prediction models can complement human clinical decision-making
Digital innovations are seen to bridge the current gap between scientific literature and the clinical application of cardiovascular prediction models. A standardized algorithm ecosystem like “Evidencio” can support healthcare professionals by improving overall quality, transparency and patient-tailored decision making.
By enabling greater accuracy and efficiency, digital tools are set to transform healthcare!
