Hybrid ensemble machine learning models

Machine learning models are usually assessed and compared in terms of predictive performance. Ensemble models, which average the predictions obtained from different models, often improve such performance. In this paper we show how to further improve the predictive accuracy of ensemble models, and allow them to achieve strong performance without retraining. To this aim we leverage the diversity among individual models, expressed by their covariance, computed on a subsample of the data ordered by the best model. We illustrate our proposal with applications to real data.