How and When to Use a Calibrated Classification Model with scikit-learn

Last Updated on September 25, 2019

Instead of predicting class values directly for a classification problem, it can be convenient to predict the probability of an observation belonging to each possible class.

Predicting probabilities allows some flexibility including deciding how to interpret the probabilities, presenting predictions with uncertainty, and providing more nuanced ways to evaluate the skill of the model.

Predicted probabilities that match the expected distribution of probabilities for each class are referred to as calibrated. The problem is, not all machine learning models are capable of predicting calibrated probabilities.

There are methods to both diagnose how calibrated predicted probabilities are and to better calibrate the predicted probabilities with the observed distribution of each class. Often, this can lead to better quality predictions, depending on how the skill of the model is evaluated.

In this tutorial, you will discover the importance of calibrating predicted probabilities and how to diagnose and improve the calibration of models used for probabilistic classification.

After completing this tutorial, you will know:

• Nonlinear machine learning algorithms often predict uncalibrated class probabilities.
• Reliability diagrams can be used to diagnose the calibration of a model, and methods can be used to better calibrate predictions for
  To finish reading, please visit source site